Image processing device, endoscope system, and method of operating image processing device

Abstract

To provide an image processing device capable of improving the speed of determining the cancer invasion depth.SOLUTION: An image processing apparatus includes an acquisition unit that acquires a first observation image obtained by irradiating a subject with first observation light and imaging return light, and a calculation unit that estimates the cancer invasiveness based on a pixel value of each pixel of the first observation image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an endoscope system, and a method for operating an image processing device. [Background technology]

[0002] Conventionally, in endoscopic observation, a subject may be irradiated with red light (see, for example, Patent Document 1). Also, an operator such as a doctor may use endoscopic observation to determine the depth of cancer invasion. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-036326 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when determining the depth of cancer invasion using endoscopic observation, depending on the experience of the surgeon, it may take time to determine the depth of cancer invasion or it may be difficult to make the determination.

[0005] The present invention has been made in consideration of the above, and aims to provide an image processing device, an endoscope system, and an operating method of an image processing device that can improve the speed at which the depth of cancer is determined. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an image processing device according to one aspect of the present invention includes an acquisition unit that acquires a first observation image by irradiating a subject with first observation light and capturing the returned light, and a calculation unit that estimates the depth of cancer invasion based on the pixel value of each pixel of the first observation image.

[0007] In the image processing device according to one aspect of the present invention, the first observation light is broadband light having wavelength components in a wide band, and the first observation image is a spectral image for each wavelength.

[0008] In addition, an image processing device according to one aspect of the present invention includes an extraction unit that extracts the pixel values ​​of each pixel in a predetermined wavelength band in the spectroscopic image, and the calculation unit estimates the depth of invasion of the cancer based on the slope of a spectroscopic spectrum generated from the pixel values ​​extracted by the extraction unit.

[0009] In the image processing device according to one aspect of the present invention, the predetermined wavelength band is included in the wavelength range from 600 nm to 700 nm.

[0010] In an image processing device according to an aspect of the present invention, the lower limit of the predetermined wavelength band is a wavelength at which a difference occurs between the change in the absorption coefficient of deoxygenated hemoglobin and the change in the absorption coefficient of oxygenated hemoglobin for each wavelength.

[0011] Furthermore, an image processing device according to one aspect of the present invention includes an extraction unit that extracts pixel values ​​of two wavelength bands of each pixel in the spectroscopic image, and the calculation unit estimates the depth of invasion of the cancer based on the pixel values ​​extracted by the extraction unit.

[0012] In the image processing device according to one aspect of the present invention, the two wavelength bands are included in the wavelength range from 600 nm to 700 nm.

[0013] In an image processing device according to an aspect of the present invention, the lower limits of the two wavelength bands are wavelengths at which there is a difference in change between the absorption coefficient of deoxygenated hemoglobin and the absorption coefficient of oxygenated hemoglobin for each wavelength.

[0014] In addition, in an image processing device according to one aspect of the present invention, the acquisition unit acquires a second observation image by irradiating the subject with second observation light and capturing returned light, wherein the first observation light is light having a peak wavelength near 630 nm, and the second observation light is light having a peak wavelength near 600 nm.

[0015] In the image processing device according to one aspect of the present invention, the calculation unit estimates the depth of invasion of cancer based on the pixel value of each pixel in the first observational image and the pixel value of each pixel in the second observational image.

[0016] In the image processing device according to one aspect of the present invention, the calculation unit normalizes the pixel value of each pixel in the first observational image by the pixel value of each pixel in the second observational image.

[0017] In the image processing device according to one aspect of the present invention, the calculation unit estimates the depth of invasion of cancer based on the pixel value of each pixel in the first observational image.

[0018] In addition, in an image processing device according to one aspect of the present invention, the acquisition unit acquires a normal light image by irradiating the subject with white light and capturing the returned light, and the calculation unit estimates the depth of cancer invasion based on the pixel value of each pixel of the first observational image and the pixel value of each pixel of the normal light image.

[0019] The image processing device according to one aspect of the present invention further includes a display control unit that causes a display unit to display the depth of invasion of the cancer for each pixel.

[0020] In the image processing device according to one aspect of the present invention, the display control unit causes the display unit to display the first observational image or the normal light image with the depth of invasion of the cancer for each pixel superimposed thereon.

[0021] The image processing device according to one aspect of the present invention further includes a determination unit that determines whether or not a region is highly likely to be cancerous based on the depth of invasion of the cancer in each pixel.

[0022] An endoscopic system according to one aspect of the present invention includes an image processing device having a light source unit that irradiates a subject with first observation light from a tip of an insertion unit that is inserted into the subject, an imaging unit that captures returned light of the first observation light to generate a first observation image, an acquisition unit that acquires the first observation image from the imaging unit, and a calculation unit that estimates the depth of cancer invasion based on pixel values ​​of each pixel of the first observation image.

[0023] In addition, in an endoscopic system according to one aspect of the present invention, the first observation light is broadband light having a wide range of wavelength components, the first observation image is a spectral image for each wavelength, the image processing device has an extraction unit that extracts the pixel values ​​of each pixel in a predetermined wavelength band in the spectral image, and the calculation unit estimates the depth of invasion of the cancer based on a slope of a spectral spectrum generated from the pixel values ​​extracted by the extraction unit.

[0024] In addition, in an endoscopic system according to one aspect of the present invention, the light source unit irradiates the subject with second observation light from the tip of the insertion unit, the imaging unit captures returned light of the second observation light to generate a second observation image, the acquisition unit acquires the first observation image and the second observation image from the imaging unit, and the calculation unit estimates the depth of invasion of cancer based on the pixel value of each pixel of the first observation image and the pixel value of each pixel of the second observation image.

[0025] In addition, a method for operating an image processing device according to one aspect of the present invention includes an acquisition unit that acquires a first observation image by irradiating a subject with first observation light and capturing the returned light, and a calculation unit that estimates the depth of cancer invasion based on the pixel value of each pixel of the first observation image.

[0026] In addition, in a method for operating an image processing device according to one aspect of the present invention, the first observation light is broadband light having a wide range of wavelength components, the first observation image is a spectral image for each wavelength, an extraction unit extracts the pixel values ​​of each pixel in a predetermined wavelength band in the spectral image, and the calculation unit estimates the depth of invasion of the cancer based on a slope of a spectral spectrum generated from the pixel values ​​extracted by the extraction unit.

[0027] In addition, in a method for operating an image processing device according to one aspect of the present invention, the acquisition unit acquires a second observation image by irradiating the subject with second observation light and capturing returned light, and the calculation unit estimates the depth of invasion of cancer based on a pixel value of each pixel in the first observation image and a pixel value of each pixel in the second observation image. [Effects of the Invention]

[0028] According to the present invention, it is possible to realize an image processing device, an endoscope system, and an operation method of an image processing device that can improve the speed at which the depth of cancer invasion can be determined. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram showing the overall configuration of an endoscope system including an image processing device according to embodiment 1-1. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the main parts of the endoscope and the control device according to embodiment 1-1. [Figure 3] FIG. 3 is a diagram showing the spectral reflectance of each tissue. [Figure 4] FIG. 4 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 1-1. [Figure 5] FIG. 5 is a diagram showing the absorption spectrum of hemoglobin. [Figure 6] FIG. 6 is a diagram showing how the spectral reflectances of cancer and non-cancer are approximated by a linear expression. [Figure 7] FIG. 7 is a diagram showing an example of an invasion depth map. [Figure 8] FIG. 8 is a diagram showing a modified example of the invasion depth map. [Figure 9] FIG. 9 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 1-2. [Figure 10] FIG. 10 is a diagram showing how the spectral reflectances of cancer and non-cancerous tissues are approximated by two linear expressions. [Figure 11] FIG. 11 is a diagram showing how the spectral reflectances of cancer and non-cancer are approximated by a linear expression. [Figure 12] FIG. 12 is a block diagram showing the functional configuration of the main parts of the endoscope and the control device according to embodiment 2-1. [Figure 13] FIG. 13 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 2-1. [Figure 14]FIG. 14 is a diagram showing the difference in pixel value between the R signal and the Am signal. [Figure 15] FIG. 15 is a diagram showing how pixel values ​​of R signals and Am signals are corrected. [Figure 16] FIG. 16 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 2-2. [Figure 17] FIG. 17 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 2-3. [Figure 18] FIG. 18 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 2-4. [Figure 19] FIG. 19 is a block diagram showing the functional configuration of the main parts of the endoscope and the control device according to embodiment 3-1. [Figure 20] FIG. 20 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 3-1. [Figure 21] FIG. 21 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 3-2. [Figure 22] FIG. 22 is a block diagram showing the functional configuration of the main parts of the endoscope and the control device according to embodiment 4-1. [Figure 23] FIG. 23 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 4-1. [Figure 24] FIG. 24 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 4-2. [Figure 25] FIG. 25 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 4-3. [Figure 26] FIG. 26 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 4-4. [Figure 27] FIG. 27 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 5-1. [Figure 28]FIG. 28 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 5-2. [Figure 29] FIG. 29 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 6-1. [Figure 30] FIG. 30 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 6-2. [Figure 31] FIG. 31 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 6-3. [Figure 32] FIG. 32 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 6-4. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, embodiments of an image processing device, an endoscope system, and an operating method of an image processing device according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments. The present invention can be generally applied to image processing devices, endoscope systems, and operating methods of image processing devices used to determine the depth of cancer invasion.

[0031] In addition, in the drawings, the same or corresponding elements are appropriately designated by the same reference numerals. It should be noted that the drawings are schematic, and the dimensional relationships and ratios of each element may differ from the actual situation. The dimensional relationships and ratios may also differ between the drawings.

[0032] (Embodiment 1-1) [Overall configuration of endoscope system] 1 is a diagram showing the overall configuration of an endoscope system including an image processing device according to embodiment 1-1. The endoscope system 1 is used, for example, in the medical field, and is a system for observing the inside of a subject (inside a living body). The endoscope system 1 includes an endoscope 2, a display device 3, and a control device 4.

[0033] The endoscope 2 continuously generates image data (RAW data) by capturing images of the inside of the subject, and sequentially outputs this image data to the control device 4. As shown in FIG. 1, the endoscope 2 includes an insertion section 21, an operation section 22, and a universal cord 23.

[0034] 1, the insertion section 21 includes a distal end portion 24 provided at the distal end of the insertion section 21, a bending portion 25 connected to the proximal end side (the operation section 22 side) of the distal end portion 24 and configured to be bendable, and a long flexible tube portion 26 connected to the proximal end side of the bending portion 25 and having flexibility.

[0035] The operation unit 22 is connected to the base end portion of the insertion unit 21. The operation unit 22 receives various operations on the endoscope 2. As shown in FIG. 1 , the operation unit 22 is provided with a bending knob 221, an insertion port 222, and a plurality of operation members 223.

[0036] The bending knob 221 is configured to be rotatable in response to a user operation by a user such as an operator. The bending knob 221, when rotated, operates a bending mechanism (not shown) such as a metal or resin wire disposed inside the insertion section 21. As a result, the bending section 25 is bent.

[0037] The insertion port 222 is connected to a treatment instrument channel (not shown) which is a duct extending from the tip of the insertion portion 21, and is an insertion port for inserting a treatment instrument or the like from the outside of the endoscope 2 into the treatment instrument channel.

[0038] The multiple operating members 223 are composed of buttons and the like that accept various operations by users such as surgeons, and output operation signals corresponding to the various operations to the control device 4 via the universal cord 23. Examples of the various operations include a release operation that instructs the endoscope 2 to capture a still image, and an operation that switches the observation mode of the endoscope 2 to a normal light observation mode or a special observation mode.

[0039] The universal cord 23 extends from the operation unit 22 in a direction different from the extending direction of the insertion unit 21, and is a cord on which a light guide 231 (see FIG. 2) made of an optical fiber or the like, a first signal line 232 (see FIG. 2) for transmitting the above-mentioned image data, a second signal line 233 (see FIG. 2) for transmitting the above-mentioned operation signal, etc. are arranged. A connector unit 27 is provided at the base end of the universal cord 23, as shown in FIG. 1. The connector unit 27 is detachably connected to the control device 4.

[0040] The display device 3 is composed of a display monitor such as a liquid crystal or organic EL (Electro Luminescence) display monitor, and under the control of the control device 4, displays a display image based on image data that has been image processed in the control device 4, as well as various information related to the endoscope 2.

[0041] The control device 4 is realized using a processor, which is a processing device having hardware such as a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), or CPU (Central Processing Unit), and a memory, which is a temporary storage area used by the processor. The control device 4 comprehensively controls the operations of each part of the endoscope 2 according to a program recorded in the memory.

[0042] [Functional configuration of the main parts of the endoscope system] Next, a description will be given of the functional configuration of the main parts of the above-mentioned endoscope system 1. Fig. 2 is a block diagram showing the functional configuration of the main parts of the endoscope and control device according to embodiment 1-1. The following description will be given in the order of the endoscope 2 and the control device 4.

[0043] [Functional configuration of the endoscope] First, a description will be given of the functional configuration of the endoscope 2. As shown in Fig. 2, the endoscope 2 includes an illumination optical system 201, an imaging optical system 202, an imaging element 203 as an imaging unit, an A / D conversion unit 204, a P / S conversion unit 205, an imaging and recording unit 206, and an imaging control unit 207. Here, the illumination optical system 201, the imaging optical system 202, the imaging element 203, the A / D conversion unit 204, the P / S conversion unit 205, the imaging and recording unit 206, and the imaging control unit 207 are each disposed within the tip portion 24.

[0044] The illumination optical system 201 is composed of one or more lenses and irradiates the subject with illumination light supplied from a light guide 231 .

[0045] The imaging optical system 202 is composed of a plurality of lenses and an actuator made of a stepping motor or a voice coil motor that moves a predetermined lens among the plurality of lenses in the optical axis direction. The imaging optical system 202 focuses light reflected from the subject, light returned from the subject, fluorescent light emitted by the subject, etc., to form an image of the subject on the light receiving surface of the image sensor 203.

[0046] The image sensor 203 is configured using an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) in which one of color filters constituting a Bayer array (RGGB) is arranged on each of a plurality of pixels arranged in a two-dimensional matrix. Under the control of the imaging control unit 207, the image sensor 203 receives light from the subject image formed by the imaging optical system 202 and performs photoelectric conversion to generate a captured image (analog signal). The image sensor 203 outputs image data to the A / D conversion unit 204.

[0047] The A / D conversion unit 204 is configured using an A / D conversion circuit, etc. Under the control of the imaging control unit 207, the A / D conversion unit 204 performs A / D conversion processing on the analog image data input from the imaging element 203 and outputs the result to the P / S conversion unit 205.

[0048] The P / S conversion unit 205 is configured using a P / S conversion circuit or the like, and performs parallel / serial conversion on the digital image data input from the A / D conversion unit 204 under the control of the imaging control unit 207, and outputs the data to the control device 4 via a first signal line 232. Note that instead of the P / S conversion unit 205, an E / O conversion unit that converts image data into an optical signal may be provided, and the image data may be output to the control device 4 via the optical signal. Alternatively, the image data may be transmitted to the control device 4 via wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0049] The imaging and recording unit 206 is configured with a non-volatile memory or a volatile memory, and records various information (for example, pixel information of the imaging element 203) related to the endoscope 2. The imaging and recording unit 206 also records various setting data and control parameters transmitted from the control device 4 via the second signal line 233.

[0050] The imaging control unit 207 is realized using a TG (Timing Generator), a processor which is a processing device having hardware such as a CPU, and a memory which is a temporary storage area used by the processor. The imaging control unit 207 controls the operations of the imaging element 203, the A / D conversion unit 204, and the P / S conversion unit 205 based on setting data received from the control device 4 via a second signal line 233.

[0051] [Functional configuration of the control device] Next, we will explain the functional configuration of the control device 4. As shown in Fig. 2, the control device 4 includes a condenser lens 401, a light source unit 402, a light source control unit 404, an S / P conversion unit 405, an image processing unit 406, an input unit 407, a recording unit 408, and a control unit 410.

[0052] The condenser lens 401 condenses the light emitted by the light source unit 402 and outputs the condensed light to the light guide 231 .

[0053] The light source unit 402 emits visible white light (normal light) under the control of the light source control unit 404, and supplies the white light as illumination light to the light guide 231. The light source unit 402 is configured using a collimator lens, a white LED (Light Emitting Diode) lamp, a driver, and the like. Note that the light source unit 402 may supply visible white light by simultaneously emitting light from a red LED lamp, a green LED lamp, and a blue LED lamp. The light source unit 402 may also be configured using a halogen lamp, a xenon lamp, or the like.

[0054] The light source control unit 404 is realized by using a processor having hardware such as an FPGA or a CPU, and a memory that is a temporary storage area used by the processor. The light source control unit 404 controls the light emission timing and light emission duration of the light source unit 402 based on control data input from the control unit 410.

[0055] Under the control of the control unit 410, the S / P conversion unit 405 performs serial / parallel conversion on image data received from the endoscope 2 via the first signal line 232 and outputs the converted data to the image processing unit 406. When the endoscope 2 outputs image data as an optical signal, an O / E conversion unit that converts an optical signal into an electrical signal may be provided instead of the S / P conversion unit 405. When the endoscope 2 transmits image data via wireless communication, a communication module that can receive wireless signals may be provided instead of the S / P conversion unit 405.

[0056] The image processing unit 406 is realized by using a processor having hardware such as a GPU or FPGA, and a memory that is a temporary storage area used by the processor. Under the control of the control unit 410, the image processing unit 406 performs predetermined image processing on the image data of parallel data input from the S / P conversion unit 405, and outputs the result to the display device 3. Examples of the predetermined image processing include demosaic processing, white balance processing, gain adjustment processing, gamma correction processing, and format conversion processing.

[0057] The input unit 407 is configured using a mouse, foot switch, keyboard, buttons, switches, a touch panel, etc., and accepts user operations by a user such as an operator, and outputs an operation signal according to the user operation to the control unit 410.

[0058] The recording unit 408 is configured using a recording medium such as a volatile memory, a non-volatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), a memory card, etc. The recording unit 408 records data including various parameters necessary for the operation of the control device 4 and the endoscope 2. The recording unit 408 also has a program recording unit 408a that records various programs for operating the endoscope 2 and the control device 4, and an image data recording unit 408b that records an image file that stores an image corresponding to the image data.

[0059] Under the control of the control unit 410, the image data recording unit 408b records a group of image data generated by the endoscope 2 successively capturing images of a plurality of observation sites on the subject in association with patient information and the like.

[0060] The control unit 410 corresponds to the image processing device according to the present disclosure. The control unit 410 is realized using a processor having hardware such as an FPGA or a CPU, and a memory that is a temporary storage area used by the processor. The control unit 410 comprehensively controls the components that make up the endoscope 2 and the control device 4. The control unit 410 has an acquisition unit 410a, an extraction unit 410b, a calculation unit 410c, and a display control unit 410d.

[0061] The acquisition unit 410a acquires a first observation image by irradiating a subject with a first observation light and capturing the returned light. In embodiment 1-1, the first observation light is white light irradiated by the light source unit 402, and is broadband light having a wide range of wavelength components. The first observation image is a spectral image for each wavelength.

[0062] The extraction unit 410b extracts pixel values ​​of each pixel in a predetermined wavelength band from the spectral image. The predetermined wavelength band is, for example, 600 nm to 700 nm, but may be any wavelength band included between 600 nm and 700 nm. The lower limit of the predetermined wavelength band is preferably a wavelength at which there is a difference in the change between the absorption coefficient of deoxygenated hemoglobin and the absorption coefficient of oxygenated hemoglobin for each wavelength.

[0063] The calculation unit 410c estimates the depth of invasion of the cancer based on the pixel value of each pixel in the first observational image. Specifically, the calculation unit 410c estimates the depth of invasion of the cancer based on the slope of the spectrum generated from the pixel values ​​extracted by the extraction unit 410b.

[0064] FIG. 3 is a diagram showing the spectral reflectance of each tissue. The horizontal axis of FIG. 3 represents wavelength, and the vertical axis represents spectral reflectance. Lines L1 to L4 in FIG. 3 represent the spectral reflectance of tissues with invasion depths of non-cancerous, M, SM, and MP, respectively. The invasion depth increases in the order MP>SM>M>non-cancerous, and the slope of the spectral spectrum increases in this order in the wavelength range of 600 to 700 nm. Using this characteristic, the calculation unit 410c estimates the invasion depth.

[0065] The display controller 410d causes the display unit to display the depth of cancer invasion for each pixel, and the first observational image and the normal light image, with the depth of cancer invasion for each pixel superimposed on them.

[0066] [Processing by image processing device] Next, an outline of the processing executed by the image processing device will be described below. Fig. 4 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 1-1.

[0067] As shown in FIG. 4, first, the acquiring unit 410a acquires a spectral image of a predetermined wavelength band (wavelengths from 600 nm to 700 nm) extracted by the extracting unit 410b (step S101).

[0068] Next, the calculation unit 410c interpolates the pixel values ​​of the saturated pixels using the pixel values ​​of the surrounding pixels (step S102). A saturated pixel is a pixel whose signal level is saturated. This preprocessing reduces the variation between pixels.

[0069] Thereafter, the calculation unit 410c approximates the spectrum for each pixel to a linear expression (step S103). Then, the calculation unit 410c uses the gradient of this linear expression as an index for estimating the invasion depth.

[0070] FIG. 5 is a diagram showing the absorption spectrum of hemoglobin. The horizontal axis of FIG. 5 represents wavelength, and the vertical axis represents absorption coefficient. Line L11 in FIG. 5 represents the absorption coefficient of deoxygenated hemoglobin, and line L12 represents the absorption coefficient of oxygenated hemoglobin. As shown in FIG. 5, the absorption coefficients of oxygenated and deoxygenated hemoglobin begin to differ in change from a wavelength of approximately 580 nm, where line L11 and line L12 intersect, and the difference in change is large at wavelengths of 600 nm to 700 nm, where the effects of changes in blood volume due to cancer are easily apparent.

[0071] FIG. 6 is a diagram showing how the spectral reflectances of cancer and non-cancerous tissues are approximated by a linear equation. The horizontal axis of FIG. 6 represents wavelength, and the vertical axis represents spectral reflectance. In FIG. 6, line L21 represents the spectral reflectance of non-cancerous tissue, and line L22 represents the spectral reflectance of cancerous tissue. Comparing the slope of line S21, which is obtained by approximating line L21 with a linear equation, and the slope of line S22, which is obtained by approximating line L22 with a linear equation, for wavelengths of 600 nm to 700 nm, the slope of line S22 is greater than the slope of line S21. By determining a correspondence between the slope of the linear equation and the depth of invasion from this relationship and recording it in recording unit 408, calculation unit 410c can estimate the depth of invasion using the slope of the linear equation. It can be seen from FIG. 6 that the slope of line S22 begins to become larger than the slope of line S21 at wavelengths where line L11 and line L12 shown in FIG. 5 intersect.

[0072] 4, the calculation unit 410c corrects the calculated gradient for each pixel (step S105). This post-processing corrects the gradient, which is an index for estimating the invasion depth.

[0073] Furthermore, the calculation unit 410c classifies the gradient for each pixel (step S106). For example, the calculation unit 410c classifies the gradient for each pixel into SM1 or shallower / SM2 or deeper.

[0074] Then, the display control unit 410d generates an image to which a color according to the classification is assigned (step S107). Note that the display control unit 410d may generate an image in which the colors are coded using values ​​obtained by averaging the calculated gradients over multiple pixels.

[0075] Furthermore, the display control unit 410d displays the image superimposed on the observation image (normal light image) (step S108). FIG. 7 is a diagram showing an example of an invasion depth map. As shown in FIG. 7, by displaying an image in which each pixel is classified as shallower than SM1 / deeper than SM2 on the display device 3, the surgeon's judgment of the depth of cancer invasion can be assisted and the speed of the judgment of the depth of cancer invasion can be improved. The display control unit 410d may superimpose an image color-coded by the depth of invasion only on a region designated by the user. The display control unit 410d may also superimpose an image color-coded by the depth of invasion only on a region where the depth of cancer invasion, automatically determined by a predetermined program, is equal to or greater than a threshold. Furthermore, the display control unit 410d may have a function to numerically display the depth of invasion at a position designated by a cursor on the image color-coded by the depth of invasion. Furthermore, the display control unit 410d may have a function to numerically display the average depth of invasion of a region designated by the user on the image color-coded by the depth of invasion.

[0076] According to the embodiment 1-1 described above, the calculation unit 410c estimates the depth of cancer invasion based on pixel values, thereby assisting the surgeon in determining the depth of cancer invasion and improving the speed at which the depth of cancer invasion can be determined.

[0077] Fig. 8 is a diagram showing a modified example of the depth of invasion map. As shown in Fig. 8, the depth of cancer invasion may be estimated in five stages: MP or deeper / SM2 / SM1 / M / non-cancer. In this way, the depth of cancer invasion may be classified into multiple stages. Furthermore, instead of being limited to the classification of SM1 or shallower / SM2 or deeper shown in Fig. 7, classification into cancer / non-cancer may also be used. Furthermore, an image may be generated in which the color changes continuously depending on the depth of cancer invasion (the calculation result of the slope of the linear equation).

[0078] The display control unit 410d may also cause the display device 3 to display various images that make it easier to visually recognize the depth of cancer invasion. For example, the display control unit 410d may highlight regions where deep infiltration is present. The display control unit 410d may also highlight regions where cancer infiltration is above a threshold. The display control unit 410d may also highlight regions where cancer is estimated to have infiltrated into the muscle layer. The display control unit 410d may also highlight protruding regions or depressed regions. Furthermore, the display control unit 410d may generate an image that combines the exemplified display modes and cause the display device 3 to display it.

[0079] (Embodiment 1-2) Next, the processing executed by the image processing device according to the embodiment 1-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0080] The extraction unit 410b extracts pixel values ​​of two wavelength bands for each pixel in the spectral image. The two wavelength bands are, for example, a first wavelength band of 600 nm to 650 nm and a second wavelength band of 650 nm to 700 nm, as long as the two wavelength bands are included in the wavelength range of 600 nm to 700 nm. The lower limits of the two wavelength bands are preferably wavelengths at which there is a difference in the change between the absorption coefficient of deoxygenated hemoglobin and the absorption coefficient of oxygenated hemoglobin for each wavelength.

[0081] FIG. 9 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 1-2.

[0082] 9, first, the acquiring unit 410a acquires a spectral image of a first wavelength band (step S201). The first wavelength band is, for example, a wavelength range of 600 nm to 650 nm.

[0083] Next, similarly to step S102, the calculation unit 410c interpolates the pixel values ​​of the saturated pixels (step S202).

[0084] Thereafter, similarly to step S103, the calculation unit 410c approximates the optical spectrum for each pixel to a linear expression (step S203).

[0085] Then, similar to step S104, the calculation unit 410c obtains the gradient of the linear expression for each pixel (step S204).

[0086] Furthermore, the processes of steps S205 to S208 are executed in parallel with steps S201 to S204. First, the acquisition unit 410a acquires a spectral image of the second wavelength band (step S205). The second wavelength band is, for example, a wavelength range of 650 nm to 700 nm.

[0087] The processing in steps S206 to S208 is the same as the processing in steps S202 to S204, and therefore a description thereof will be omitted.

[0088] FIG. 10 is a diagram showing how the spectral reflectances of cancer and non-cancerous tissues are approximated by two linear equations. The horizontal axis of FIG. 10 represents wavelength, and the vertical axis represents spectral reflectance. In FIG. 10, line L21 represents the spectral reflectance of non-cancerous tissue, and line L22 represents the spectral reflectance of cancerous tissue. Calculation unit 410c calculates the slope of a straight line S31, which approximates line L21 with the linear equation, and the slope of a straight line S32, which approximates line L22 with the linear equation, over wavelengths of 600 nm to 650 nm. Furthermore, calculation unit 410c calculates the slope of a straight line S33, which approximates line L21 with the linear equation, and the slope of a straight line S34, which approximates line L22 with the linear equation, over wavelengths of 650 nm to 650 nm.

[0089] Next, the calculation unit 410c calculates the average value of the gradient for each pixel (step S209). In this way, by dividing and approximating the wavelength band and taking the average value, the gradient can be appropriately determined.

[0090] Thereafter, the processes of steps S105 to S108 are executed in the same manner as in embodiment 1-1.

[0091] According to the above-described embodiment 1-2, the index for estimating the invasion depth is calculated by dividing into two wavelength bands, so that the speed of determining the invasion depth of cancer can be further improved.

[0092] (Embodiments 1-3) Next, the processing executed by the image processing device according to embodiment 1-3 will be described. The processing executed by the image processing device according to embodiment 1-3 may be the same as the processing of embodiment 1-1 shown in Fig. 4, and therefore the description will be omitted. However, in step S101, the acquisition unit 410a acquires a spectral image of a predetermined wavelength band (wavelengths from 640 nm to 650 nm) extracted by the extraction unit 410b (step S101).

[0093] Fig. 11 is a diagram showing how the spectral reflectances of cancer and non-cancerous tissues are approximated by a linear equation. The horizontal axis of Fig. 11 represents wavelength, and the vertical axis represents spectral reflectance. In Fig. 11, line L21 represents the spectral reflectance of non-cancerous tissue, and line L22 represents the spectral reflectance of cancerous tissue. Calculation unit 410c calculates the slope of a straight line S41 obtained by approximating line L21 with the linear equation, and the slope of a straight line S42 obtained by approximating line L22 with the linear equation, over wavelengths of 640 nm to 650 nm.

[0094] Using a statistical method (T method), it was possible to estimate that there is a high correlation between the spectral slope and the depth of invasion at wavelengths of 645 / 650 / 655 / 660 nm. Furthermore, when the relationship between the spectral slope and the depth of invasion was confirmed using individual case data, it was found that the spectral slope at 645 nm had the fewest exceptions. Therefore, by using the spectral reflectance at wavelengths of 640 to 650 nm to determine the spectral slope at 645 nm, it is possible to further improve the speed at which the depth of invasion of cancer can be determined.

[0095] The predetermined wavelength band may be set so that the wavelength of 645 nm is the median value, and for example, the predetermined wavelength band may be set to wavelengths of 630 nm to 660 nm.

[0096] (Embodiment 2-1) Next, the processing executed by the image processing device according to embodiment 2-1 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0097] 12 is a block diagram showing the functional configuration of the main parts of an endoscope and a control device according to embodiment 2-1. As shown in FIG. 12, in an endoscope system 1A, a control device 4A includes a first light source unit 402A and a second light source unit 403A.

[0098] The first light source section 402A may have the same configuration as the light source section 402, and therefore a description thereof will be omitted.

[0099] The second light source unit 403A emits special light having a predetermined wavelength band under the control of the light source control unit 404, thereby supplying the special light as illumination light to the light guide 231. Here, the special light is, for example, red light (first observation light) having a peak wavelength near 630 nm used in red dichromatic imaging (RDI) and amber light (second observation light) having a peak wavelength near 600 nm. The special light may also be light used in narrow band imaging (NBI) using narrow band light including 390 to 445 nm and 530 to 550 nm.

[0100] The acquiring unit 410a acquires a first observation image obtained by irradiating the subject with the first observation light and capturing the returned light, and a second observation image obtained by irradiating the subject with the second observation light and capturing the returned light.

[0101] The calculation unit 410c estimates the depth of invasion of the cancer based on the pixel values ​​of each pixel in the first observational image and the pixel values ​​of each pixel in the second observational image.

[0102] FIG. 13 is a flowchart showing an outline of the process executed by the image processing device according to the embodiment 2-1.

[0103] As shown in FIG. 13, first, the acquiring unit 410a acquires an R image (first observational image) (step S301).

[0104] Next, the calculation unit 410c interpolates the pixel values ​​of pixels that do not have an R signal (step S302).

[0105] Thereafter, similarly to step S102, the calculation unit 410c interpolates the pixel values ​​of the saturated pixels (step S303).

[0106] Furthermore, the processes of steps S304 to S306 are executed in parallel with steps S301 to S303. First, the acquisition unit 410a acquires an Am image (second observational image) (step S304).

[0107] The processing in steps S305 and S306 is similar to the processing in steps S302 and S303, and therefore a description thereof will be omitted.

[0108] The calculation unit 410c calculates the difference in pixel values ​​between the R signal and the Am signal (step S307). FIG. 14 is a diagram showing the difference in pixel values ​​between the R signal and the Am signal. FIG. 14 illustrates a line LR indicating red light and a line LAm indicating amber light, with respect to a line LSR indicating spectral reflectance. The difference Δ1 obtained by subtracting the pixel value Am of amber light from the pixel value R of red light corresponds to the slope of the spectral spectrum, and therefore the depth of invasion can be estimated based on the magnitude of this difference Δ1.

[0109] Returning to FIG. 13, the calculation unit 410c corrects the difference between the pixel values ​​(step S308). FIG. 15 is a diagram showing how the pixel values ​​of the R signal and the Am signal are corrected. As shown in FIG. 15, the pixel value R of red light is corrected to a pixel value R', and the pixel value Am of amber light is corrected to a pixel value Am' so that the difference Δ1 is less than the difference Δ2. This post-processing corrects the difference Δ1, which is an index for estimating the depth of invasion. The calculation method for the correction is not particularly limited, but for example, the pixel values ​​R and Am can be cubed to correct the pixel values ​​so that the difference Δ1 is less than the difference Δ2. However, the pixel values ​​R and Am may also be raised to the second power, fourth power, or higher.

[0110] 13, the calculation unit 410c classifies the pixel value difference Δ2 (step S309). For example, the calculation unit 410c classifies the pixel value difference Δ2 into shallower than SM1 / deeper than SM2.

[0111] Thereafter, the processes of steps S107 and S108 are executed in the same manner as in embodiment 1-1.

[0112] According to the embodiment 2-1 described above, the depth of penetration can be estimated using only two components, red light and amber light, which makes it possible to use it with all endoscopes equipped with an RDI function and reduces the processing load.

[0113] In the embodiment 2-1, an example has been described in which the calculation unit 410c calculates the difference in pixel values ​​between the R signal and the Am signal. However, the calculation unit 410c may calculate the ratio of the pixel values ​​between the R signal and the Am signal, and estimate the depth of cancer invasion based on the calculated ratio.

[0114] (Embodiment 2-2) Next, the processing executed by the image processing device according to embodiment 2-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0115] The calculation unit 410c estimates the depth of invasion of the cancer based on the pixel value of each pixel in the first observational image.

[0116] 16 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 2-2. Steps S301 to S303 are the same as those in embodiment 2-1, and therefore a description thereof will be omitted.

[0117] Thereafter, the calculation unit 410c uses the pixel value of the R signal as an index (step S401). In the embodiment 2-1, the difference between the pixel value of red light and the pixel value of amber light is used as an index of the penetration depth, but the pixel value of amber light is sufficiently smaller than the pixel value of red light. Therefore, the pixel value of the R signal may be used as an index of the penetration depth by approximating the difference between the pixel value of red light and the pixel value of amber light.

[0118] Subsequently, the calculation unit 410c corrects the pixel values ​​(step S402). Through this post-processing, the pixel values ​​of the R signals, which are indexes for estimating the depth of invasion, are corrected.

[0119] Furthermore, the calculation unit 410c classifies the index (step S403). For example, the calculation unit 410c classifies the pixel values ​​of the R signal into SM1 or shallower / SM2 or deeper.

[0120] Thereafter, the processes of steps S107 and S108 are executed in the same manner as in embodiment 1-1.

[0121] According to the above-described embodiment 2-2, the invasion depth can be estimated using only red light, thereby reducing the processing load.

[0122] (Embodiment 2-3) Next, the processing executed by the image processing device according to the embodiment 2-3 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0123] 17 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 2-3. Steps S301 to S306 are the same as those in embodiment 2-1, and therefore a description thereof will be omitted.

[0124] 17, the calculation unit 410c normalizes the pixel value of each pixel in the R image (first observational image) by the pixel value of each pixel in the Am image (second observational image) (step S501). By normalizing the pixel value of the R signal by the pixel value of the Am signal at each pixel and then calculating the difference, the slope of the optical spectrum can be clearly expressed.

[0125] The subsequent processing is the same as in embodiment 2-1, and therefore a description thereof will be omitted.

[0126] According to the above-described embodiment 2-3, the pixel values ​​of the R signal are normalized by the pixel values ​​of the Am signal, thereby making it possible to further improve the speed at which the depth of cancer invasion can be determined.

[0127] (Embodiment 2-4) Next, the processes executed by the image processing devices according to the embodiments 2 to 4 will be described. The description of the same processes as those already described will be omitted as appropriate.

[0128] 18 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 2-4. Steps S301 to S306 are the same as those in embodiment 2-1, and therefore a description thereof will be omitted.

[0129] 18, the calculation unit 410c normalizes the pixel values ​​of the R signal by the pixel values ​​of the Am signal (step S501). As in embodiment 2-3, normalizing the pixel values ​​of the R signal by the pixel values ​​of the Am signal at each pixel makes it possible to clearly represent the slope of the optical spectrum.

[0130] The subsequent processing is the same as in embodiment 2-2, and therefore a description thereof will be omitted.

[0131] According to the above-described embodiments 2-4, the pixel values ​​of the R signal are normalized by the pixel values ​​of the Am signal, thereby making it possible to further improve the speed at which the depth of cancer invasion can be determined.

[0132] (Embodiment 3-1) Next, the processing executed by the image processing device according to embodiment 3-1 will be described. Processing similar to that already described will be omitted as appropriate. Fig. 19 is a block diagram showing the functional configuration of the main parts of the endoscope and control device according to embodiment 3-1. As shown in Fig. 19, in the endoscope system 1B, the control unit 410B of the control device 4B has a determination unit 410e.

[0133] The determining unit 410e determines whether or not a region is highly likely to contain cancer based on the depth of cancer invasion of each pixel.

[0134] 20 is a flowchart showing an outline of the process executed by the image processing device according to embodiment 3-1. Steps S101 to S108 are the same as those in embodiment 1-1, and therefore a description thereof will be omitted.

[0135] Thereafter, the calculation unit 410c sets a judgment region for judging whether the depth is SM2 or deeper or SM1 or shallower in accordance with the user's input to the input unit 407 (step S601). Note that the judgment region may be set to a region that is automatically set by a predetermined program. Alternatively, a predetermined program may extract multiple candidate regions, and the region selected by the user from among them may be set to the judgment region.

[0136] Next, the calculation unit 410c extracts a representative value of the probability of the determination region (step S602). The probability is the probability that the invasion depth calculated based on the gradient calculated in step S106 is SM2 or deeper. The representative value may be, for example, the average value of the probability for each pixel in the determination region, or it may be the median, mode, maximum value, etc. of the probability for each pixel in the determination region. The representative value may also be a value set by the user, or a value selected by the user from multiple candidate representative values ​​automatically calculated by a predetermined program.

[0137] Then, the determination unit 410e determines whether the probability is greater than the threshold value (step S603).

[0138] If the determining unit 410e determines that the probability is greater than the threshold (step S603: Yes), the determining unit 410e determines that the depth of cancer invasion in the determination region is SM2 or deeper, and outputs the determination result (step S604).

[0139] If the determining unit 410e determines that the probability is not greater than the threshold (step S603: No), the determining unit 410e determines that the depth of cancer invasion in the determination region is shallower than SM1, and outputs the determination result (step S605).

[0140] According to the above-described embodiment 3-1, the determining unit 410e outputs the determination result, and therefore the surgeon can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0141] (Embodiment 3-2) Next, the processing executed by the image processing device according to embodiment 3-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0142] 21 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 3-2. Steps S201 to S108 are the same as those in embodiment 1-2, and therefore a description thereof will be omitted.

[0143] Thereafter, the processes of steps S601 to S605 are executed in the same manner as in embodiment 3-1.

[0144] According to the above-described embodiment 3-2, the determining unit 410e outputs the determination result, and therefore the operator can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0145] (Embodiment 3-3) Next, the processing executed by the image processing device according to embodiment 3-3 will be described. Processing similar to that already described will be omitted where appropriate. The processing executed by the image processing device according to embodiment 3-3 may be similar to the processing of embodiment 3-1 shown in FIG. 20, and therefore will not be described again. However, in step S101, the acquisition unit 410a acquires a spectral image of a predetermined wavelength band (wavelengths from 640 nm to 650 nm) extracted by the extraction unit 410b (step S101).

[0146] According to the above-described embodiment 3-3, similar to embodiment 1-3, the speed at which the depth of cancer invasion can be determined can be further improved by calculating the spectral slope at a wavelength of 645 nm, and similar to embodiment 3-1, since the judgment unit 410e outputs the judgment result, the surgeon can refer to the judgment result and shorten the time required to judge the depth of cancer invasion.

[0147] (Embodiment 4-1) Next, processing executed by an image processing device according to embodiment 4-1 will be described. Processing similar to that already described will be omitted where appropriate. FIG. 22 is a block diagram showing the functional configuration of the main parts of an endoscope and a control device according to embodiment 4-1. As shown in FIG. 22, in an endoscope system 1C, a control device 4C includes a first light source unit 402C and a second light source unit 403C. Furthermore, a control unit 410C of the control device 4C includes a determination unit 410e.

[0148] First light source section 402C and second light source section 403C may have the same configuration as first light source section 402A and second light source section 403A, and therefore a description thereof will be omitted.

[0149] 23 is a flowchart showing an outline of the process executed by the image processing device according to embodiment 4-1. Steps S301 to S108 are the same as those in embodiment 2-1, and therefore a description thereof will be omitted.

[0150] Thereafter, the processes of steps S601 to S605 are executed in the same manner as in embodiment 3-1.

[0151] According to the above-described embodiment 4-1, the determining unit 410e outputs the determination result, and therefore the surgeon can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0152] (Embodiment 4-2) Next, the processing executed by the image processing device according to embodiment 4-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0153] 24 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 4-2. As shown in FIG. 24, steps S301 to S108 are the same as those in embodiment 2-2, and therefore a description thereof will be omitted.

[0154] Thereafter, the processes of steps S601 to S605 are executed in the same manner as in embodiment 3-1.

[0155] According to the above-described embodiment 4-2, the determining unit 410e outputs the determination result, and therefore the operator can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0156] (Embodiment 4-3) Next, the processing executed by the image processing device according to embodiment 4-3 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0157] 25 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 4-3. Steps S301 to S108 are the same as those in embodiment 2-3, and therefore a description thereof will be omitted.

[0158] Thereafter, the processes of steps S601 to S605 are executed in the same manner as in embodiment 3-1.

[0159] According to the above-described embodiment 4-3, the determining unit 410e outputs the determination result, and therefore the surgeon can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0160] (Embodiment 4-4) Next, the processing executed by the image processing device according to embodiment 4-4 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0161] Fig. 26 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 4-4. As shown in Fig. 26, steps S301 to S108 are the same as those in embodiment 2-4, and therefore a description thereof will be omitted.

[0162] Thereafter, the processes of steps S601 to S605 are executed in the same manner as in embodiment 3-1.

[0163] According to the above-described embodiment 4-4, the determining unit 410e outputs the determination result, and therefore the operator can shorten the time required to determine the depth of invasion of cancer by referring to the determination result.

[0164] (Embodiment 5-1) Next, the processing executed by the image processing device according to the embodiment 5-1 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0165] The acquisition unit 410a acquires a normal light image by irradiating a subject with white light and capturing the returned light.

[0166] The calculation unit 410c estimates the depth of invasion of the cancer based on the pixel value of each pixel in the first observational image and the pixel value of each pixel in the normal light image.

[0167] 27 is a flowchart showing an outline of the processing executed by the image processing device according to embodiment 5-1. As shown in Fig. 27, the processing of steps S701 to S705 is executed in parallel with steps S101 to S105. First, the acquisition unit 410a acquires a normal light image (step S701).

[0168] Next, the calculation unit 410c interpolates pixel values ​​of missing pixels in the normal light image (step S702).

[0169] Thereafter, the calculation unit 410c interpolates the pixel values ​​of the saturated pixels in the same manner as in step S102 (step S703).By the pre-processing of steps S702 and S703, variations between pixels are reduced.

[0170] Then, the calculation unit 410c converts the RGB signals into L*a*b* colors (step S704).

[0171] Furthermore, the calculation unit 410c corrects a*, which corresponds to red in the L*a*b* color space (step S705).

[0172] Then, the calculation unit 410c calculates the probability of being deeper than SM2 (step S706). First, (Equation 1) y = α × (slope) + β × (a*) + γ, (Equation 2) P = e y / (1+e y )=1 / (1+e -y ) and tens to hundreds of data on the depth of cancer determined by a doctor or the like (for example, data where SM2 or deeper (P=1) and SM1 or shallower (P=0) are known), the coefficients α, β, and γ of (Equation 1) are calculated by logistic regression analysis. Note that the (slope) and (a*) in (Equation 1) use values ​​that have been corrected in the same way as in steps S105 and S705. The probability P can be calculated by substituting the corrected value of the slope calculated in step S105 and the corrected value of a* calculated in step S705 into this calculation formula.

[0173] Thereafter, the calculation unit 410c classifies the probabilities (step S707). For example, the calculation unit 410c classifies the probabilities into those equal to or greater than a threshold and those less than a threshold.

[0174] Thereafter, the processes of steps S107 to S605 are executed in the same manner as in embodiment 3-1.

[0175] According to the above-described embodiment 5-1, in addition to the features of embodiment 3-1, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0176] (Embodiment 5-2) Next, the processing executed by the image processing device according to the embodiment 5-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0177] Fig. 28 is a flowchart showing an outline of processing executed by an image processing device according to embodiment 5-2. In embodiment 5-2, processing similar to the processing of embodiment 3-2 shown in Fig. 21 and the processing of embodiment 5-1 shown in Fig. 27 is executed.

[0178] According to the above-described embodiment 5-1, in addition to the embodiment 3-2, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0179] (Embodiment 5-3) Next, the processing executed by the image processing device according to embodiment 5-3 will be described. Processing similar to that already described will be omitted where appropriate. The processing executed by the image processing device according to embodiment 5-3 may be similar to the processing of embodiment 5-1 shown in FIG. 27, and therefore will not be described. However, in step S101, the acquisition unit 410a acquires a spectral image of a predetermined wavelength band (wavelengths from 640 nm to 650 nm) extracted by the extraction unit 410b (step S101).

[0180] According to the above-described embodiment 5-3, in addition to the embodiment 3-3, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer is determined can be further improved.

[0181] (Embodiment 6-1) Next, the processing executed by the image processing device according to embodiment 6-1 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0182] Fig. 29 is a flowchart showing an outline of processing executed by an image processing device according to embodiment 6-1. As shown in Fig. 29, embodiment 6-1 executes processing similar to the processing of embodiment 4-1 shown in Fig. 23 and the processing of embodiment 5-1 shown in Fig. 27.

[0183] According to the above-described embodiment 6-1, in addition to the features of embodiment 4-1, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0184] (Embodiment 6-2) Next, the processing executed by the image processing device according to embodiment 6-2 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0185] Fig. 30 is a flowchart showing an outline of processing executed by an image processing device according to embodiment 6-2. As shown in Fig. 30, embodiment 6-2 executes processing similar to the processing of embodiment 4-2 shown in Fig. 24 and the processing of embodiment 5-1 shown in Fig. 27.

[0186] According to the above-described embodiment 6-2, in addition to the features of embodiment 4-2, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0187] (Embodiment 6-3) Next, the processing executed by the image processing device according to embodiment 6-3 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0188] Fig. 31 is a flowchart showing an outline of processing executed by an image processing device according to embodiment 6-3. As shown in Fig. 31, embodiment 6-3 executes processing similar to the processing of embodiment 4-3 shown in Fig. 25 and the processing of embodiment 5-1 shown in Fig. 27.

[0189] According to the above-described embodiment 6-3, in addition to the features of embodiment 4-3, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0190] (Embodiment 6-4) Next, the processing executed by the image processing device according to embodiment 6-4 will be described. The description of the same processing as that already described will be omitted as appropriate.

[0191] Fig. 32 is a flowchart showing an outline of processing executed by an image processing device according to embodiment 6-4. As shown in Fig. 32, embodiment 6-4 executes processing similar to the processing of embodiment 4-4 shown in Fig. 26 and the processing of embodiment 5-1 shown in Fig. 27.

[0192] According to the above-described embodiment 6-4, in addition to the embodiment 4-4, the invasion depth is determined using a*, so that the speed at which the invasion depth of cancer can be determined can be further improved.

[0193] Further advantages and modifications will readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0194] 1, 1A, 1B, 1C Endoscope System 2 Endoscopy 3 Display device 4, 4A, 4B, 4C control device 21 Insertion section 22 Control section 23 Universal Code 24 Tip 25 Curved section 26 Flexible tube section 27 Connector part 201 Illumination optical system 202 Imaging Optical System 203 Image sensor 204 A / D conversion section 205 P / S conversion unit 206 Imaging and recording unit 207 Imaging control unit 221 Curved Knob 222 Insertion port 223 Operating member 231 Light Guide 232 First signal line 233 Second signal line 401 Condenser Lens 402 Light source section 402A, 402C 1st light source section 403A, 403C 2nd light source section 404 Light source control unit 405 S / P conversion unit 406 Image Processing Unit 407 Input section 408 Recording Department 408a Program Recording Section 408b Image data recording unit 409 Communications Department 410, 410B, 410C control section 410a Acquisition Department 410b Extraction part 410c calculation section 410d Display control unit 410e Judgment section

Claims

1. an acquisition unit that irradiates a subject with first observation light and acquires a first observation image by capturing returned light; a calculation unit that estimates the depth of cancer invasion based on the pixel value of each pixel in the first observational image; An image processing device comprising:

2. the first observation light is broadband light having a wide range of wavelength components, The image processing apparatus according to claim 1 , wherein the first observation image is a spectral image for each wavelength.

3. an extracting unit that extracts the pixel value of each pixel in a predetermined wavelength band in the spectral image; The image processing device according to claim 2 , wherein the calculation unit estimates the depth of invasion of the cancer based on a gradient of a spectrum generated from the pixel values ​​extracted by the extraction unit.

4. 4. The image processing device according to claim 3, wherein the predetermined wavelength band is included in the wavelength range from 600 nm to 700 nm.

5. 4. The image processing device according to claim 3, wherein the lower limit of the predetermined wavelength band is a wavelength at which a difference occurs between the change in the absorption coefficient of deoxygenated hemoglobin and the change in the absorption coefficient of oxygenated hemoglobin for each wavelength.

6. an extracting unit that extracts the pixel values ​​of two wavelength bands of each pixel in the spectral image; The image processing device according to claim 2 , wherein the calculation unit estimates the depth of invasion of the cancer based on the pixel values ​​extracted by the extraction unit.

7. The image processing device according to claim 6 , wherein the two wavelength bands are included in the wavelength range from 600 nm to 700 nm.

8. The image processing device according to claim 6 , wherein the lower limits of the two wavelength bands are wavelengths at which a difference occurs between the change in the absorption coefficient of deoxygenated hemoglobin and the change in the absorption coefficient of oxygenated hemoglobin for each wavelength.

9. the acquisition unit acquires a second observation image by irradiating the object with second observation light and capturing returned light; the first observation light is light having a peak wavelength in the vicinity of 630 nm, 2. The image processing apparatus according to claim 1, wherein the second observation light has a peak wavelength in the vicinity of 600 nm.

10. The image processing apparatus according to claim 9 , wherein the calculation unit estimates the depth of invasion of the cancer based on the pixel value of each pixel of the first observational image and the pixel value of each pixel of the second observational image.

11. The image processing apparatus according to claim 9 , wherein the calculation unit normalizes the pixel value of each pixel of the first observational image by the pixel value of each pixel of the second observational image.

12. The image processing apparatus according to claim 9 , wherein the calculation unit estimates the depth of invasion of the cancer based on a pixel value of each pixel in the first observational image.

13. the acquisition unit acquires a normal light image by irradiating the subject with white light and capturing a return light image; The image processing apparatus according to claim 1 , wherein the calculation unit estimates the depth of invasion of the cancer based on a pixel value of each pixel in the first observational image and a pixel value of each pixel in the normal light image.

14. The image processing device according to claim 1 , further comprising a display control unit that causes a display unit to display the depth of invasion of the cancer for each pixel.

15. The image processing device according to claim 14 , wherein the display control unit causes the display unit to display the first observational image or the normal light image with the depth of invasion of the cancer for each pixel superimposed thereon.

16. The image processing device according to claim 1 , further comprising a determination unit that determines whether or not a region has a high possibility of cancer based on the depth of invasion of the cancer in each pixel.

17. a light source unit that irradiates a first observation light onto the subject from a tip of an insertion unit that is inserted into the subject; an imaging unit that captures an image of the return light of the first observation light to generate a first observation image; an image processing device including: an acquisition unit that acquires the first observational image from the imaging unit; and a calculation unit that estimates the depth of cancer invasion based on a pixel value of each pixel in the first observational image; An endoscope system comprising:

18. the first observation light is broadband light having a wide range of wavelength components, the first observation image is a spectral image for each wavelength, the image processing device includes an extraction unit that extracts the pixel value of each pixel in a predetermined wavelength band in the spectral image, The endoscope system according to claim 17 , wherein the calculation unit estimates the depth of invasion of the cancer based on a gradient of a spectrum generated from the pixel values ​​extracted by the extraction unit.

19. the light source unit irradiates the subject with second observation light from the distal end of the insertion unit; the imaging unit captures an image of the returned light of the second observation light to generate a second observation image; the acquisition unit acquires the first observational image and the second observational image from the imaging unit; The endoscope system according to claim 17 , wherein the calculation unit estimates the depth of invasion of the cancer based on the pixel value of each pixel in the first observational image and the pixel value of each pixel in the second observational image.

20. the acquiring unit acquires a first observation image by irradiating the object with the first observation light and capturing the return light; A method for operating an image processing device, wherein a calculation unit estimates the depth of cancer invasion based on the pixel value of each pixel of the first observational image.

21. the first observation light is broadband light having a wide range of wavelength components, the first observation image is a spectral image for each wavelength, an extraction unit extracts the pixel value of each pixel in a predetermined wavelength band in the spectral image; The method of claim 20 , wherein the calculation unit estimates the depth of invasion of the cancer based on the gradient of a spectrum generated from the pixel values ​​extracted by the extraction unit.

22. the acquiring unit acquires a second observation image by irradiating the object with second observation light and capturing a return light; 21. The method of claim 20, wherein the calculation unit estimates the depth of invasion of the cancer based on the pixel values ​​of each pixel of the first observational image and the pixel values ​​of each pixel of the second observational image.

Images