Image processing apparatus, endoscope system, method of operating an image processing apparatus, and program for an image processing apparatus

Abstract

The purpose of the present invention is to provide an image processing device (16), an endoscope system (10), a method for operating an image processing device, and a program for an image processing device with which it is possible to obtain diagnosis assistance information quickly and at a higher accuracy.　The image processing device (16): acquires a plurality of types of candidate images based on an endoscope image; performs a control for displaying, on a display (18), a display image based on at lease one type of candidate image; performs a first analysis process on one or a plurality of types of candidate images set in advance; selects, as an optimum image, at least one type of candidate image from the plurality of types of candidate images, on the basis of a first analysis process result obtained through the first analysis process; and performs a second analysis process on the optimum image and thereby obtains a second analysis process result.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus for obtaining diagnostic support information, an endoscope system, an operation method of the image processing apparatus, and a program for the image processing apparatus.

Background Art

[0002] In the medical field, diagnosis using an endoscope system including a light source device, an endoscope, and a processor device is widely performed. In addition to displaying an image obtained by photographing an observation target with an endoscope (hereinafter referred to as an endoscope image) in a natural color on a display or the like and using it for diagnosis, in some cases, various endoscope images in which colors or structures such as blood vessels are emphasized are used for diagnosing the observation target by a method called image-enhanced endoscopy or image-enhanced endoscopy (IEE).

[0003] In addition, by analyzing various endoscope images by IEE or the like, CAD (Computer-Aided Diagnosis) technology for generating diagnostic support information including determination results such as the stage of a disease from the range of a region where there may be a lesion in the observation target and / or the degree of inflammation, etc. has been developed. For example, an endoscope system that highly accurately determines the severity or progression of a disease such as the stage of ulcerative colitis using various endoscope images by IEE is known (Patent Document 1). Also, an endoscope apparatus that obtains diagnostic support information after selecting an image with an appropriate brightness for CAD is known (Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Since various endoscopic images by IEE etc. are used by doctors etc. for diagnosis, they are often images with coloring that causes no discomfort when viewed by humans. Endoscopic images with good visibility for humans obtained by IEE are not necessarily endoscopic images from which good diagnostic support information can be obtained by image analysis using CAD etc. That is, there may be endoscopic images that are not easily visible to humans but are suitable for image analysis using CAD etc. Therefore, by performing CAD etc. using an appropriate type of endoscopic image for image analysis using CAD etc., there is a possibility of obtaining diagnostic support information with higher accuracy.

[0006] Also, by using CAD etc. to obtain detailed diagnostic support information in real time during an endoscopic examination, for example, a doctor can discover an area with a high possibility of a lesion and examine this area in detail during a single endoscopic examination. This is preferable because there is no need for a second endoscopic examination, but it is necessary to obtain diagnostic support information quickly during the examination. Also, from the perspective of reducing the burden on the subject of the endoscopic examination and improving the efficiency of the endoscopic examination, it is preferable to obtain diagnostic support information quickly while performing CAD etc.

[0007] An object of the present invention is to provide an image processing apparatus, an endoscopic system, a method of operating the image processing apparatus, and a program for the image processing apparatus that can obtain diagnostic support information quickly and with high accuracy.

Means for Solving the Problem

[0008] The image processing apparatus of the present invention includes an image processor. The image processor acquires a plurality of types of candidate images based on an endoscopic image obtained by photographing an observation target using an endoscope, and performs control to display a display image based on at least one type of candidate image among the plurality of types of candidate images on a display. The image processor performs a first analysis process on one or a plurality of types of candidate images set in advance among the plurality of types of candidate images, selects at least one type of candidate image from the plurality of types of candidate images as an optimal image based on the first analysis process result obtained by the first analysis process, and performs a second analysis process on the optimal image to obtain a second analysis process result.

[0009] Preferably, the image processor performs control to display the second analysis process result on the display.

[0010] Preferably, the image processor performs control to display the second analysis process result by superimposing it on the display image.

[0011] Preferably, the first analysis process and the second analysis process are analysis processes with different contents from each other.

[0012] Preferably, candidate images are generated by performing an enhancement process on the endoscopic image, and the image processor acquires a plurality of types of candidate images by distinguishing the types of candidate images according to the presence or type of the enhancement process.

[0013] Preferably, the enhancement process is a color enhancement process and / or a structure enhancement process.

[0014] Further, the endoscope system of the present invention includes an image processing apparatus and a light source unit that emits illumination light for irradiating an observation target.

[0015] Preferably, the image processor acquires an endoscopic image obtained by photographing an observation target illuminated by each of a plurality of types of illumination lights having different spectral spectra emitted by the light source unit as different types of candidate images.

[0016] The light source unit preferably emits each of a plurality of types of illumination light having different spectral spectra in a light emission cycle having a preset order repeatedly.

[0017] The image processor preferably selects at least one optimal image from a plurality of types of candidate images obtained in one light emission cycle.

[0018] A light source processor that emits first illumination light in a first light emission pattern during a first illumination period, emits second illumination light in a second light emission pattern during a second illumination period, and switches between the first illumination light and the second illumination light, and an imaging sensor that outputs a first endoscope image obtained by photographing an observation target illuminated by the first illumination light and a second endoscope image obtained by photographing the observation target illuminated by the second illumination light, and the image processor preferably acquires the first endoscope image and the second endoscope image as candidate images.

[0019] The image processor preferably acquires an endoscope image obtained by photographing an observation target illuminated by white illumination light emitted by the light source unit as one type of candidate image.

[0020] The image processor preferably acquires an endoscope image obtained by photographing an observation target illuminated by illumination light including narrow-band light in a preset wavelength band emitted by the light source unit as one type of candidate image.

[0021] In addition, the method for operating an image processing apparatus according to the present invention includes a candidate image acquisition step of acquiring a plurality of types of candidate images based on an endoscopic image obtained by photographing an observation target using an endoscope, a display image control step of performing control to display a display image based on at least one type of candidate image among the plurality of types of candidate images on a display, a first analysis processing step of performing a first analysis process on one or a plurality of types of candidate images set in advance among the plurality of types of candidate images, an optimal image selection step of selecting at least one type of candidate image from the plurality of types of candidate images as an optimal image based on the first analysis processing result obtained by the first analysis processing, and a second analysis processing step of performing a second analysis process on the optimal image to obtain a second Analysis analysis processing step of obtaining an analysis processing result.

[0022] In addition, a program for an image processing apparatus according to the present invention causes a computer to have a candidate image acquisition function of acquiring a plurality of types of candidate images based on an endoscopic image obtained by photographing an observation target using an endoscope, a display control function of performing control to display a display image based on at least one type of candidate image among the plurality of types of candidate images on a display, a first analysis processing function of performing a first analysis process on one or a plurality of types of candidate images set in advance among the plurality of types of candidate images, an optimal image selection function of selecting at least one type of candidate image from the plurality of types of candidate images as an optimal image based on the first analysis processing result obtained by the first analysis processing, and a second analysis processing function of performing a second analysis process on the optimal image to obtain diagnostic support information. It is a program for an image processing apparatus for realizing.

Effects of the Invention

[0023] According to the present invention, diagnostic support information can be obtained quickly and with high accuracy.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Embodiments for Carrying Out the Invention

[0025] As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a keyboard 19. The endoscope 12 captures an image of an observation target. The light source device 14 emits illumination light for irradiating the observation target. The processor device 16 performs system control of the endoscope system 10. The display 18 is a display unit that displays a display image based on the endoscope image, diagnostic support information, and the like. The keyboard 19 is an input device that performs setting input and the like to the processor device 16 and the like.

[0026] In the present embodiment, the endoscope system 10 has three modes as observation modes: a normal observation mode, a special observation mode, and a diagnostic support mode. In the normal observation mode, a normal observation image with a natural color tone is displayed on the display 18 as a display image by irradiating the observation target with normal light such as white light and then capturing an image. In the special observation mode, a special image that emphasizes a specific structure or the like is displayed on the display 18 as a display image by illuminating the observation target with special light having a different wavelength band or spectral spectrum from the normal light and then capturing an image. In the diagnostic support mode, in addition to displaying the display image on the display 18, diagnostic support information is obtained and notified to a doctor or the like who is a user of the endoscope system 10. The notification of the diagnostic support information is performed by displaying it on the display 18 or by other methods. When displaying on the display 18, for example, it may be performed by superimposing it on the display image or by displaying it separately from the display image on the display 18.

[0027] The endoscope 12 has an insertion portion 12a that is inserted into a subject having an observation target, an operation portion 12b provided at the proximal end portion of the insertion portion 12a, a bending portion 12c provided at the distal end side of the insertion portion 12a, and a distal end portion 12d. By operating the angle knob 12e of the operation portion 12b, the bending portion 12c bends. As a result, the distal end portion 12d faces in a desired direction. In addition to the angle knob 12e, the operation portion 12b is provided with a treatment tool insertion port (not shown), a scope button No. 1 12f, a scope button No. 2 12g, and a zoom operation portion 12h. The treatment tool insertion port is an entrance for inserting treatment tools such as biopsy forceps, a snare, or an electric scalpel. The treatment tool inserted into the treatment tool insertion port protrudes from the distal end portion 12d. Various operations can be assigned to the scope buttons. For example, the scope button No. 1 12f is a freeze button and is used for an operation to acquire a still image. The scope button No. 2 12g is used for an operation to switch the observation mode. By operating the zoom operation portion 12h, the observation target can be enlarged or reduced for imaging.

[0028] As shown in FIG. 2, the light source device 14 includes a light source unit 20 that emits illumination light and a light source processor 22 that controls the operation of the light source unit 20. The light source unit 20 emits illumination light for illuminating the observation target. The illumination light includes light emission such as excitation light used for emitting the illumination light. The light source unit 20 includes, for example, a light source of a laser diode, an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp, and emits at least white illumination light (hereinafter referred to as white light) or excitation light used for emitting white light. White includes so-called pseudo-white that is substantially equivalent to white in imaging the observation target using the endoscope 12.

[0029] The light source unit 20 includes, as necessary, a phosphor that emits light upon receiving excitation light, or an optical filter or the like that adjusts the wavelength band, spectral spectrum, or light quantity of illumination light or excitation light. In addition, the light source unit 20 can emit illumination light composed of at least narrow-band light (hereinafter referred to as narrow-band light). "Narrow-band" means that it is a substantially single wavelength band in relation to the characteristics of the observation target and / or the spectral characteristics of the color filter included in the image sensor (imaging sensor) 45. For example, when the wavelength band is about ± 20 nm or less (preferably about ± 10 nm or less), this light is narrow-band.

[0030] In addition, the light source unit 20 can emit a plurality of types of illumination light having different spectral spectra from each other. The plurality of types of illumination light may include narrow-band light. Further, the light source unit 20 can emit light having a specific wavelength band or spectral spectrum necessary for taking an image used for calculating biological information such as the oxygen saturation of hemoglobin included in the observation target.

[0031] In this embodiment, the light source unit 20 has four-color LEDs of a V-LED 20a, a B-LED 20b, a G-LED 20c, and an R-LED 20d. As shown in FIG. 3, the V-LED 20a emits purple light V with a center wavelength of 405 nm and a wavelength band of 380 to 420 nm. The B-LED 20b emits blue light B with a center wavelength of 460 nm and a wavelength band of 420 to 500 nm. The G-LED 20c emits green light G with a wavelength band ranging from 480 to 600 nm. The R-LED 20d emits red light R with a center wavelength of 620 to 630 nm and a wavelength band ranging from 600 to 650 nm. Note that the center wavelengths of the V-LED 20a and the B-LED 20b have a width of about ± 20 nm, preferably about ± 5 nm to about ± 10 nm. Note that the purple light V is short-wavelength light used to emphasize and display superficial blood vessels, dense portions of superficial blood vessels, submucosal hemorrhage, and subepithelial hemorrhage used in a special observation mode or a diagnostic support mode, and preferably includes 410 nm at the center wavelength or peak wavelength. Further, the purple light V and / or the blue light B are preferably narrow-band light.

[0032] The light source processor 22 controls the timing of turning on, turning off, or shielding each light source constituting the light source unit 20, as well as the light intensity, light emission amount, etc. As a result, the light source unit 20 can emit a plurality of types of illumination light having different spectral spectra at a preset period and light emission amount. In the present embodiment, the light source processor 22 controls the turning on and off of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d, the light intensity or light emission amount during lighting, or the insertion and removal of the optical filter, etc., by inputting independent control signals to each. By independently controlling each of the LEDs 20a to 20d, the light source processor 22 can emit purple light V, blue light B, green light G, or red light R independently while changing the light intensity or the amount of light per unit time. Therefore, the light source processor 22 can emit a plurality of types of illumination light having different spectral spectra from each other, for example, white illumination light, a plurality of types of illumination light having different spectral spectra, or illumination light composed of at least narrow-band light, etc.

[0033] During the normal observation mode, the light source processor 22 controls each of the LEDs 20a to 20d so as to emit white light in which the light intensity ratio among the purple light V, blue light B, green light G, and red light R is Vc:Bc:Gc:Rc. Note that each of Vc, Bc, Gc, or Rc is greater than 0 (zero) and not 0.

[0034] Also, during the special observation mode, the light source processor 22 controls each of the LEDs 20a to 20d so as to emit special light in which the light intensity ratio between the purple light V, blue light B, green light G, and red light R as short-wavelength narrow-band light is Vs:Bs:Gs:Rs. The light intensity ratio Vs:Bs:Gs:Rs is different from the light intensity ratio Vc:Bc:Gc:Rc used during the normal observation mode and is appropriately determined according to the observation purpose. Therefore, the light source unit 20 can emit a plurality of types of special light having different spectral spectra from each other under the control of the light source processor 22. For example, when emphasizing the superficial blood vessels, it is preferable to make Vs larger than the other Bs, Gs, and Rs, and when emphasizing the medium-deep blood vessels, make Gs larger than the other Vs, Bsand it is preferably made larger than Rs.

[0035] In addition, in this specification, the light intensity ratio excluding Vc, Bc, Gc, or Rc includes the case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it includes the case where any one or two or more of each semiconductor light source do not light up. For example, even when only one semiconductor light source is lit and the other three are not lit, such as when the light intensity ratio between violet light V, blue light B, green light G, and red light R is 1:0:0:0, it is considered to have a light intensity ratio.

[0036] Also, in this embodiment, the light source processor 22 preferably automatically switches and emits a plurality of types of illumination lights having mutually different spectral spectra in order to acquire a plurality of types of candidate images in the diagnostic support mode. Each of the plurality of types of illumination lights is preferably emitted repeatedly in a preset order. Therefore, each of the plurality of types of illumination lights preferably forms a specific pattern consisting of a preset order, and the illumination light preferably emits the specific pattern repeatedly.

[0037] For example, specifically, the light source processor 22 emits the first illumination light in the first emission pattern in the first illumination period and emits the second illumination light in the second emission pattern in the second illumination period. The illumination light emitted in the first illumination period is the first illumination light, and the illumination light emitted in the second illumination period is the second illumination light. The first illumination light is preferably white light in order to obtain an endoscope image used for a display image. On the other hand, the second illumination light is preferably special light that can obtain an image suitable for a computer to perform a specific analysis process by illuminating an observation target for use in the recognition process. For example, when performing an analysis process regarding superficial blood vessels, it is preferably made the violet light V. Note that the first illumination light and the second illumination light may each include a plurality of types of illumination lights having mutually different spectral spectra.

[0038] The first light emission pattern is the light emission order of the first illumination light, and the second light emission pattern is the light emission order of the second illumination light. The elements constituting each light emission pattern are frames, which are the units of shooting. A frame refers to a period that at least includes the period from a specific timing in the image sensor 45 to the completion of signal reading. One shooting and one image acquisition are performed in one frame. The first illumination light and the second illumination light emit either one of them and do not emit simultaneously. One light emission cycle consists of at least one first light emission pattern and one second light emission pattern, and the light emission cycle is constituted by combining the first light emission pattern and the second light emission pattern. Illumination is performed by repeating the light emission cycle. Therefore, the light source unit 20 repeatedly emits each of a plurality of types of illumination lights having different spectral spectra in a light emission cycle having a preset order. Details such as the number of frames constituting each of the first light emission pattern or the second light emission pattern, or the types of illumination lights are preset.

[0039] For example, the first light emission pattern is preferably the first A light emission pattern or the first B light emission pattern. As shown in FIG. 4, in the first A light emission pattern, the number of frames FL of the first illumination light L1 in the first illumination period P1 is the same in each first illumination period P1. Therefore, in the light emission cycle Q1, the number of frames FL of the first illumination light L1 in the first illumination period P1 is all set to two. As shown in FIG. 5, in the first B light emission pattern, the number of frames FL of the first illumination period P1 is different in each first illumination period P1. Therefore, in the light emission cycle Q2, the number of frames FL of the first illumination light L1 in the first illumination period P1 includes two cases and three cases. In the first A light emission pattern and the first B light emission pattern, the first illumination light L1 has the same spectral spectrum and is white light.

[0040] The second light emission pattern is preferably the second A light emission pattern, the second B light emission pattern, the second C light emission pattern, or the second D light emission pattern. As shown in FIG. 4, in the second A light emission pattern, the number of frames FL of the second illumination light L2a in the second illumination period P2 is the same in each second illumination period P2. Therefore, in the light emission cycle Q1, the number of frames FL of the second illumination light L2a in the second illumination period P2 is all set to one. Note that the second illumination light L2 may include illumination lights with different spectral spectra, which are distinguished as the second illumination light L2a and the second illumination light L2b, and are collectively referred to as the second illumination light L2 when described as the second illumination light L2. Therefore, in the second A light emission pattern, when the second illumination light L2 is the second illumination light L2b, in the second illumination period P2, the second illumination light L2b is emitted with one frame FL. As shown in FIG. 5, also in the light emission cycle Q2, similar to the light emission cycle Q1, the second illumination light L2 is emitted in the second A light emission pattern.

[0041] As shown in FIG. 6, in the second B light emission pattern, in the light emission cycle Q3, the number of frames FL of the second illumination period P2 is the same in each second illumination period P2, and the spectral spectrum of the second illumination light L2 is the second illumination light L2a or the second illumination light L2b in each second illumination period P2, and is different. As shown in FIG. 7, in the second C light emission pattern, in the light emission cycle Q4, the number of frames FL of the second illumination period P2 is different in each second illumination period P2, and the spectral spectrum of the second illumination light L2 is the second illumination light L2a in each second illumination period P2, and is the same.

[0042] As shown in FIG. 8, in the second D light emission pattern, in the light emission cycle Q5, the number of frames FL of the second illumination period P2 is different in each second illumination period P2, and the spectral spectrum of the second illumination light L2 is the second illumination light L2a or the second illumination light L2b in each second illumination period P2, and is different.

[0043] As described above, in the diagnostic support mode, the light source processor 22 repeats a light emission cycle configured by combining the first light emission pattern and the second light emission pattern. As shown in FIG. 4, the light emission cycle Q1 is composed of the first A light emission pattern and the second A light emission pattern. As shown in FIG. 5, the light emission cycle Q2 is composed of the first B light emission pattern and the second A light emission pattern. As shown in FIG. 6, the light emission cycle Q3 is composed of the first A light emission pattern and the second B light emission pattern. As shown in FIG. 7, the light emission cycle Q4 is composed of the first A light emission pattern and the second C light emission pattern. As shown in FIG. 8, the light emission cycle Q5 is composed of the first A light emission pattern and the second D light emission pattern. In the first light emission pattern, the spectral spectrum of the first illumination light L1 may be different in each first illumination period P1.

[0044] Also, in the diagnostic support mode, the light source processor 22 may change the first light emission pattern or the second light emission pattern based on the analysis processing results of each analysis process described later. The change of the light emission pattern includes the change of the type of illumination light. Specifically, for example, based on the analysis processing results, the second light emission pattern is changed from the second A Light emission pattern to the second B light emission pattern, or the second A light emission pattern using the second illumination light L2a is changed to the second A light emission pattern using the second illumination light L2b, etc. may be switched.

[0045] Here, it is preferable that the first illumination period P1 is longer than the second illumination period P2, and it is preferable that the first illumination period P1 is 2 frames or more. For example, in FIG. 4, in the light emission cycle Q1 in which the first light emission pattern is the first A Light emission pattern and the second light emission pattern is the second A light emission pattern, the first illumination period P1 is 2 frames and the second illumination period P2 is 1 frame. Since the first illumination light L1 is used for generating the display image displayed on the display 18, it is preferable that a bright display image can be obtained by illuminating the observation object with the first illumination light L1 .

[0046] As shown in FIG. 2, the light emitted from each of the LEDs 20a to 20d enters the light guide 41 through an optical path coupling portion (not shown) composed of a mirror, a lens, or the like. The light guide 41 is incorporated in the endoscope 12 and a universal cord (not shown). The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. The light guide 41 propagates the light from the optical path coupling portion to the distal end portion 12d of the endoscope 12.

[0047] At the distal end portion 12d of the endoscope 12, an illumination optical system 30a and an imaging optical system 30b are provided. The illumination optical system 30a has an illumination lens 42, and the illumination light propagated by the light guide 41 is emitted toward the observation target through the illumination lens 42.

[0048] The imaging optical system 30b includes an objective lens 43, a zoom lens 44, and an image sensor 45. The image sensor 45 uses the reflected light of the illumination light returning from the observation target (including, in addition to the reflected light, scattered light, fluorescence emitted by the observation target, or fluorescence caused by a drug administered to the observation target, etc.) through the objective lens 43 and the zoom lens 44 to image the observation target. The zoom lens 44 moves by operating the zoom operation unit 12h to magnify or reduce the observation target image.

[0049] The image sensor 45 has a color filter of one color out of a plurality of color filters for each pixel. In the present embodiment, the image sensor 45 is a color sensor having a color filter of the primary color system. Specifically, the image sensor 45 has an R pixel having a red color filter (R filter), a G pixel having a green color filter (G filter), and a B pixel having a blue color filter (B filter).

[0050] Note that as the image sensor 45, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor can be used. Also, although the image sensor 45 of the present embodiment is a primary color system color sensor, a complementary color system color sensor can also be used. The complementary color system color sensor has, for example, a cyan pixel provided with a cyan color filter, a magenta pixel provided with a magenta color filter, a yellow pixel provided with a yellow color filter, and a green pixel provided with a green color filter. When a complementary color system color sensor is used, the image obtained from the pixels of each color can be converted into an image similar to the image obtained by a primary color system color sensor by performing complementary color - primary color conversion. The same applies when the sensor of the primary color system or the complementary color system has one or more types of pixels having characteristics other than those described above, such as a W pixel (a white pixel that receives light in almost the entire wavelength band). Also, although the image sensor 45 of the present embodiment is a color sensor, a monochrome sensor without a color filter may be used.

[0051] The endoscope 12 includes an imaging processor 46 that controls the image sensor 45. The control of the imaging processor 46 differs for each observation mode. In the normal observation mode, the imaging processor 46 controls the image sensor 45 to capture an observation target illuminated with normal light. As a result, a Bc image signal is output from the B pixel of the image sensor 45, a Gc image signal is output from the G pixel, and an Rc image signal is output from the R pixel.

[0052] In the special observation mode, the imaging processor 46 is controls the image sensor 45 to capture an observation target illuminated with special light. As a result, a Bs image signal is output from the B pixel of the image sensor 45, a Gs image signal is output from the G pixel, and an Rs image signal is output from the R pixel.

[0053] In the diagnosis support mode, the imaging processor 46 isThe image sensor 45 is controlled to photograph an observation object illuminated by the first illumination light L1 or the second illumination light L2. As a result, for example, when the first illumination light L1 is illuminated, a B1 image signal is output from the B pixel of the image sensor 45, a G1 image signal is output from the G pixel, and an R1 image signal is output from the R pixel. Also, when the second illumination light L2 is illuminated, for example, a B2 image signal is output from the B pixel of the image sensor 45, a G2 image signal is output from the G pixel, and an R2 image signal is output from the R pixel.

[0054] In the processor device 16, programs related to processes performed by a central control unit 51, an image acquisition unit 52, an image processing unit 56, a display control unit 57, etc., as described later, are incorporated in a memory (not shown). The program operates by a central control unit 51 constituted by an image processor included in the processor device 16 that functions as an image processing device, thereby realizing the functions of the central control unit 51, the image acquisition unit 52, the image processing unit 56, and the display control unit 57.

[0055] The central control unit 51 performs overall control of the endoscope system 10, such as synchronous control of the irradiation timing of the illumination light and the imaging timing. When various settings are input using the keyboard 19 or the like, the central control unit 51 inputs the settings to each part of the endoscope system 10, such as the light source processor 22, the imaging processor 46, or the image processing unit 56.

[0056] The image acquisition unit 52 acquires an image of the observation object photographed using pixels of each color from the image sensor 45, that is, a RAW image. Also, the RAW image is an image (endoscope image) before demosaicing processing is performed. As long as it is an image before demosaicing processing, an image obtained by performing any processing such as noise reduction processing on the image acquired from the image sensor 45 is also included in the RAW image.

[0057] The image acquisition unit 52 includes a DSP (Digital Signal Processor) 53, a noise reduction unit 54, and a conversion unit 55 in order to perform various processes on the acquired RAW image as needed.

[0058] The DSP 53 includes, for example, an offset processing unit, a defect correction processing unit, a demosaicing processing unit, a linear matrix processing unit, a YC conversion processing unit, etc. (none of which are shown). The DSP 53 performs various processes on the RAW image or an image generated using the RAW image using these units.

[0059] The offset processing unit performs offset processing on the RAW image. The offset processing is a process of reducing the dark current component from the RAW image and setting an accurate zero level. The offset processing may be referred to as clamp processing. The defect correction processing unit performs defect correction processing on the RAW image. The defect correction processing is a process of correcting or generating the pixel value of the RAW pixel corresponding to the defective pixel of the image sensor 45 when the image sensor 45 includes pixels (defective pixels) having defects due to the manufacturing process or changes over time.

[0060] The demosaicing processing unit performs demosaicing processing on the RAW images of each color corresponding to each color filter. The demosaicing processing is a process of generating missing pixel values by interpolation in the RAW image due to the arrangement of the color filters. The linear matrix processing unit performs linear matrix processing on the endoscope image generated by allocating one or more RAW images to the RGB color channels. The linear matrix processing is a process of enhancing the color reproducibility of the endoscope image. The YC conversion processing performed by the YC conversion processing unit is a process of converting the endoscope image generated by allocating one or more RAW images to the RGB color channels into an endoscope image having a luminance channel Y and color difference channels Cb and Cr.

[0061] The noise reduction unit 54 performs noise reduction processing on the endoscope image having the luminance channel Y, the color difference channel Cb, and the color difference channel Cr using, for example, the moving average method or the median filter method. The conversion unit 55 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into an endoscope image having BGR color channels again.

[0062] The image processing unit 56 performs necessary image processing or calculations on the endoscopic image output by the image acquisition unit 52. As shown in FIG. 9, the image processing unit 56 includes a normal observation image processing unit 61, a special observation image processing unit 62, and a diagnostic support image processing unit 63. The normal observation image processing unit 61 performs image processing for normal observation images on the input Rc image signal, Gc image signal, and Bc image signal for one frame. The image processing for normal observation images includes 3×3 matrix processing, tone conversion processing, color conversion processing such as 3D LUT (Look Up Table) processing, color enhancement processing, or structure enhancement processing such as spatial frequency enhancement. The Rc image signal, Gc image signal, and Bc image signal subjected to the image processing for normal observation images are normal observation images, and are input to the display control unit 57 as display images in the normal observation mode.

[0063] The special observation image processing unit 62 performs image processing for special observation images on the input Rs image signal, Gs image signal, and Bs image signal for one frame. The image processing for special observation images includes 3×3 matrix processing, tone conversion processing, color conversion processing such as 3D LUT (Look Up Table) processing, color enhancement processing, or structure enhancement processing such as spatial frequency enhancement. The Rs image signal, Gs image signal, and Bs image signal subjected to the image processing for special observation images are special observation images, and are input to the display control unit 57 as display images in the special observation mode.

[0064] The diagnostic support image processing unit 63 performs image analysis processing and the like in the diagnostic support mode and generates diagnostic support information. The diagnostic support information is shown to users such as doctors. As shown in FIG. 10, the diagnostic support image processing unit 63 includes a candidate image acquisition unit 71, a first analysis processing unit 72, an optimal image selection unit 73, a second analysis processing unit 74, and a display image generation unit 75.

[0065] The candidate image acquisition unit 71 generates and acquires a plurality of types of candidate images based on the endoscopic image output by the image acquisition unit 52. The types of candidate images are distinguished by either or both of the following two points. The first point is to distinguish by the spectral spectrum of the illumination light when photographing the observation target. Therefore, the candidate image acquisition unit 71 acquires, as one type of candidate image each, the endoscopic images obtained by photographing the observation target illuminated by each of a plurality of types of illumination lights emitted by the light source unit and having different spectral spectra. The second point is to distinguish by the method of image processing (hereinafter referred to as candidate image generation image processing) for generating candidate images with respect to the endoscopic image.

[0066] As a method of candidate image generation image processing, for example, it includes methods of image processing such as enhancement processing, and specifically, Color difference It includes expansion processing and / or structure enhancement processing, etc. When distinguishing candidate images by the method of candidate image generation image processing, it includes not performing candidate image generation image processing. Therefore, the endoscopic image for which candidate image generation image processing is not performed on the endoscopic image output by the image acquisition unit 52 in the case of is also one type of candidate image. Therefore, even when the combination of the spectral spectrum of the illumination light and the candidate image generation image processing is different, it is regarded as one type of candidate image. Candidate images with different spectral spectra of the illumination light or different image processing are different types of candidate images.

[0067] As shown in FIG. 11, the candidate image acquisition unit 71 includes each candidate image generation unit that generates each of a plurality of types of candidate images. For example, it includes a first candidate image generation unit 81, a second candidate image generation unit 82, a third candidate image generation unit 83, a fourth candidate image generation unit 84, a fifth candidate image generation unit 85, and an nth candidate image generation unit 86. n is an integer of 6 or more. n can be set according to the number of types of a plurality of types of candidate images. Each candidate image acquisition unit performs the following illumination light and / or candidate image generation image processing respectively.

[0068] The first candidate image generation unit 81 performs first candidate image processing (hereinafter referred to as first image processing) for generating a first candidate image. The first image processing is the spectral spectrum for the first illumination lightby This is a process performed on the B1 image signal, G1 image signal, and R1 image signal obtained by emitting the first illumination light of white light. The first image process is the same as the normal display image process in the normal observation image processing unit 61, and a first candidate image similar to the normal display image is obtained. The first candidate image is one type of candidate image. Therefore, the candidate image acquisition unit 71 acquires, as one type of candidate image, an image obtained by photographing an observation target illuminated by white illumination light.

[0069] The second candidate image generation unit 82 performs a second candidate image processing for generating a second candidate image (hereinafter referred to as the second image process). The second image process is a process performed on the B2 image signal, G2 image signal, and R2 image signal obtained by emitting the second illumination light L2 with the second illumination light spectral spectrum SP1. As shown in FIG. 12, the second illumination light L2 emitted by the second illumination light spectral spectrum SP1 is preferably light in which the peak intensity of purple light V is greater than that of other colors of blue light B, green light G, and red light R. The second image process is a pseudo-color process that assigns the B2 image signal to the B channel and G channel for display and assigns the G2 image signal to the R channel for display. By this pseudo-color process, a second candidate image in which blood vessels or structures at a specific depth such as superficial blood vessels are emphasized is obtained. The second candidate image is one type of candidate image.

[0070] The third candidate image generation unit 83 performs a third candidate image processing for generating a third candidate image (hereinafter referred to as the third image process). The third image process is a process performed on the B2 image signal, G2 image signal, and R2 image signal obtained by emitting the second illumination light with the second illumination light spectral spectrum SP2. As shown in FIG. 13, the second illumination light emitted by the second illumination light spectral spectrum SP2 is preferably light that emits only purple light V (the peak wavelength is, for example, 400 to 420 nm). The third image process is a process of assigning the B2 image signal to the B channel, G channel, and R channel for display and adjusting the color tone and gradation balance. By the third image process, a third candidate image in which extremely superficial blood vessels shallower than the superficial blood vessels are emphasized is obtained. The third candidate image is one type of candidate image.

[0071] The fourth candidate image generation unit 84 performs fourth candidate image processing (hereinafter referred to as fourth image processing) for generating a fourth candidate image. The fourth image processing is a process performed on the B1 image signal, G1 image signal, and R1 image signal obtained by emitting the first illumination light, in addition to the B2 image signal, G2 image signal, and R2 image signal obtained by emitting the second illumination light with the second illumination light spectral spectrum SP3. As shown in FIG. 14, the second illumination light spectral spectrum SP3 is light having a wavelength band with a difference in absorption coefficients between oxyhemoglobin and deoxyhemoglobin Blue light B (the peak wavelength is, for example, 470 to 480 nm) and is preferably such.

[0072] As shown in FIG. 15, the fourth candidate image generation unit 84 includes an oxygen saturation signal ratio calculation unit 84a that performs a signal ratio calculation process for calculating a first signal ratio (B2 / G1) representing the ratio of the B2 image signal to the G1 image signal and a second signal ratio (R1 / G1) representing the ratio of the R1 image signal to the G1 image signal, an oxygen saturation calculation unit 84c that refers to the oxygen saturation calculation table 84b and calculates the oxygen saturation corresponding to the first signal ratio and the second signal ratio, and an oxygen saturation image generation unit 84d that generates an oxygen saturation image based on the oxygen saturation. The oxygen saturation image becomes the fourth candidate image obtained by the fourth image processing. The fourth candidate image is one type of candidate image.

[0073] Note that the oxygen saturation calculation table 84b stores the correlation between the oxygen saturation, the first signal ratio, and the second signal ratio. Specifically, as shown in FIG. 16, the oxygen saturation calculation table 84b is a two-dimensional table that defines isoclines ELx, EL1, EL2, EL3, Ely, etc. of oxygen saturation in a two-dimensional space with the first signal ratio (B2 / G1) and the second signal ratio (R1 / G1) as axes. For example, the isocline ELx represents an oxygen saturation of 0%, the isocline EL1 represents an oxygen saturation of 30%, the isocline EL2 represents an oxygen saturation of 50%, and the isocline EL3 represents an oxygen saturation of 80%. Note that the positions and shapes of the isoclines with respect to the first signal ratio (B2 / G1) and the second signal ratio (R1 / G1) are obtained in advance by physical simulation of light scattering. Note that the first signal ratio (B2 / G1) and the second signal ratio (R1 / G1) are preferably on a log scale.

[0074] The fifth candidate image generation unit 85 performs fifth candidate image processing (hereinafter referred to as fifth image processing) for generating a fifth candidate image. The fifth image processing is Color difference an expansion process, specifically, a process performed on the B2 image signal, the G2 image signal, and the R2 image signal obtained by emitting the second illumination light with the second illumination light spectroscopic spectrum SP4. As shown in FIG. 17, the second illumination light spectroscopic spectrum SP4 is preferably light in which the peak intensities of the purple light V and the blue light B are greater than the peak intensities of the green light G and the red light R. Also, compared with the Spectral analysis spectrum SP2 of the second illumination light, it is preferable that the intensity of the red light R is greater.

[0075] to the fifth candidate image generation unit 85As shown in FIG. 18, a signal ratio calculation unit 85a for color difference expansion that performs a signal ratio calculation process for calculating a first signal ratio (B2 / G2) representing the ratio of the B2 image signal to the G2 image signal and a second signal ratio (G2 / R2) representing the ratio of the R2 image signal to the G2 image signal, a color difference expansion processing unit 85b that performs a color difference expansion process for expanding the color difference between a plurality of observation target ranges based on the first signal ratio and the second signal ratio, and a color difference expansion image generation unit 85c that generates a color difference expansion image based on the first signal ratio and the second signal ratio after the color difference expansion process are provided. The color difference expansion image becomes a fifth candidate image obtained by the fifth image processing. The fifth candidate image is one type of candidate image.

[0076] Regarding the color difference expansion process, as shown in FIG. 19, it is preferable to expand the distance between a plurality of observation target ranges in a two-dimensional space composed of the first signal ratio (B2 / G2) and the second signal ratio (G2 / R2). Specifically, in the two-dimensional space, while maintaining the position of the first range (indicated by 1 surrounded by a circle) among the plurality of observation target ranges, it is preferable to expand the distance between the first range and the second range (indicated by 2 surrounded by a circle), the distance between the first range and the third range (indicated by 3 surrounded by a circle), and the distance between the first range and the fourth range (indicated by 4 surrounded by a circle). The color difference expansion process is preferably performed by a method of adjusting the radial distance and the angle after performing polar coordinate conversion on the first signal ratio and the second signal ratio. Note that the first range is a normal part where no lesion or the like exists, and the second to fourth ranges are preferably abnormal parts where a lesion or the like may exist. Due to the color difference expansion process, the range A1 in the two-dimensional space before the color difference expansion process is expanded to the range A2 after the color difference expansion process, so the color difference is emphasized, and for example, an image in which the color difference between the abnormal part and the normal part is emphasized is obtained. before the color difference expansion process

[0077] As described above, a plurality of types of candidate images are generated by performing image processing of various methods on the endoscopic image. The n-th candidate image generation unit 86 generates the n-th type of candidate image. The method or content of the image processing is not limited to the above. For example, in addition to the color difference expansion processing, enhancement processing such as structure enhancement processing may be performed. The types of candidate images are distinguished according to the presence or absence of enhancement processing on the endoscopic image or the type of enhancement processing, and the distinguished candidate images are each obtained as one type of candidate image. Note that the endoscopic image on which the enhancement processing is performed may be the one after any one of the first to n-th image processes or the one without performing the image process.

[0078] The structure enhancement processing is a process performed on the acquired endoscopic image so that the blood vessels in the observation target are represented as an emphasized endoscopic image. Specifically, as the endoscopic image, any one of the B1 image signal, G1 image signal, and R1 image signal obtained by emitting the first illumination light, or the B2 image signal, G2 image signal, and R2 image signal obtained by emitting the second illumination light is used. In the structure enhancement processing, in the acquired endoscopic image, a density histogram, which is a graph with the pixel value (luminance value) on the horizontal axis and the frequency on the vertical axis, is obtained and stored in advance in a memory (not shown) of the image processing unit 56 or the like. Tone correction is performed using a tone correction table. The tone correction table has a tone correction curve representing the input value on the horizontal axis and the output value on the vertical axis, and shows the correspondence between the input value and the output value. For example, tone correction is performed based on a substantially S-shaped tone correction curve to widen the dynamic range of the acquired endoscopic image. As a result, in the original image before the enhancement processing of the structure enhancement, the portion with a low density becomes even lower in density, and the portion with a high density becomes even higher. For example, the density difference between the blood vessel region and the region where no blood vessels exist widens, and the contrast of the blood vessels is improved. Therefore, the endoscopic image processed by the structure enhancement processing has improved contrast of the blood vessels and enhanced visibility of the blood vessel structure, and can be preferably used, for example, for determination of a region with a high blood vessel density as a specific region more easily and with high accuracy.

[0079] Further, the candidate image acquisition unit 71 illuminates with illumination light including narrow-band light in a preset wavelength band emitted by the light source unit 20Observation It is preferable to obtain an endoscopic image obtained by photographing the object as one type of candidate image. Therefore, it is preferable that the plurality of types of candidate images include at least one endoscopic image obtained by illumination light composed of narrow-band light. An endoscopic image obtained by photographing an observation object illuminated by illumination light including purple light V and / or blue light B, which is preferably narrow-band light, may be generated as one type of candidate image.

[0080] In addition, as the narrow band of the narrow-band light, it is preferably a short wave of 480 nm or less. Further, it is preferable that the central wavelength or peak wavelength of the narrow-band light includes a wavelength of 410 nm. Further, the narrow-band light is preferably monochromatic light having only one narrow band. Further, it is preferable to obtain, as one type of candidate image, an endoscopic image obtained by coloring an endoscopic image mainly composed of narrow-band light.

[0081] The endoscopic image obtained by coloring the endoscopic image mainly composed of narrow-band light is obtained, for example, by a method of generating a color image from a specific color image by photographing an observation object with a specific monochromatic light, assigning the specific color image to a plurality of color channels, and adjusting the balance of each color channel. In this case, the coloring is L * a * b * In the color space, it is preferable to expand the distance between the color of the relatively low-frequency component among the observation object images representing the observation object and the color of the relatively high-frequency component among the observation object images. The candidate image based on such an endoscopic image can be made into an endoscopic image in which fine specific structures such as blood vessels are easier to grasp, for example, in a zoom image or the like, by adjusting the coloring corresponding to the observation object image. In addition to making the endoscopic image easy for humans to visually recognize, by adjusting the coloring, it can be made into an endoscopic image that gives good analysis results for computer-aided analysis processing, which is preferable.

[0082] Illumination light including specific narrow-band light can obtain a candidate image in which specific structures such as blood vessels existing at a specific depth of a specific mucous membrane, blood vessels of a specific thickness, or glandular ducts are emphasized. Note that the emphasis in the candidate image includes not only the emphasis on human vision but also the emphasis for a computer to perform CAD or the like. Therefore, it is preferable that the candidate image is emphasized so that good analysis processing results can be obtained when using CAD or the like. The generated multiple types of candidate images are sent to the first analysis processing unit 72. As shown in FIG. 20, for example, three types of candidate images, i.e., a first candidate image, a second candidate image, and a fifth candidate image, are acquired. In FIG. 20 and the like, descriptions for explaining each process described in the right column are given in the left column with "Candidate image acquisition:" or the like.

[0083] The first analysis processing unit 72 performs a first analysis process on one or more types of candidate images preset among the multiple types of candidate images. The number of types of candidate images for which the first analysis process is performed is set arbitrarily. When k types of candidate images are acquired, the type of candidate image for which the first analysis process is performed can be any one from 1 to k. Here, k is an integer of 2 or more. The type of candidate image for which the first analysis process is performed can also be preset.

[0084] The first analysis result obtained by the first analysis process is used to select at least one type of candidate image from the multiple types of candidate images. The second analysis processing unit 74 performs a second analysis process on the selected candidate image. At least one type of candidate image selected as the object for the second analysis processing unit 74 to perform the second analysis process is defined as the optimal image. By the second analysis processing unit 74 performing the second analysis process on the optimal image, a second analysis result is obtained. Since the second analysis result is notified to the user as the final diagnostic support information, it is preferable that the first analysis result can select an optimal image that can obtain good results by the second analysis process. The first analysis result is preferably diagnostic support information. The diagnostic support information based on the first analysis result is defined as the first diagnostic support information. One or more pieces of first diagnostic support information are obtained corresponding to the number of candidate images on which the first analysis process is performed.

[0085] The first analysis result is information based on the candidate image. For example, Tip in addition to information obtained from the candidate image such as the distance between the 12d and the observation target, or the brightness of the entire candidate image or a specific region, it includes the name of the subject such as mucosa included in the observation target, the name of the site, the name of the disease, the name of a specific structure, or the name of an object not derived from a living body such as a treatment instrument. Regarding a lesion or a disease, it can be the presence or absence, an index value, a position or region, a boundary line with a normal region, a probability, a degree of progression, or a severity. Also, it can be a specific state such as a pathological state, bleeding, or a treatment scar in the observation target shown in the candidate image. The site name is preferably a characteristic site shown in the candidate image. For example, in the upper digestive tract, it can be the esophagus, the cardiac part, the gastric fundus, the gastric body, the pylorus, the gastric angle, or the duodenal bulb, etc. In the large intestine, it can be the cecum, the ileocecal part, the ascending colon, the transverse colon, the descending colon, the sigmoid colon, or the rectum, etc. Specific structures include blood vessels, glandular ducts, raised parts such as polyps or cancers, or depressed parts, etc. Objects not derived from a living body include treatment instruments such as biopsy forceps, snares, or foreign body removal devices that can be attached to an endoscope, or treatment instruments for the abdomen used in laparoscopic surgery. Examples of the name of a lesion or a disease include lesions or diseases found in endoscopic examinations of the upper digestive tract or the large intestine, such as inflammation, erythema, bleeding, ulcers, or polyps, or gastritis, Barrett's esophagus, cancer, or ulcerative colitis, etc. The value of the biological information is the value of the biological information of the observation target, such as oxygen saturation, vascular density, or the value of fluorescence by a pigment, etc.

[0086] Also, the first diagnostic support information may be a determination or discrimination result. The determination or discrimination may be various scores such as discrimination between a tumor and non-tumor, the stage or severity of various diseases, the Mayo Score, or the Geboes Score, etc.

[0087] The Mayo score is a score indicating the endoscopic severity of ulcerative colitis. Based on the findings of the affected area in the large intestine using an endoscope, it is determined into any one of mild cases of grade 0 and 1, moderate cases of grade 2, or severe cases of grade 3 according to the presence and degree of disease characteristics, etc. For example, grade 0 is denoted as Mayo0. Therefore, the diagnostic support information is any one from Mayo0 to Mayo3.

[0088] Also, the Geboes score is a score indicating the pathological severity of ulcerative colitis. Based on the findings of the biopsy tissue using a microscope, it is determined into any one from the mild stage of Geboes0 to the pathological remission from Geboes0 to Geboes2A or the pathological non-remission from Geboes2B to Geboes5 according to the presence and degree of disease characteristics, etc. Therefore, the diagnostic support information is any one from Geboes0 to Geboes5, or either Geboes2A or Geboes2B.

[0089] Also, for example, the stage in gastric cancer is comprehensively determined based on the observation of the lesion and biopsy, etc., and classified into stages I to IV. Therefore, the diagnostic support information is any one from stage I to stage IV.

[0090] Also, the first analysis processing result includes imaging conditions such as the electronic zoom ratio obtained from the candidate image. Also, in some cases, it may be information from an information management server such as HIS (Hospital Information System) or RIS (Radiology Information System) through communication, or an image server such as PACS (Picture Archiving and Communication System for medical application). Also included is the accuracy, etc., of the first analysis processing result itself obtained by the image analysis processing.

[0091] The first analysis processing unit 72 may perform the first analysis processing on a plurality of types of candidate images by the same method, or may perform the first analysis processing on each type of the plurality of types of candidate images by different methods. This is because depending on the type of candidate image, the type of first analysis processing result that can obtain good results by the image analysis processing may be different. By performing the first analysis processing for each type of candidate image, it is possible to perform an image analysis processing suitable for the candidate image, and finally it is preferable because an optimal image with higher accuracy for obtaining diagnostic support information can be selected. The first analysis processing performed for each type of candidate image is preferably implemented independently and in parallel.

[0092] In this case, as shown in FIG. 21, the first analysis processing unit 72 includes first analysis processing units such as a first image first analysis processing unit 91, a second image first analysis processing unit 92, a third image first analysis processing unit 93, a fourth image first analysis processing unit 94, a fifth image first analysis processing unit 95, and an nth image first analysis processing unit 96 provided for each type of candidate image. n is an integer of 6 or more, and includes first analysis processing units for each image in a number corresponding to the number of types of candidate images. The first image first analysis processing unit 91 performs the first analysis processing on the first candidate image. Similarly, the second image first analysis processing unit 92 performs the first analysis processing on the second candidate image, the third image first analysis processing unit 93 performs the first analysis processing on the third candidate image, the fourth image first analysis processing unit 94 performs the first analysis processing on the fourth candidate image, the fifth image first analysis processing unit 95 performs the first analysis processing on the fifth candidate image, and the nth image first analysis processing unit 96 performs the first analysis processing on the nth candidate image. For example, when it is set in advance that the objects for which the first analysis processing is to be performed are three types, namely, the first candidate image, the second candidate image, and the fifth candidate image, when three types of candidate images, namely, the first candidate image, the second candidate image, and the fifth candidate image, are acquired, the first analysis processing is performed in three first analysis processing units, namely, the first image first analysis processing unit 91, the second image first analysis processing unit 92, and the fifth image first analysis processing unit 95.

[0093] Note that when the first analysis processing is performed on a plurality of types of candidate images by the same method, each first analysis processing unit may perform the analysis processing on different types of candidate images. That is, eachFirst The analysis processing unit may be commonly used for candidate images of different types.

[0094] As a method of the first analysis processing, a method by which first diagnostic support information is obtained as a first analysis processing result can be used. For example, a method using values based on an image such as pixel values and / or luminance values of a candidate image, a method using values of biological information such as oxygen saturation or vascular density calculated from an image, a method using information such as shooting conditions included in a candidate image, or a method using correspondence information in which a specific state in an observation target and a candidate image obtained by shooting the observation target including the specific state are associated in advance, and the like can be mentioned.

[0095] Note that the specific state in the observation target can be the same as an example of the first diagnostic support information. The first analysis processing unit 72 preferably includes a correspondence information acquisition unit (not shown) that acquires correspondence information in which a specific state of an observation target and a candidate image obtained by shooting the observation target in the specific state are associated in advance. The correspondence information is information in which, when the specific state of the observation target is known in advance, a candidate image obtained by shooting this observation target is associated with information such as the specific state of the observation target or the area of the specific state. The first analysis processing unit 72 or each first analysis processing unit preferably performs the first analysis processing on a newly acquired candidate image based on the correspondence information.

[0096] As shown in FIG. 22, by inputting a newly acquired candidate image whose specific state is unknown to the correspondence information acquisition unit, the specific state in the newly acquired candidate image can be estimated and output as a first analysis processing result using the correspondence information in which the candidate image included in the correspondence information acquisition unit and the specific state of the observation target are associated. Further, the correspondence information acquisition unit may perform learning to further acquire, as correspondence information, each newly acquired candidate image and the specific state included in the first analysis processing result output by estimation.

[0097] The correspondence information is preferably provided in each of the first analysis processing units from the first image first analysis processing unit 91 to the nth image first analysis processing unit 96. By providing a correspondence information acquisition unit associated with a specific state of a specific type for each type of candidate image, various types of candidate images can obtain good results through image recognition processing.

[0098] For example, when the type of the candidate image is the second candidate image which is a candidate image emphasizing blood vessels, the first analysis Process is performed in the second image first analysis processing unit 92. The second image first analysis processing unit 92 includes a correspondence information acquisition unit including correspondence information regarding a specific state regarding the blood vessels of the observation target. The correspondence information acquisition unit performs the first analysis processing of the second candidate image based on the correspondence information, and outputs details such as regions regarding the specific state of the observation target included in this second candidate image. Note that the output of details regarding the specific state also includes contents such as "not including the specific state".

[0099] Each correspondence information acquisition unit is, for example, a learned model in machine learning. Since the specific state of the observation target in the newly acquired candidate image can be obtained as the first analysis processing result more quickly and with high accuracy, it is preferable to perform the first analysis processing using the learned model in machine learning as the correspondence information acquisition unit. In the present embodiment, as each correspondence information acquisition unit, the first analysis processing for outputting the specific state of the observation target is performed using the learned model in machine learning. Note that in this case, it is preferable to use the learned models learned for each type of candidate image in order to obtain good analysis processing results. Therefore, for example, it is preferable that the correspondence information included in the first image first analysis processing unit 91 and the correspondence information included in the second image first analysis processing unit 92 are different learned models from each other.

[0100] As shown in FIG. 23, for example, in three first analysis processing units, namely the first image first analysis processing unit 91, the second image first analysis processing unit 92, and the fifth image first analysis processing unit 95, the first analysis processing of the first candidate image, the second candidate image, and the fifth candidate image is respectively performed using the learned model. PerformFirst diagnostic support information is obtained from each of the first analysis processing units, and a total of three pieces of first diagnostic support information are obtained.

[0101] Based on the first analysis result obtained by the first analysis process, the optimal image selection unit 73 selects at least one type of candidate image as the optimal image from the plurality of types of candidate images acquired by the candidate image acquisition unit 71. Since the first analysis process is performed on one or more types of candidate images, one or more pieces of first diagnostic support information, which are the first analysis results, are obtained corresponding to the number of candidate images on which the first analysis process is performed. When a plurality of pieces of first diagnostic support information are obtained, the combined information is used as the first diagnostic support information.

[0102] As a method for selecting the optimal image, various methods can be used. For example, a correspondence table associating the first diagnostic support information with the type of candidate image that is most preferable for the second analysis process among the plurality of types of candidate images can be prepared in advance and used.

[0103] When the light source unit 20 repeats the light emission cycle, it is preferable that the optimal image selection unit 73 selects at least one optimal image from the plurality of types of candidate images obtained in one light emission cycle. This is preferable because every time the light emission cycle is switched, the latest optimal image is always selected and the latest diagnostic support information is obtained.

[0104] As shown in FIG. 24, for example, when three types of candidate images are acquired and three pieces of first diagnostic support information are obtained, the first diagnostic support information includes the name of the subject to be observed and Tip the distance between 12d and the object to be observed. Specifically, assume that all three pieces of first diagnostic support information have a subject name of "mucosa" and Tip the distance between 12d and the object to be observed is "distant view". The optimal image selection unit 73 synthesizes the three pieces of first diagnostic support information, and assuming that the subject name is "mucosa" and Tip the distance between 12d and the object to be observed is "distant view", using a correspondence table (not shown) provided in the optimal image selection unit 73, the first candidate image of one type of candidate image is selected as the optimal image.

[0105] The second analysis processing unit 74 obtains a second analysis processing result by performing second analysis processing on the optimal image. The second analysis processing result is notified to the user as the final diagnostic support information. The second analysis processing result is preferably diagnostic support information. The diagnostic support information based on the second analysis processing result is defined as the second diagnostic support information. Since the second analysis processing is usually performed on the optimal image, which is one selected candidate image, one piece of second diagnostic support information can be obtained.

[0106] The details of the second diagnostic support information can be the same as those of the first diagnostic support information. Further, the second analysis processing unit 74 may perform the second analysis processing by different methods for each type of candidate image that is set as the optimal image. This is because, similar to the first analysis processing, depending on the type of candidate image, the types of second diagnostic support information that can obtain good results by image analysis processing may be different. It is preferable to perform the second analysis processing for each type of candidate image because the second diagnostic support information can be obtained with higher accuracy.

[0107] In this case, as shown in FIG. 25, the second analysis processing unit 74 includes a first image second analysis processing unit 101, a second image second analysis processing unit 102, a third image second analysis processing unit 103, a fourth image second analysis processing unit 104, a fifth image second analysis processing unit 105, and an nth image second analysis processing unit 106 provided for each type of candidate image. n is an integer of 6 or more, and the second analysis processing unit for each image corresponding to the number of types of candidate images is provided. The first image second analysis processing unit 101 performs the second analysis processing when the optimal image is the first candidate image. The same applies to the second image second analysis processing unit 102 and subsequent units.

[0108] As the method of the second analysis process, a method for obtaining second diagnostic support information as the result of the second analysis process can be used, and it can be the same as the method of the first analysis process. In some cases, the first analysis processing unit 72 may also serve as the second analysis processing unit 74. However, in order to quickly obtain the second diagnostic support information, it is preferable to perform the first analysis process and the second analysis process independently. Further, it is preferable that the first analysis process and the second analysis process are analysis processes with different contents. This is because when diagnostic support information with the same content is obtained by adopting analysis processes with the same content, the same analysis process will be performed twice, which may require wasted time.

[0109] In addition, the second diagnostic support result is preferably a determination regarding the disease or Discrimination a result. The second diagnostic support result is notified to a user such as a doctor as the final diagnostic support result. The doctor and the like perform an endoscopy while diagnosing the observation target with reference to this second diagnostic support result. The second diagnostic support result is preferably, for example, an index value regarding the disease, an index value regarding the stage of the disease, the stage of the disease, the severity of the disease, the pathological state of the observation target, or the disease location.

[0110] The second diagnostic support information is, for example, the Mayo score which is an index of the endoscopic severity of ulcerative colitis. In this case, the second diagnostic support information is any one of 0 to 3 of the Mayo score. Further, for example, the second diagnostic support information is the Geboes score which is an index of the pathological stage of ulcerative colitis. In this case, the second diagnostic support information is any one of Geboes 0 to Geboes 5, or Geboes 2A or Geboes 2B. Further, for example, the second diagnostic support information is the stage in gastric cancer. Therefore, in this case, the second diagnostic support information is any one of stages I to IV.

[0111] The severity or progression of a disease serves as an important factor for judgment in determining treatment strategies, etc. In particular, for ulcerative colitis, etc., where medical treatment according to severity is central, accurate diagnosis at an early stage of onset is important. Therefore, it is beneficial to determine the severity with high accuracy through endoscopic examination using an endoscopic system 10 or the like.

[0112] Also, the first analysis process and the second analysis process are preferably performed in combination according to preset contents. The candidate image selected by the first analysis process result preferably obtains second diagnosis support information with higher accuracy as the second analysis process result in the second analysis process. Therefore, there may be cases where the contents of the preferable first analysis process and the second analysis process can be preset. For example, when the first analysis process result by the first analysis process is the name of the subject to be observed and Tip the distance between 12d and the observation target, in the case of a short distance, there is a possibility of performing detailed observation such as a target area, and since it may be an endoscopic image showing the fine structure of the mucosal surface, etc., the second analysis process preferably adopts a type of analysis process method that obtains, as the second analysis process result, biological information of the disease, determination of the stage or severity, or the area of the lesion including the boundary line with the normal part. On the other hand, in the case of a long distance, there is a possibility of performing screening to observe the overall properties, and since it may be an endoscopic image showing the overall state of the site, it is preferable to adopt a type of analysis process method that obtains, as the second analysis process result, the site name or the area of the lesion or disease, etc.

[0113] Note that Tip when the distance between 12d and the observation target is a short distance, the candidate image may be an endoscopic image obtained by magnified observation using zoom. As shown in FIG. 26, for example, when the first candidate image is selected as the optimal image, the first image second analysis processing unit 101 performs the second analysis process. As a result of the second analysis process, second diagnosis support information, which is the second analysis process result, is obtained.

[0114] The display image generation unit 75 generates a display image to be displayed on the display 18. The display image is an endoscopic image based on at least one type of candidate image among a plurality of types of candidate images, and the display image is displayed on the display 18. Which type of candidate image the display image is to be can be set in advance. Since the display image to be displayed on the display is for a user such as a doctor to view and make a diagnosis or determine an examination policy, etc., it is preferably an endoscopic image with good visibility for humans. For example, as shown in FIG. 27, when the display image is the first candidate image of the same type as the normal observation image using white light, the display image generation unit 75 performs necessary image processing in the same manner as the normal observation image processing unit 61 to generate the display image. The generated display image is sent to the display control unit 57.

[0115] In the case of the normal observation mode, the display control unit 57 displays the normal observation image on the display 18, and in the case of the special observation mode, it displays the special observation image on the display 18. Also, in the case of the diagnosis support mode, it performs control to display the display image on the display 18. The display image is preferably continuously displayed on the display 18. In addition, in the diagnosis support mode, although it is not displayed as the display image, other acquired candidate images may be switched according to an instruction and displayed as the display image on the display 18. As shown in FIG. 28, for example, the display image 111 is displayed on the display 18 together with the mode name display 112 which is the current mode name, or the endoscopic image type name display 113 indicating the type of the candidate image from which the display image is derived.

[0116] By configuring as described above, the processor device 16 that functions as an image processing device or the endoscope system 10 including the image processing device obtains first diagnostic support information from candidate images of a preset type among a plurality of types of candidate images, and then uses the first diagnostic support information to determine an optimal image for performing a second analysis process. Compared with the case of selecting an image for performing an analysis process without selecting an endoscope image obtained by imaging and instead selecting an image for performing an analysis process based on the information obtained therefrom, the time for selecting an image for performing an analysis process can be saved. Further, even when the number of pieces of first diagnostic support information obtained by the first analysis process is small, since the content of the first diagnostic support information for obtaining a favorable processing result in the second analysis process can be set, the accuracy of the second diagnostic support information, which is the finally obtained diagnostic support information, is high. Therefore, the processor device 16 that functions as an image processing device or the endoscope system 10 including the image processing device can quickly and with high accuracy obtain diagnostic support information by CAD. In particular, for ulcers, etc. where medical treatment centered on the severity is crucial, it is beneficial to accurately determine the severity with high accuracy, and the endoscope system 10 or the like can be preferably used. Colitis For those such as ulcers where accurate diagnosis at an early stage of onset is important because accurate determination of the severity is beneficial, the endoscope system 10 or the like can be preferably used.

[0117] Note that since the second analysis processing result only needs to be notified to the user, in addition to being displayed on the display 18, it may also be notified by voice or the like. However, it is preferable that the display control unit 57 performs control to display the second analysis processing result on the display 18. The display form on the display 18 is preferably a form that does not hinder the visibility of the display image for the user and can be grasped at a glance. Therefore, the second diagnostic support information, which is the second analysis processing result, may be displayed in an area other than the display image 111 of the display 18, or the second diagnostic support information may be displayed by superimposing it on the display image 111 by performing image processing on the display image 111. As shown in FIG. 29, for example, when the second analysis processing result is the second diagnostic support information and is the Mayo score, the lesion area and the score value may be superimposed on the display image 111 and shown by a colored frame display 114 and a short text display 115.

[0118] When the second analysis processing result is displayed on the display 18, the user can quickly and at a glance grasp highly reliable diagnostic support information. Also, even for a lesion that is difficult to distinguish in a diagnosis using a normal observation image, since the analysis processing is performed on the optimal image suitable for the analysis processing, it is possible to prevent overlooking the lesion. After the second analysis processing result is displayed on the display image 111, the user can quickly switch the type of the display image to an endoscopic image of a type that facilitates detailed observation of the lesion shown in the second analysis processing result and observe it in detail.

[0119] In particular, when switching the display image 111 to a candidate image of a type that is not the display image 111, since the candidate image of that type has already been obtained although it is not being displayed, it can be quickly switched to a display image.

[0120] Also, it is preferable that the candidate image acquisition unit 71 acquires, as one type of candidate image, an endoscopic image obtained by photographing an observation target illuminated by the white illumination light emitted by the light source unit 20. Since the endoscopic image with white illumination light is an image that can be recognized as a natural color for humans, by using it as a display image, users such as doctors can smoothly perform endoscopic examinations.

[0121] In addition, when the light source unit 20 emits the first illumination light in the first illumination pattern during the first illumination period, emits the second illumination light in the second illumination pattern during the second illumination period, and includes a light source processor for switching between the first illumination light and the second illumination light, and an imaging sensor that outputs a first endoscopic image obtained by photographing an observation target illuminated by the first illumination light and a second endoscopic image obtained by photographing an observation target illuminated by the second illumination light, it is preferable that the candidate image acquisition unit 71 acquires the first endoscopic image and the second endoscopic image as candidate images. Thereby, by combining the first illumination pattern and the second illumination pattern, it is possible to obtain a plurality of types of candidate images with various illumination lights, which is preferable.

[0122] Note that the first analysis processing unit 72 may perform the first analysis processing on one type of candidate image preset among a plurality of types of candidate images. An embodiment in this case will be described below. As shown in FIG. 30, three types of candidate images, namely, a first candidate image, a second candidate image, and a fifth candidate image, are acquired. The first candidate image is an endoscopic image that is the same as a normal display image obtained by white light. The second candidate image is an endoscopic image in which blood vessels or structures at a specific depth, such as superficial blood vessels, are emphasized by pseudo-color processing. The fifth candidate image is a color difference emphasized image, which is an endoscopic image in which the color difference between an abnormal part and a normal part is emphasized. These candidate images are based on endoscopic images taken in the endoscopic diagnosis of the large intestine of patients with ulcerative colitis.

[0123] As shown in FIG. 31, among these candidate images, the display image generation unit 75 generates a display image 111 based on the first candidate image. Therefore, the display image 111 based on the first candidate image is displayed on the display 18. Further, the first analysis processing unit 72 performs the first analysis processing by a learned model on the first candidate image, which is one type of candidate image preset, by the first image first analysis processing unit 91 to obtain, as the first diagnostic support information, the subject name of the observation target and Tip the distance between 12d and the observation target. Specifically, as the first diagnostic support information, the subject name of the observation target is "mucosa", and Tip the distance between 12d and the observation target is the information of "distant view".

[0124] The first diagnostic support information is sent to the optimal image selection unit 73. Based on the first analysis result obtained by the first analysis process, the optimal image selection unit 73 selects the fifth candidate image as the optimal image from three types of candidate images, namely, the first candidate image, the second candidate image, and the fifth candidate image, which are acquired by the candidate image acquisition unit 71. The optimal image selection unit 73 has in advance information that the fifth candidate image is an endoscopic image with color difference enhancement processing and is effective for diagnosis when the observation condition is a long shot. The fifth candidate image selected by the optimal image selection unit 73 is sent to the second analysis processing unit 74. The second analysis processing unit 74 performs the second analysis process based on the selected fifth candidate image and obtains the second diagnostic support information as the second analysis processing result. Specifically, the second analysis processing unit 74 uses a learned model by machine learning for the fifth candidate image and calculates the Mayo score, which is an index of endoscopic severity for ulcerative colitis, as "3" from the state of the mucosa, and uses this as the second diagnostic support information.

[0125] The display control unit 57 continuously controls the display of the display image 111 based on the first candidate image on the display 18, and performs image processing of the display image 111 so as to superimpose the second diagnostic support information, which is the second analysis processing result, on the display image 111 as soon as the second analysis processing result is obtained by the second analysis processing unit 74. Specifically, since the second diagnostic support information is "Mayo score: 3", it is displayed by the diagnostic support information display 116 of "Mayo: 3" in the lower right part of the display image 111.

[0126] Also, an embodiment in another case will be described below. As shown in FIG. 32, three types of candidate images, namely, the first candidate image, the second candidate image, and the fifth candidate image, are acquired. The first candidate image, the second candidate image, or the fifth candidate image is the same as that described above. These candidate images are based on endoscopic images taken in the endoscopic diagnosis of the large intestine of patients with ulcerative colitis.

[0127] As shown in FIG. 33, among these candidate images, the display image generation unit 75 generates a display image 111 based on the first candidate image. Therefore, the display image 111 based on the first candidate image is displayed on the display 18. Also, the first analysis processing unit 72 performs the first analysis processing by the learned model on the first candidate image, which is one type of candidate image set in advance, by the first image first analysis processing unit 91 to obtain the name of the subject to be observed and the Tip distance between the 12d and the subject to be observed. Specifically, as the first diagnostic support information, the name of the subject to be observed is "mucosa", and Tip the information that the distance between the 12d and the subject to be observed is "close view" is obtained.

[0128] The first diagnostic support information is sent to the optimal image selection unit 73. The optimal image selection unit 73 selects the second candidate image as the optimal image from three types of candidate images, namely, the first candidate image, the second candidate image, and the fifth candidate image, based on the first analysis result obtained by the first analysis processing. The optimal image selection unit 73 has information in advance that the second candidate image is an endoscopic image in which pseudo-color processing is performed and the blood vessels or structures of the superficial blood vessels are emphasized, and is effective for diagnosis when the observation condition is a close view. The second candidate image selected by the optimal image selection unit 73 is sent to the second analysis processing unit 74. The second analysis processing unit 74 performs the second analysis processing based on the selected second candidate image and obtains the second diagnostic support information as the second analysis processing result. Specifically, the second analysis processing unit 74 uses a learned model by machine learning on the second candidate image to calculate the Geboes score, which is an index of the pathological stage for ulcerative colitis, as "3" from the state of the mucosa, and uses this as the second diagnostic support information.

[0129] The display control unit 57 controls the display 18 to continuously display the display image 111 based on the first candidate image. However, as soon as the second analysis processing unit 74 obtains the second analysis processing result, the image processing of the display image 111 is performed so that the second diagnostic support information, which is the second analysis processing result, is superimposed on the display image 111. Specifically, since the second diagnostic support information is "Geboes score: 3", it is displayed in the lower right part of the display image 111 by the diagnostic support information display 116 of "Geboes: 3".

[0130] As described above, according to the endoscope system 10 and the like, in an endoscopic examination, particularly for ulcers centered on medical treatment according to severity Colitis etc., since an accurate diagnosis at an early stage of onset is important, these severities can be determined automatically, quickly, and with high accuracy.

[0131] Next, a series of processes for displaying diagnostic support information performed by the processor device 16, which is an image analysis processing device, or the endoscope system 10 will be described with reference to the flowchart shown in FIG. 34. The candidate image acquisition unit 71 acquires a plurality of types of candidate images (step ST110). The display image generation unit 75 generates a display image based on at least one type of candidate image among the plurality of types of candidate images (step ST120). The display control unit 57 controls the display 18 to display the display image (step ST130).

[0132] In order to perform the first analysis process by the first analysis processing unit 72, candidate images of a preset type are selected in advance (step ST140). The first analysis process is performed on the selected candidate images (step ST150). A first analysis processing result is obtained by the first analysis process (step ST160). The optimal image selection unit 73 selects at least one type of optimal image from a plurality of types of candidate images based on the first analysis processing result (step ST170). The second analysis processing unit 74 performs a second analysis process on the selected optimal image (step ST180). A second analysis processing result is obtained by the second analysis process (step ST190). The display control unit 57 performs control to superimpose the second analysis processing result on the display image and display it on the display 18 (step ST200).

[0133] Note that the step ST120 in which the display image generation unit 75 generates a display image and the step ST130 in which the display control unit 57 performs control to display the display image on the display may be performed in parallel with the step ST140 of selecting candidate images of a preset type in order to perform the first analysis process.

[0134] In the above-described embodiments, modifications, and the like, the processor device 16 functions as an image processing device. However, an image processing device including the image processing unit 56 may be provided separately from the processor device 16. In addition, as shown in FIG. 35, the image processing unit 56 can be provided in a diagnostic support device 911 that acquires RAW images captured by the endoscope 12 directly from the endoscope system 10 or indirectly from a PACS (Picture Archiving and Communication Systems) 910. Further, as shown in FIG. 36, the image processing unit 56 can be provided in a medical service support device 930 that is connected via a network 926 to various inspection devices such as a first inspection device 921, a second inspection device 922,..., and a K-th inspection device 923 including the endoscope system 10.

[0135] Each of the above-described embodiments and modifications can be implemented by arbitrarily combining some or all of them. Further, in each of the above-described embodiments and modifications, the endoscope 12 is a so-called flexible endoscope having a flexible insertion portion 12a. However, the present invention is also suitable when using a capsule endoscope that is swallowed by the observation target or a rigid endoscope (laparoscope) used in surgical operations or the like.

[0136] The above-described embodiments, modifications, etc. include a program for an image processing apparatus for realizing, in a computer, a candidate image acquisition function for acquiring a plurality of types of candidate images based on an endoscope image obtained by photographing an observation target using an endoscope, a display control function for performing control to display a display image based on at least one type of candidate image among the plurality of types of candidate images on a display, a first analysis processing function for performing a first analysis process on one or a plurality of types of candidate images set in advance among the plurality of types of candidate images, an optimal image selection function for selecting at least one type of candidate image from the plurality of types of candidate images as an optimal image based on the first analysis processing result obtained by the first analysis processing, and a second analysis processing function for obtaining diagnosis support information by performing a second analysis process on the optimal image.

[0137] In the above-described embodiment, the hardware structure of a processing unit such as an image processor or a light source processor 22 that executes various processes such as the central control unit 51, the image acquisition unit 52, the image processing unit 56, and the display control unit 57 included in the processor device 16 which is an image processing apparatus is various processors as shown below. The various processors include a CPU (Central Processing Unit) which is a general-purpose processor that executes software (program) and functions as various processing units, a programmable logic device (PLD) which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit which is a processor having a circuit configuration designed specifically for executing various processes.

[0138] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). Also, a plurality of processing units may be composed of one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Second, as represented by a System On Chip (SoC), there is a form in which a processor that realizes the functions of an entire system including a plurality of processing units with one integrated circuit (IC) chip is used. Thus, various processing units are configured by using one or more of the above various processors as a hardware structure.

[0139] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements.

[0140] Note that the present invention can be used not only in an endoscope system for acquiring an endoscope image or the like, a processor device, and other related devices, but also in a system or device for acquiring a medical image (including a moving image) other than an endoscope image. For example, the present invention can be applied to an ultrasonic inspection device, an X-ray imaging device (including a computed tomography (CT) inspection device, a mammography device, etc.), an MRI (magnetic resonance imaging) device, and the like.

Explanation of Reference Numerals

[0141] 10 Endoscope system 12 Endoscope 12a Insertion portion 12b Operation portion 12c Bending portion 12d Tip portion 12e Angle knob 12f Scope Button No. 1 12g Scope Button No. 2 12h Zoom Operation Section 14 Light Source Device 16 Processor Device 18 Display 19 Keyboard 20 Light Source Section 20a V-LED 20b B-LED 20c G-LED 20d R-LED 22 Light Source Processor 30a Illumination Optical System 30b Imaging Optical System 41 Light Guide 42 Illumination Lens 43 Objective Lens 44 Zoom Lens 45 Image Sensor 46 Imaging Processor 51 Central Control Section 52 Image Acquisition Section 53 DSP 54 Noise Reduction Section 55 Conversion Section 56 Image Processing Section 57 Display Control Section 61 Normal Observation Image Processing Section 62 Special Observation Image Processing Section 63 Diagnostic Support Image Processing Section 71 Candidate Image Acquisition Section 72 First Analysis Processing Section 73 Optimal Image Selection Section 74 Second Analysis Processing Section 75 Display Image Generation Section 81 First Candidate Image Generation Section 82 Second Candidate Image Generation Section 83 Third Candidate Image Generation Section 84 Fourth Candidate Image Generation Section 84a Oxygen Saturation Signal Ratio Calculation Section 84b Oxygen Saturation Calculation Table 84c Oxygen Saturation Calculation Section 84d Oxygen Saturation Image Generation Section 85 Fifth candidate image generation unit 85a Signal ratio calculation unit for color difference expansion 85b Color difference expansion processing unit 85c Color difference expansion image generation unit 86 nth candidate image generation unit 91 First image first analysis processing unit 92 Second image first analysis processing unit 93 Third image first analysis processing unit 94 Fourth image first analysis processing unit 95 Fifth image first analysis processing unit 96 nth image first analysis processing unit 101 First image second analysis processing unit 102 Second image second analysis processing unit 103 Third image second analysis processing unit 104 Fourth image second analysis processing unit 105 Fifth image second analysis processing unit 106 nth image second analysis processing unit 111 Display image 112 Mode name display 113 Endoscope image type name display 114 Frame display 115 Text display 116 Diagnostic support information display 910 PACS 911 Diagnostic support device 921 First inspection device 922 Second inspection device 923 Kth inspection device 926 Network 930 Medical business support device P1 First lighting period P2 Second lighting period FL Frame L1 First lighting light L2a, L2b Second lighting light Q1, Q2, Q3, Q4, Q5 Emission period SP1, SP2, SP3, SP4 Spectral spectrum for second lighting light A1, A2 Range Isolines of oxygen saturation of ELx, EL1, EL2, EL3, EL4, Ely Steps ST110 to ST200

Claims

1. An image processing apparatus including an image processor for images, wherein the image processor for images acquires a plurality of types of candidate images based on an endoscopic image obtained by photographing an observation target using an endoscope, performs control to display a display image based on at least one type of the plurality of types of candidate images on a display, performs a first analysis process on one or more types of the plurality of types of candidate images set in advance, selects at least one type of the plurality of types of candidate images as an optimal image based on a first analysis result obtained by the first analysis process, obtains a second analysis result by performing a second analysis process on the optimal image, wherein the second analysis process is an image processing for performing a process determined according to the type of the candidate image selected as the optimal image from among a plurality of different processes. An image processing apparatus.

2. The image processing apparatus according to claim 1, wherein the image processor for images performs control to display the second analysis result on the display.

3. The image processing apparatus according to claim 1, wherein the image processor for images performs control to display the second analysis result by superimposing it on the display image.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the first analysis process and the second analysis process are analysis processes having different contents.

5. generates the candidate image by performing an enhancement process on the endoscopic image, The image processing apparatus according to any one of claims 1 to 4, wherein the image processor for images acquires a plurality of types of candidate images by distinguishing the types of the candidate images according to the presence or type of the enhancement process.

6. The image processing apparatus according to claim 5, wherein the enhancement process is a color enhancement process and / or a structure enhancement process.

7. An endoscope system including the image processing apparatus according to any one of claims 1 to 6, and a light source unit that emits illumination light for irradiating the observation target.

8. The endoscope system according to claim 7, wherein the image processor for images acquires the endoscopic image obtained by photographing the observation target illuminated by each of a plurality of types of illumination light having different spectral spectra emitted by the light source unit as different types of candidate images.

9. The endoscope system according to claim 7 or 8, wherein the light source unit emits each of a plurality of types of illumination light having different spectral spectra in a light emission cycle having a preset order repeatedly.

10. The endoscope system according to claim 9, wherein the image processor selects at least one of the optimal images from the plurality of types of candidate images obtained in one light emission cycle.

11. A light source processor that emits first illumination light in a first light emission pattern during a first illumination period, emits second illumination light in a second light emission pattern during a second illumination period, and switches between the first illumination light and the second illumination light; An imaging sensor that outputs a first endoscope image obtained by photographing an observation target illuminated by the first illumination light and a second endoscope image obtained by photographing the observation target illuminated by the second illumination light; The endoscope system according to any one of claims 7 to 10, wherein the image processor acquires the first endoscope image and the second endoscope image as the candidate images.

12. The endoscope system according to any one of claims 7 to 11, wherein the image processor acquires the endoscope image obtained by photographing the observation target illuminated by white illumination light emitted by the light source unit as one of the candidate images.

13. The endoscope system according to any one of claims 7 to 12, wherein the image processor acquires the endoscope image obtained by photographing the observation target illuminated by illumination light including narrow-band light in a preset wavelength band emitted by the light source unit as one of the candidate images.

14. When the optimal image is the candidate image subjected to the color difference enhancement process, the second analysis process performs a Mayo score calculation process. The image processing apparatus according to any one of claims 1 to 6.

15. When the optimal image is the candidate image in which the surface layer blood vessels or structure are emphasized, the second analysis process performs a Geboes score calculation process. The image processing apparatus according to any one of claims 1 to 6.

16. A candidate image acquisition step of acquiring a plurality of types of candidate images based on an endoscope image obtained by photographing an observation target using an endoscope; A display image control step of performing control to display a display image based on at least one of the plurality of types of candidate images on a display; A first analysis processing step of performing a first analysis process on one or more types of the candidate images set in advance among the plurality of types of the candidate images; An optimal image selection step of selecting at least one type of candidate image as an optimal image from the plurality of types of candidate images based on the first analysis processing result obtained by the first analysis processing; A second analysis processing step of obtaining a second analysis processing result by performing a second analysis process on the optimal image, and The second analysis process is an operation method of an image processing apparatus which is a process determined according to the type of the candidate image selected as the optimal image from a plurality of different processes.

17. For a computer, A candidate image acquisition function of acquiring a plurality of types of candidate images based on an endoscopic image obtained by photographing an observation target using an endoscope; A display control function of performing control to display a display image based on at least one type of the candidate images among the plurality of types of candidate images on a display; A first analysis processing function of performing a first analysis process on one or more types of the candidate images set in advance among the plurality of types of candidate images; An optimal image selection function of selecting at least one type of candidate image as an optimal image from the plurality of types of candidate images based on the first analysis processing result obtained by the first analysis processing; For realizing a second analysis processing function of obtaining diagnosis support information by performing a second analysis process on the optimal image, and The second analysis process is a program for an image processing apparatus which is a process determined according to the type of the candidate image selected as the optimal image from a plurality of different processes.

Images