Image processing device, endoscope system, image processing method, and program

Abstract

The present invention provides an image processing device, an endoscope system, an image processing method, and a program that enable oxygen saturation imaging with correction for in-plane unevenness caused by imaging system factors in endoscopes. [Solution] The image processing device according to this disclosure acquires a first spectral image obtained by imaging a living organism irradiated with a first light using an endoscope, acquires a second spectral image obtained by imaging a living organism irradiated with a second light having a different wavelength band from the first light using an endoscope, corrects the brightness distribution of the first spectral image with first correction data stored in advance in association with the endoscope, outputs an image showing the distribution of oxygen saturation of blood in the living organism from the corrected first spectral image and the second spectral image, and calculates the first correction data using a first reference spectral image obtained by imaging a reference region with a known brightness distribution using an endoscope and the brightness distribution of the reference region.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an endoscope system, an image processing method, and a program, and particularly relates to an endoscope having an oxygen saturation imaging function.

Background Art

[0002] Oxygen saturation imaging using a special light endoscope is a technique for imaging information indicating oxygen saturation using a plurality of spectral signal ratios obtained for each pixel. A slight variation in the signal value for each input pixel becomes noise in the output image. In-plane unevenness caused by the optical system of the endoscope system is one of the problems that need to be corrected.

[0003] The main causes of in-plane unevenness can be broadly divided into two categories: those due to the combination of the light source and the illumination system of the endoscope, and those due to the imaging system factors of the endoscope. Generally, shading correction is known as a method for correcting unevenness in the optical system. For example, Patent Document 1 describes an endoscope device that corrects color shading.

[0004] In shading correction, a correction process is performed in which in-plane unevenness of the illumination system factors is extracted in advance using a reference reflector or the like, and a filter for canceling out the in-plane unevenness is created. However, since the appearance of illumination unevenness changes depending on the distance and angle from the subject, the shape of the subject, and the individual differences of the light sources, the applicable range is limited.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] On the other hand, in-plane unevenness in the imaging system of an endoscope occurs due to the unique unevenness characteristics of each individual endoscope. If a method for correcting these unique unevenness characteristics is established, it will be possible to correct the in-plane unevenness component of the imaging system regardless of individual differences in the subject and light source.

[0007] The present invention has been made in view of these circumstances, and aims to provide an image processing device, an endoscope system, an image processing method, and a program that realize oxygen saturation imaging corrected for in-plane unevenness caused by imaging system factors of an endoscope. [Means for solving the problem]

[0008] To achieve the above objective, the image processing apparatus according to the first aspect of this disclosure comprises a processor, which acquires a first spectral image obtained by imaging a living organism irradiated with a first light using an endoscope, acquires a second spectral image obtained by imaging a living organism irradiated with a second light having a different wavelength band from the first light using an endoscope, corrects the brightness distribution of the first spectral image with first correction data stored in advance in association with the endoscope, outputs an image showing the distribution of oxygen saturation of blood in a living organism from the corrected first spectral image and the second spectral image, and the first correction data is calculated using a first reference spectral image obtained by imaging a reference region with a known brightness distribution using an endoscope and the brightness distribution of the reference region.

[0009] In the image processing apparatus according to the second aspect of the present disclosure, the processor, in the image processing apparatus according to the first aspect, corrects the brightness distribution of the second spectral image with second correction data stored in advance in association with the endoscope, outputs an image showing the distribution of oxygen saturation of blood in a living body from the corrected first spectral image and the corrected second spectral image, and preferably the second correction data is calculated using a second reference spectral image obtained by imaging a reference region with a known brightness distribution using an endoscope and the brightness distribution of the reference region.

[0010] The image processing apparatus according to the third aspect of this disclosure is an image processing apparatus according to the second aspect, wherein the processor acquires a third spectral image obtained by imaging a living organism with an endoscope that is irradiated with a third light having a different wavelength band from the first and second light, corrects the brightness distribution of the third spectral image with third correction data stored in advance in association with the endoscope, outputs an image showing the distribution of oxygen saturation of blood in the living organism from the corrected first spectral image, the corrected second spectral image, and the corrected third spectral image, and preferably the third correction data is calculated using a third reference spectral image obtained by imaging a reference region with a known brightness distribution with an endoscope and the brightness distribution of the reference region.

[0011] In the image processing apparatus according to the fourth aspect of this disclosure, in the image processing apparatus according to any one aspect of the first to third aspects, it is preferable that the first light has a peak wavelength in the range of 460 to 480 nm, and the second light has a peak wavelength in the range of 540 to 580 nm.

[0012] In the fifth aspect of the present disclosure, the image processing apparatus is an image processing apparatus according to any one of the first to fourth aspects, wherein the reference region is the inner surface of an integrating sphere, and the brightness distribution of the reference region is preferably uniform.

[0013] In the sixth aspect of this disclosure, the image processing apparatus is preferably, in the fifth aspect of the image processing apparatus, the first reference spectral image is an image obtained by imaging the inner surface of an integrating sphere irradiated with reference light containing light in the wavelength band of the first light from an endoscope.

[0014] In the image processing apparatus according to the seventh aspect of this disclosure, in the image processing apparatus according to any one aspect of the first to sixth aspects, it is preferable that the first correction data is stored in association with the individual identifier of the endoscope.

[0015] In the image processing apparatus according to the eighth aspect of this disclosure, in the image processing apparatus according to the seventh aspect, it is preferable that the first correction data is stored in the memory of the endoscope.

[0016] The image processing apparatus according to the ninth aspect of the present disclosure is an image processing apparatus according to any one aspect of the first to eighth aspects, wherein the endoscope has an image sensor in which a first pixel group and a second pixel group having different wavelength bands for photoelectric conversion from the first pixel group are arranged in two dimensions, and it is preferable that the first spectral image is captured using the first pixel group and the second spectral image is captured using the second pixel group.

[0017] The image processing apparatus according to the tenth aspect of the present disclosure is an image processing apparatus according to any one aspect of the first to eighth aspects, wherein the endoscope has a first image sensor and a second image sensor which has a different wavelength band for photoelectric conversion from the first image sensor, and it is preferable that the first spectral image is captured using the first image sensor and the second spectral image is captured using the second image sensor.

[0018] To achieve the above objective, the endoscopic system according to the 11th aspect of this disclosure is an endoscopic system comprising: an endoscope having an image sensor; a light source that generates a plurality of irradiation lights with different wavelength bands; a processor device that controls the endoscope and the light source to image a living body irradiated with the irradiation light; and an image processing device according to any one of the first to tenth aspects.

[0019] To achieve the above objective, an image processing method according to a twelfth aspect of this disclosure is an image processing method performed by a processor, wherein the processor acquires a first spectral image obtained by imaging a living organism irradiated with a first light using an endoscope, acquires a second spectral image obtained by imaging a living organism irradiated with a second light having a different wavelength band from the first light using an endoscope, corrects the brightness distribution of the first spectral image with first correction data stored in advance in association with the endoscope, outputs an image showing the distribution of oxygen saturation of blood in a living organism from the corrected first spectral image and the second spectral image, and the first correction data is calculated using a first reference spectral image obtained by imaging a reference region with a known brightness distribution using an endoscope and the brightness distribution of the reference region.

[0020] In order to achieve the above object, the program according to the 13th aspect of the present disclosure is a program that causes a computer to execute the image processing method according to the 12th aspect. A non-temporary and computer-readable storage medium in which the program according to the 13th aspect is stored is also included in the present disclosure. The program according to the 13th aspect may be a program product.

[0021] In the image processing method according to the 12th aspect and the program according to the 13th aspect, it is possible to adopt a configuration including the same specific aspects as the above-described image processing apparatus.

Advantages of the Invention

[0022] According to the present invention, it is possible to realize oxygen saturation imaging in which in-plane unevenness of an imaging system factor of an endoscope is corrected.

Brief Description of the Drawings

[0023] [Figure 1] It is a schematic diagram of an endoscope system for realizing oxygen saturation imaging. [Figure 2] It is a diagram showing displays on a display and an extended display in a normal mode. [Figure 3] It is a diagram showing displays on a display and an extended display in an oxygen saturation mode. [Figure 4] It is a block diagram showing functions of an endoscope system. [Figure 5] It is a graph showing the emission spectrum of white light. [Figure 6] It is a graph showing the emission spectra of the first illumination light, the second illumination light, and the third illumination light. [Figure 7] It is a graph showing the spectral sensitivity of an imaging sensor. [Figure 8] It is a table showing illumination and an image signal to be acquired in a normal mode. [Figure 9] It is a table showing illumination and an image signal to be acquired in an oxygen saturation mode. [Figure 10]This is an explanatory diagram illustrating the light emission control and display control in oxygen saturation mode. [Figure 11] This graph shows the hemoglobin reflectance spectrum, which varies depending on blood concentration. [Figure 12] This figure shows a table for calculating oxygen saturation. [Figure 13] The Functions of Expansion Processor Devices. [Figure 14] This is a block diagram showing the functions of the image processing unit. [Figure 15] This is an explanatory diagram showing how to calculate oxygen saturation. [Figure 16] This is a schematic diagram showing the acquisition of a reference spectral image. [Figure 17] This flowchart shows how to generate a correction map. [Figure 18] This flowchart shows a method for generating oxygen saturation images using a correction map. [Figure 19] This figure shows another embodiment of the imaging optical system. [Modes for carrying out the invention]

[0024] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, identical components are denoted by the same reference numerals, and redundant descriptions are omitted where appropriate.

[0025] [Endoscopic system that enables oxygen saturation imaging] Endoscopic oxygen saturation imaging is a technique for calculating hemoglobin oxygen saturation from a small amount of spectral information of visible light. Hemoglobin oxygen saturation is synonymous with the ratio of oxyhemoglobin to deoxyhemoglobin. To calculate oxygen saturation, for example, the living body is imaged using multiple illumination lights with different wavelength bands. At a minimum, the living body is imaged using light with wavelength bands in which oxyhemoglobin and deoxyhemoglobin have different extinction coefficients. Then, a predetermined calculated value is calculated using the signal value of each pixel of the obtained image, and the oxygen saturation of the blood in the living body is determined using an oxygen saturation calculation table that shows the correlation between the calculated value and oxygen saturation. Oxygen saturation is not limited to an absolute value, but may be a relative value, such as evaluating the magnitude of oxygen saturation in stages.

[0026] As shown in Figure 1, the endoscope system 20 comprises an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an expansion processor device 17, and an expansion display 18. The endoscope 12 has an individual identifier, which is an identifier for each individual unit. The endoscope 12 is optically or electrically connected to the light source device 13 and electrically connected to the processor device 14. The expansion processor device 17 is electrically connected to the light source device 13 and the processor device 14. Each of these connections is not limited to wired connections, but may also be wireless. They may also be connected via a network.

[0027] Endoscope 12 is a rigid endoscope, or laparoscope, that is inserted into the body cavity of a patient to perform surgical treatment and image organs within the body cavity from the serosal side. Alternatively, endoscope 12 may be a flexible endoscope inserted through the patient's nose, mouth, or anus. The patient is the object into which endoscope 12 is inserted. The patient has an object of observation, and the object of observation is a living organism.

[0028] The endoscope 12 comprises an insertion section 12a for insertion into the patient's abdominal cavity and an operating section 12b provided at the proximal end of the insertion section 12a. The tip portion of the insertion section 12a (hereinafter referred to as the tip) houses an illumination optical system and an imaging optical system. These generate an image signal. The generated image signal is output to the processor device 14. The operating section 12b is provided with a mode switching switch 12c and a zoom operation switch 12d for zoom operation, etc. The mode switching switch 12c is used for switching observation modes, etc.

[0029] The endoscope system 20 has two observation modes: a normal mode and an oxygen saturation mode. As shown in Figure 2, in normal mode, a natural-colored white light image NP1, which is an endoscopic image obtained by imaging the observation target using white light as illumination, is displayed on the display 15, while nothing is displayed on the extended display 18.

[0030] As shown in Figure 3, in oxygen saturation mode, the oxygen saturation of the blood of the observed subject is calculated based on the endoscopic image obtained by imaging the subject, and the calculated oxygen saturation is displayed on the extended display 18 as an oxygen saturation image OP. In addition, in oxygen saturation mode, due to the difference in illumination light between normal mode and oxygen saturation mode, a white light equivalent image NP2 with fewer short-wavelength components than the white light image is displayed on the display 15.

[0031] The processor unit 14 controls the entire endoscope system 20. It also generates endoscopic images by performing image processing on image signals transmitted from the endoscope 12. The display 15 displays the endoscopic images generated by the processor unit 14.

[0032] The user interface 16 is an interface for performing input operations such as device settings on the processor device 14, and can be a keyboard, mouse, microphone, tablet, foot switch, etc. If the display 15 or extended display 18 has a touch panel, the display 15 or extended display 18 may also serve as the user interface 16.

[0033] As shown in Figure 4, the light source device 13 comprises a light source unit 21 and a light source processor 22 that controls the light source unit 21. The light source unit 21 has, for example, multiple semiconductor light sources that generate multiple illumination lights (an example of "irradiation light") with different wavelength bands. The light source unit 21 emits illumination light to illuminate the object of observation by turning on or off each of the multiple semiconductor light sources, and when they are turned on, by controlling the amount of light emitted from each semiconductor light source.

[0034] In this embodiment, the light source unit 21 has five colored LEDs: V-LED (Violet Light Emitting Diode) 21a, BS-LED (Blue Short-wavelength Light Emitting Diode) 21b, BL-LED (Blue Long-wavelength Light Emitting Diode) 21c, G-LED (Green Light Emitting Diode) 21d, and R-LED (Red Light Emitting Diode) 21e.

[0035] Instead of these LEDs, the light source unit 21 can use a combination of an LD (Laser Diode), a phosphor, and a band-limiting filter, or a combination of a lamp such as a xenon lamp and a band-limiting filter.

[0036] V-LED21a is a violet light source that emits violet light V at 410 nm ± 10 nm. BS-LED21b is a blue light source that emits a second blue light BS at 450 nm ± 10 nm. BL-LED21c is a blue light source that emits a first blue light BL at 470 nm ± 10 nm. Specifically, the first blue light BL has a peak wavelength within the range of 460 nm to 480 nm.

[0037] The first blue light BL emitted by BL-LED21c and the second blue light BS emitted by BS-LED21b have different central wavelengths and wavelength bands. The central wavelength and wavelength band of the first blue light BL are the central wavelength and wavelength band within the blue wavelength band where the difference in the absorption coefficients of oxyhemoglobin and deoxyhemoglobin is approximately maximum.

[0038] The G-LED21d is a green light source that emits broadband green light G with a central wavelength of 540 nm. The green light G has a peak wavelength in the range of 540 nm to 580 nm. The R-LED21e is a red light source that emits broadband red light R with a central wavelength of 620 nm.

[0039] The center wavelength and peak wavelength of each LED 21a to 21e may be the same or different.

[0040] The light source processor 22 independently controls the on / off state and the amount of light emitted when each LED 21a to 21e is lit by inputting control signals to each LED 21a to 21e independently. The control of on / off states etc. by the light source processor 22 differs depending on the mode, and the details will be described later.

[0041] The light emitted from each LED 21a to 21e is incident on the light guide 24 via an optical path coupling section 23, which is composed of mirrors, lenses, etc. The light guide 24 is built into the endoscope 12 and the universal cord (a cord connecting the endoscope 12, the light source device 13, and the processor device 14). The light guide 24 propagates the light from the optical path coupling section 23 to the tip of the endoscope 12.

[0042] The endoscope 12 is equipped with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and illumination light propagated by the light guide 24 is irradiated onto the object to be observed through the illumination lens 32. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Light from the object to be observed, which is irradiated with illumination light, enters the imaging sensor 36 through the objective lens 35. As a result, an image of the object to be observed is formed on the imaging sensor 36.

[0043] The imaging sensor 36 is a single-chip color image sensor that captures an object being observed while illuminated by illumination light. The imaging sensor 36 has a two-dimensional arrangement of a first pixel group, a second pixel group with a different wavelength band for photoelectric conversion from the first pixel group, and a third pixel group with a different wavelength band for photoelectric conversion from the first and second pixel groups. Here, each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. Therefore, when an object being observed is captured by the imaging sensor 36, three types of images are obtained: a B image, a G image, and an R image. Preferably, the imaging sensor 36 is a Bayer array color imaging sensor in which the ratio of the number of B pixels, G pixels, and R pixels is 1:2:1.

[0044] As the image sensor 36, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. Alternatively, instead of the primary color image sensor 36, a complementary color image sensor equipped with complementary color filters for C (cyan), M (magenta), Y (yellow), and G (green) may be used. When using a complementary color image sensor, a four-color image signal of CMYG is output. By converting the four-color image signal of CMYG to a three-color image signal of RGB through complementary-to-primary color conversion, image signals for each RGB color similar to those of the image sensor 36 can be obtained.

[0045] The imaging processor 37 drives and controls the imaging sensor 36. The CDS / AGC circuit 40 (Correlated Double Sampling / Automatic Gain Control) performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 36. The image signal that has passed through the CDS / AGC circuit 40 is converted into a digital image signal by the A / D converter 41 (Analog / Digital). The digital image signal after A / D conversion is input to the processor device 14.

[0046] The processor unit 14 comprises a DSP (Digital Signal Processor) 45, an image processing unit 50, an image communication unit 51, a display control unit 52, and a central control unit 53. Programs related to various processes are stored in a program memory (not shown) within the processor unit 14. The central control unit 53, which is configured by the processor, executes the programs in the program memory, thereby realizing the functions of the DSP 45, the image processing unit 50, the image communication unit 51, the display control unit 52, and the central control unit 53.

[0047] The DSP45 performs various signal processing on the image signal received from the endoscope 12, including defect correction, offset processing, gain correction processing, demosaicing, linear matrix processing, white balance processing, gamma conversion, YC conversion, and noise reduction processing. In defect correction processing, the signals of defective pixels in the imaging sensor 36 are corrected. In offset processing, the dark current component is removed from the image signal after defect correction processing, and an accurate zero level is set. Gain correction processing adjusts the signal level of each image signal by multiplying the image signal of each color after offset processing by a specific gain. After gain correction processing, demosaicing and linear matrix processing to improve color reproduction are applied to each color image signal.

[0048] After linear matrix processing, white balance processing is performed, and then gamma conversion processing adjusts the brightness and saturation of each image signal. Subsequently, YC conversion processing is performed, and the luminance signal Y and chrominance signals Cb and Cr are output to the image processing unit 50. The DSP 45 performs noise reduction processing, such as the moving average method or the median filter method.

[0049] The image processing unit 50 performs various image processing operations on the image signal from the DSP 45. Image processing includes color conversion processing such as 3x3 matrix processing, grayscale conversion processing, and 3D oxygen saturation calculation table processing, as well as color enhancement processing and structural enhancement processing such as spatial frequency enhancement. The image processing unit 50 performs image processing according to the mode. In normal mode, the image processing unit 50 generates a white light image NP1 by performing image processing for normal mode. In oxygen saturation mode, the image processing unit 50 generates a white light equivalent image NP2 and transmits the image signal from the DSP 45 to the expansion processor device 17 via the image communication unit 51. The expansion processor device 17 generates an oxygen saturation image OP based on the transmitted endoscopic image signal.

[0050] The display control unit 52 performs display control to display image signals such as the white light image NP1 and oxygen saturation image OP from the image processing unit 50, as well as other information, on the display 15.

[0051] The extended processor unit 17 receives image signals from the processor unit 14 and performs various image processing operations. The extended processor unit 17 functions as an image processing device. The image processing in the extended processor unit 17 is, for example, a process for generating an oxygen saturation image OP performed in oxygen saturation mode. The image processing in the extended processor unit 17 also includes a process for generating a correction map, which will be described later.

[0052] In oxygen saturation mode, the extended processor unit 17 calculates the oxygen saturation of the blood being observed and generates an oxygen saturation image OP, which is an image of the calculated oxygen saturation. The generated oxygen saturation image OP is displayed on the extended display 18. Details of the process by which the extended processor unit 17 generates the oxygen saturation image OP and the process of generating the correction map will be described later.

[0053] [Control of the light source and imaging unit] In normal mode, the light source processor 22 simultaneously lights up V-LED21a, BS-LED21b, G-LED21d, and R-LED21e, and turns off BL-LED21c. As a result, the light source unit 21 emits white light 55, which includes violet light V with a central wavelength of 410 nm, a second blue light BS with a central wavelength of 450 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm, as shown in Figure 5. The graph shown in the figure schematically illustrates the light intensity of each wavelength band.

[0054] In oxygen saturation mode, the light emission is repeated for three frames, each with a different emission pattern. In the first frame, the light source processor 22 simultaneously lights up BL-LED21c, G-LED21d, and R-LED21e, and turns off V-LED21a and BS-LED21b. As a result, the light source unit 21 emits a broadband first illumination light 56, which includes a first blue light BL with a central wavelength of 470 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm, as shown in F6A of Figure 6.

[0055] In the second frame, the light source processor 22 simultaneously lights up BS-LED21b, G-LED21d, and R-LED21e, and turns off V-LED21a and BL-LED21c. As a result, the light source unit 21 emits a second illumination light 57, which includes a second blue light BS with a central wavelength of 450 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm, as shown in F6B of Figure 6.

[0056] In the third frame, the light source processor 22 turns on the G-LED 21d and turns off the V-LED 21a, BS-LED 21b, BL-LED 21c, and R-LED 21e. As a result, the light source unit 21 emits broadband green light G in the green band as the third illumination light 58, as shown in F6C of Figure 6. Note that in the oxygen saturation mode, the frames required to obtain the image signal necessary for calculating oxygen saturation are the first and second frames, so light emission may occur only in the first and second frames.

[0057] As shown in Figure 7, the B color filter BF provided in the B pixel of the image sensor 36 primarily transmits light in the blue band, specifically light with a wavelength range of 380 to 560 nm (blue transmission band). The peak wavelength at which transmittance is maximum is around 460 to 480 nm. The G color filter GF provided in the G pixel of the image sensor 36 primarily transmits light in the green band, specifically light with a wavelength range of 450 to 630 nm (green transmission band). The R color filter RF provided in the R pixel of the image sensor 36 primarily transmits light in the red band, specifically light with a wavelength range of 580 to 760 nm (red transmission band).

[0058] As shown in Figure 8, in normal mode, the imaging processor 37 controls the imaging sensor 36 to capture an image of the object being observed, illuminated by white light 55 consisting of violet light V, second blue light BS, green light G, and red light R, in one frame at a time. As a result, a Bc image signal is output from the B pixel, a Gc image signal from the G pixel, and an Rc image signal from the R pixel of the imaging sensor 36.

[0059] As shown in Figure 9, in oxygen saturation mode, when the first illumination light 56, which includes a first blue light BL (an example of "first light"), green light G, and red light R, illuminates the object of observation in the first frame, the imaging processor 37 outputs a B1 image signal from the B pixels of the imaging sensor 36, a G1 image signal from the G pixels, and an R1 image signal from the R pixels as the first illumination light image.

[0060] In the second frame, if the second illumination light 57, which includes a second blue light BS, a green light G (an example of the "second light"), and a red light R (an example of the "third light"), illuminates the object of observation, the imaging processor 37 outputs a B2 image signal from the B pixels, a G2 image signal from the G pixels, and an R2 image signal from the R pixels of the imaging sensor 36 as a second illumination light image.

[0061] In the third frame, when the third illumination light 58, which is green light G, illuminates the object being observed, the imaging processor 37 outputs a B3 image signal from the B pixels, a G3 image signal from the G pixels, and an R3 image signal from the R pixels of the imaging sensor 36 as a third illumination light image.

[0062] In oxygen saturation mode, as shown in Figure 10, the first illumination light 56 ​​is emitted in the first frame (1stF), the second illumination light 57 is emitted in the second frame (2ndF), and the third illumination light 58 is emitted in the third frame (3rdF). After that, the second illumination light 57 of the second frame is emitted, and the first illumination light 56 ​​of the first frame is emitted. The white light equivalent image NP2 obtained based on the emission of the second illumination light 57 of the second frame is displayed on the display 15. In addition, the oxygen saturation image OP obtained based on the emission of the first to third illumination lights of the first to third frames is displayed on the extended display 18.

[0063] In oxygen saturation mode, the B1 image signal included in the first illumination light image, and the G2 and R2 image signals included in the second illumination light image are used from the three image signals described above. The image consisting of the B1 image signal is the first spectral image, the image consisting of the G2 image signal is the second spectral image, and the image consisting of the R2 image signal is the third spectral image.

[0064] The B1 image signal contains image information relating to at least the first blue light BL from the light transmitted through the B color filter BF in the first illumination light 56. The B1 image signal (image signal for oxygen saturation) includes image information relating to the first blue light BL, specifically image information for the wavelength band B1 in which the reflection spectrum changes with changes in the oxygen saturation of blood hemoglobin. Preferably, the wavelength band B1 is a wavelength band of 460 nm to 480 nm, including 470 nm, where the difference between the reflection spectrum of oxyhemoglobin shown by curves 55b and 56b and the reflection spectrum of deoxyhemoglobin shown by curves 55a and 56a is maximized, as shown in Figure 11.

[0065] In Figure 11, curve 55a represents the reflection spectrum of reduced hemoglobin at high blood concentration, and curve 55b represents the reflection spectrum of oxyhemoglobin at high blood concentration. On the other hand, curve 56a represents the reflection spectrum of reduced hemoglobin at low blood concentration, and curve 56b represents the reflection spectrum of oxyhemoglobin at low blood concentration.

[0066] The G2 image signal contains image information in a wavelength band G2 related to at least green light G from the light transmitted through the G color filter GF in the second illumination light 57. The wavelength band G2 is preferably 500 nm to 580 nm, as shown in Figure 11. More preferably 540 nm to 580 nm.

[0067] The R2 image signal contains image information for at least the red light R wavelength band R2 of the light transmitted through the R color filter RF in the second illumination light 57. The wavelength band R2 is preferably a wavelength band of 610 nm to 630 nm, for example, as shown in Figure 11.

[0068] Since the B1, G2, and R2 image signals all exhibit brightness dependence, the G2 image signal is used as the normalization signal to create an oxygen saturation calculation table 76a (see Figure 12) for calculating oxygen saturation. Here, the signal ratio ln(B1 / G2), obtained by normalizing the B1 image signal with the G2 image signal, and the signal ratio ln(R2 / G2), obtained by normalizing the R2 image signal with the G2 image signal, are used. Note that "ln" in the signal ratio ln(B1 / G2) represents the natural logarithm (similarly for the signal ratio ln(R2 / G2)).

[0069] When the relationship between the signal ratio ln(B1 / G2) and the signal ratio ln(R2 / G2) and oxygen saturation is represented in a two-dimensional coordinate system with the signal ratio ln(R2 / G2) on the X-axis and the signal ratio ln(B1 / G2) on the Y-axis, as shown in Figure 12, oxygen saturation is represented by contour lines EL along the Y-axis. Contour line ELH represents an oxygen saturation of "100%", and contour line ELL represents an oxygen saturation of "0%". Contour lines are distributed such that the oxygen saturation gradually decreases from contour line ELH to contour line ELL. In Figure 12, contour lines for "80%", "60%", "40%", and "20%" are distributed.

[0070] Furthermore, oxygen saturation may be expressed in a two-dimensional coordinate system with the signal ratio ln(R2 / G2) as the X-axis and {ln(B1 / G2)+ln(R2 / R1)} as the Y-axis, using the signal ratio ln(R2 / R1) obtained by normalizing the R2 image signal with the R1 image signal, taking into account the inter-frame light intensity ratio correction. The R1 image signal contains image information for at least the wavelength band R1 related to red light R from the light transmitted through the R color filter RF in the first illumination light 56. The wavelength band R1 is the same as that of the wavelength band R2.

[0071] [Expansion Processor Device] As shown in Figure 13, the extended processor unit 17 comprises an image acquisition unit 60a and an image processing unit 60b. The image acquisition unit 60a receives image signals transmitted from the processor unit 14 via the image communication unit 51. The image processing unit 60b performs image processing to generate an oxygen saturation image and processing to generate a correction map.

[0072] As shown in Figure 14, the image processing unit 60b comprises a correction map generation unit 61, an oxygen saturation image generation unit 62, and a display control unit 63. The extended processor device 17 has programs for various processes incorporated into a program memory (not shown). The functions of the correction map generation unit 61, the oxygen saturation image generation unit 62, and the display control unit 63 are realized by the execution of the programs in the program memory by a central control unit (not shown) configured by the processor.

[0073] The correction map generation unit 61 generates a correction map to correct in-plane unevenness caused by imaging system factors of the endoscope 12. The correction map generation unit 61 includes a map memory 71. The correction map generation unit 61 may associate the generated correction map with the endoscope 12 and store it in the map memory 71.

[0074] The oxygen saturation image generation unit 62 comprises a base image generation unit 72, a calculation value calculation unit 73, an in-plane uniformity correction unit 74, an oxygen saturation calculation unit 75, an oxygen saturation calculation table 76, and a color tone adjustment unit 77. The base image generation unit 72 generates a base image based on the image signal from the processor device 14. The base image is used as the base for the oxygen saturation image OP. Preferably, the base image is an image that can grasp morphological information such as the shape of the object being observed. The base image consists of a B2 image signal, a G2 image signal, and an R2 image signal. The base image may also be a narrowband light image in which blood vessels or structures (glandular structures, etc.) are highlighted using narrowband light, etc.

[0075] The calculation value calculation unit 73 calculates a calculated value by performing calculations based on the B1 image signal, G2 image signal, and R2 image signal included in the image signal for oxygen saturation. Specifically, the calculation value calculation unit 73 calculates the signal ratio of the B1 image signal to the G2 image signal (B1 / G2) and the signal ratio of the R2 image signal to the G2 image signal (R2 / G2) as calculated values ​​to be used for calculating oxygen saturation. It is preferable to logarithmically transform the signal ratio (B1 / G2) and the signal ratio (R2 / G2) into ln. In addition, the calculated values ​​may be color difference signals Cr, Cb, or saturation S, hue H, etc., calculated from the B1 image signal, G2 image signal, and R2 image signal.

[0076] The in-plane uniformity correction unit 74 corrects the brightness distribution of the spectral image. The brightness distribution of the spectral image may be the distribution of signal values ​​of pixels in the spectral image. The in-plane uniformity correction unit 74 reads a correction map corresponding to the spectral image from the map memory 71 and corrects the in-plane uniformity. For example, the correction coefficient for each pixel of the correction map for the first spectral image is multiplied by the signal value of each pixel in the B1 image before correction to calculate the corrected B1 image signal. Similarly, the corrected G2 image signal and the corrected R2 image signal are calculated. The calculation value calculation unit 73 may calculate calculation values ​​by calculation processing based on the corrected B1 image signal, the corrected G2 image signal, and the corrected R2 image signal.

[0077] The oxygen saturation calculation unit 75 refers to the oxygen saturation calculation table 76 and calculates the oxygen saturation based on the calculated values. The oxygen saturation calculation table 76 stores the correlation between the signal ratio (B1 / G2) and (R2 / G2), which are one of the calculated values, and the oxygen saturation. Regarding the correlation, when the signal ratio ln(B1 / G2) is represented on the Y axis and the signal ratio ln(R2 / G2) is represented on the X axis in a two-dimensional coordinate system, the state of oxygen saturation is represented by contour lines EL extending in the X-axis direction, and if the oxygen saturation is different, the contour lines EL will be distributed at different positions in the Y-axis direction. The oxygen saturation calculation table 76 includes the oxygen saturation calculation table 76a (see Figure 12), which is represented in a two-dimensional coordinate system.

[0078] The oxygen saturation calculation unit 75 refers to the oxygen saturation calculation table 76 and calculates the oxygen saturation corresponding to the signal ratio (B1 / G2) and (R2 / G2) for each pixel. For example, as shown in Figure 15, the oxygen saturation calculation table 76a is referred to, and if the signal ratio of a particular pixel is ln(B1 * / G2 * ), ln(R2 * / G2 * If ), then the signal ratio is ln(B1 * / G2 * ), ln(R2 * / G2 * The oxygen saturation corresponding to ) is "40%". Therefore, the oxygen saturation calculation unit 75 calculates the oxygen saturation of the specific pixel as "40%".

[0079] The color adjustment unit 77 generates an oxygen saturation image OP by performing a composite color processing that changes the color tone of the base image using the oxygen saturation calculated by the oxygen saturation calculation unit 75. In the base image, the color adjustment unit 77 maintains the color tone in areas where the oxygen saturation exceeds the threshold, and changes the color tone in areas where the oxygen saturation is below the threshold to a color tone that changes according to the oxygen saturation. As a result, the color tone of normal areas where the oxygen saturation exceeds the threshold is maintained, while only the color tone of abnormal areas below the threshold where the oxygen saturation is low is changed. This makes it possible to grasp the oxygen state of abnormal areas while observing the morphological information of normal areas.

[0080] In addition, the color adjustment unit 77 may generate an oxygen saturation image OP by pseudo-color processing, assigning a color corresponding to the oxygen saturation level, regardless of the actual oxygen saturation level. When pseudo-color processing is performed, the base image is not required.

[0081] The display control unit 63 controls the display of the oxygen saturation image OP generated by the oxygen saturation image generation unit 62 on the extended display 18.

[0082] [Method for generating correction maps] The correction map for correcting in-plane uniformity due to imaging system factors of the endoscope 12 is calculated using a reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope 12, and the brightness distribution of the reference region. The correction map has a correction coefficient for each pixel. The correction map is generated for each spectral image.

[0083] As shown in Figure 16, an integrating sphere IS is used to acquire a reference spectral image. The integrating sphere IS is a hollow sphere coated on its inner surface with a highly reflective white material such as barium sulfate. When the tip of the insertion section 12a of the endoscope 12 is inserted into the integrating sphere IS and illumination light is generated from the light source device 13, the illumination light emitted from the tip of the insertion section 12a is repeatedly reflected and diffused on the inner surface of the integrating sphere IS. Therefore, the inner surface of the integrating sphere IS becomes a region with a uniform luminance distribution, which is the distribution of light intensity, and becomes a reference region with a known luminance distribution. By imaging the inner surface of the integrating sphere IS in this state with the endoscope 12, a reference spectral image can be obtained.

[0084] Furthermore, the reference spectral image only needs to capture a reference region with a known brightness distribution; it does not need to be a region with a uniform brightness distribution.

[0085] Furthermore, the endoscope 12 is equipped with a scope ROM 12e (an example of "memory"). The generated correction map may be stored in the scope ROM 12e. ROM is an abbreviation for Read Only Memory.

[0086] Figure 17 is a flowchart illustrating the method for generating a correction map. The method for generating a correction map is achieved by the expansion processor device 17 executing a correction map generation program. The method for generating a correction map may also be achieved by a computer (not shown) executing a correction map generation program. In this case, the endoscope system 20 may obtain the correction map created by the computer via a network (not shown) or a storage medium (not shown). The expansion processor device 17 or the computer may read and execute a correction map generation program stored in a temporary, computer-readable storage medium.

[0087] The correction map generation method may be performed when the endoscope 12 is shipped from the factory, or it may be performed after shipment. If it is performed after shipment, the endoscope system 20 may be provided with an operating mode for performing the correction map generation method. The user can perform the correction map generation method by operating the mode switching switch 12c to set the endoscope system 20 to the correction map generation mode.

[0088] First, the user inserts the tip of the insertion section 12a of the endoscope 12 into the integrating sphere IS, as shown in Figure 16.

[0089] In step S1, the correction map generation unit 61 of the extended processor device 17 acquires a first reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope 12. Here, the light source processor 22 of the light source device 13 controls each LED 21a to 21e to emit the first blue light BL. The endoscope 12 illuminates the inner surface of the integrating sphere IS with the first blue light BL (an example of "reference light") in the illumination optical system 30, and the imaging optical system 31 receives the reflected light of the first blue light BL. As a result, the imaging processor 37 outputs a Br image signal from the B pixel of the imaging sensor 36 as the first reference spectral image.

[0090] In step S1, the correction map generation unit 61 may acquire the Br image signal from the B pixel of the imaging sensor 36 as a first reference spectral image, obtained by emitting a broadband first illumination light 56 ​​including a first blue light BL, a green light G, and a red light R and taking an image. The first reference spectral image may be an image obtained by interpolating the signal values ​​of the positions of the G pixel and R pixel of the imaging sensor 36.

[0091] The processor unit 14 transmits the first reference spectral image received from the endoscope 12 to the extended processor unit 17 via the image communication unit 51.

[0092] In step S2, the correction map generation unit 61 performs a process to remove high-frequency component noise (random noise) from the first reference spectral image acquired in step S1. The random noise removal process may be performed using an averaging filter, a median filter, or a low-pass filter. The following processes may use the signal values ​​of the first reference spectral image after random noise removal.

[0093] In step S3, the correction map generation unit 61 calculates a correction target value. Here, since the brightness distribution on the inner surface of the integrating sphere IS, which is the reference region, is uniform, the correction target value is a single value for the first reference spectral image. The correction map generation unit 61 may calculate the average value of the signal value for each pixel in the central region of the first reference spectral image as the correction target value. The central region of the first reference spectral image may be a circular region having an area of ​​(2 / 3) times the area of ​​the first reference spectral image.

[0094] If the luminance distribution of the reference region is not uniform, a correction target value should be calculated according to the known luminance distribution of the reference region.

[0095] In step S4, the correction map generation unit 61 calculates a correction coefficient for each pixel based on the correction target value calculated in step S3, and generates a correction map. The correction coefficient for each pixel may be calculated as (correction target value) / (signal value for each pixel in the first reference spectral image).

[0096] In step S5, the correction map generation unit 61 reduces the resolution of the correction map generated in step S4 as needed. For example, the correction map generation unit 61 may reduce the resolution of the correction map to 1 / 4 by averaging the signal values ​​of four adjacent pixels in the vertical and horizontal directions of the correction map to obtain a new signal value for one pixel. The correction map generation unit 61 may reduce the resolution if the file size of the correction map is too large for the storage capacity of the map memory 71 (see Figure 14) or the scope ROM 12e (see Figure 16).

[0097] In step S6, the correction map generation unit 61 saves the correction map generated in step S4. If the resolution was reduced in step S5, the correction map generation unit 61 saves the reduced-resolution correction map. The correction map generation unit 61 may save the correction map in the map memory 71 or in the scope ROM 12e of the endoscope 12. The correction map generation unit 61 may obtain the individual identifier of the endoscope 12 and save the correction map associated with the individual identifier of the endoscope 12. The individual identifier of the endoscope 12 may be stored in the scope ROM 12e or may be input from the user interface 16.

[0098] According to the method for generating a correction map, a correction map (an example of "first correction data") can be generated and saved to correct the brightness distribution of the first spectral image obtained by imaging an observation target irradiated with the first blue light BL using the endoscope 12. Furthermore, generating a correction map to correct in-plane unevenness of the endoscope 12 and saving the generated correction map to the scope ROM 12e corresponds to a manufacturing method for the endoscope 12.

[0099] In step S1, multiple first reference spectral images may be acquired. If there are multiple first reference spectral images, one first reference spectral image may be generated by averaging the signal values ​​for each pixel at the same position in the multiple first reference spectral images, and then the processing from step S2 onward may be carried out. Alternatively, one correction map may be generated by averaging the coefficients for each pixel of the multiple correction maps generated for each first reference spectral image. Furthermore, random noise removal processing may be performed by averaging the multiple first reference spectral images.

[0100] Here, we have described an example of generating a correction map (B correction map) to correct the brightness distribution of the first spectral image. In step S1, green light G is irradiated onto the inner surface of the integrating sphere IS and imaged with the endoscope 12. By acquiring the Gr image signal output from the G pixel of the imaging sensor 36 as the second reference spectral image, a G correction map (an example of "second correction data") can be generated and saved to correct the brightness distribution of the second spectral image obtained by imaging the observation target irradiated with green light G with the endoscope 12.

[0101] Similarly, in step S1, red light R is irradiated onto the inner surface of the integrating sphere IS and imaged with the endoscope 12. By acquiring the Rr image signal output from the R pixel of the imaging sensor 36 as a third reference spectral image, an R correction map (an example of "third correction data") can be generated and saved to correct the brightness distribution of the third spectral image obtained by imaging the observation target irradiated with red light R with the endoscope 12.

[0102] Alternatively, in step S1, the inner surface of the integrating sphere IS may be irradiated with a second illumination light 57 and imaged with the endoscope 12. The Gr image signal output from the G pixel of the imaging sensor 36 may be acquired as a second reference spectral image, and the Rr image signal output from the R pixel of the imaging sensor 36 may be acquired as a third reference spectral image, thereby generating a G correction map and an R correction map.

[0103] [Method for generating oxygen saturation images] Figure 18 is a flowchart showing an example of an oxygen saturation image generation method ("image processing method") performed in the oxygen saturation mode, one of the observation modes of the endoscope system 20. The oxygen saturation image generation method is achieved by the expansion processor device 17 executing an oxygen saturation image generation program. The expansion processor device 17 may also read and execute an oxygen saturation image generation program stored in a temporary, computer-readable storage medium.

[0104] When the mode switching switch 12c is set to oxygen saturation mode, in step S11, the calculation unit 73 obtains the individual identifier of the endoscope 12 and obtains a correction map from the map memory 71 that has been previously stored in association with the obtained individual identifier. The calculation unit 73 may also obtain the correction map from the scope ROM 12e of the endoscope 12. Here, the calculation unit 73 obtains a B correction map for correcting the B1 image signal, a G correction map for correcting the G2 image signal, and an R correction map for correcting the R2 image signal.

[0105] Next, the endoscope system 20 performs imaging in oxygen saturation mode. That is, the endoscope system 20 acquires a first illumination light image, a second illumination light image, and a third illumination light image. In step S12, the image acquisition unit 60a acquires the B1 image signal output from the B pixel of the imaging sensor 36 as the first spectral image from the first illumination light image. The first spectral image is an image that includes a wavelength range of 460 nm to 480 nm.

[0106] In step S13, the image acquisition unit 60a acquires the G2 image signal output from the G pixel of the imaging sensor 36 as the second spectral image from the second illumination light image. The second spectral image is an image that includes the wavelength range of 540 nm to 580 nm.

[0107] In step S14, the image acquisition unit 60a acquires the R2 image signal output from the R pixel of the imaging sensor 36 as a third spectral image from the second illumination light image. The third spectral image is an image that includes the wavelength range of 610 nm to 630 nm. In this way, multiple spectral images may be acquired from a single second illumination light image. Furthermore, each spectral image may be an image obtained by interpolating the signal values ​​of the positions of pixels of other colors in the imaging sensor 36.

[0108] In step S15, the in-plane uniformity correction unit 74 performs correction processing using a correction map corresponding to the signal value of each pixel. If the correction map has a low resolution, an expansion process may be performed in advance to increase the resolution of the correction map to the resolution of the spectral image. Here, the in-plane uniformity correction unit 74 corrects the in-plane uniformity for the first spectral image, the second spectral image, and the third spectral image using the B correction map, the G correction map, and the R correction map, respectively.

[0109] Specifically, the in-plane uniformity correction unit 74 calculates (pre-correction B1 signal value) × (correction coefficient of the B correction map) for each pixel of the first spectral image to obtain the corrected B1 signal value. The in-plane uniformity correction unit 74 also calculates (pre-correction G2 signal value) × (correction coefficient of the G correction map) for each pixel of the second spectral image to obtain the corrected G2 signal value. Furthermore, the in-plane uniformity correction unit 74 calculates (pre-correction R2 signal value) × (correction coefficient of the R correction map) for each pixel of the third spectral image to obtain the corrected R2 signal value.

[0110] In step S16, the oxygen saturation image generation unit 62 calculates the oxygen saturation of the blood being observed using the signal value corrected in step S15, and generates an oxygen saturation image by imaging the calculated oxygen saturation.

[0111] Specifically, the oxygen saturation calculation unit 75 refers to the oxygen saturation calculation table 76 and calculates the oxygen saturation corresponding to the signal ratio ln(R2 / G2) and signal ratio ln(B1 / G2) for each pixel. The color adjustment unit 77 generates an oxygen saturation image using the oxygen saturation calculated by the oxygen saturation calculation unit 75. Furthermore, the display control unit 63 displays the generated oxygen saturation image on the extended display 18.

[0112] According to the method for generating oxygen saturation images, it is possible to obtain an image showing the distribution of oxygen saturation after correcting for in-plane unevenness due to imaging system factors unique to each individual endoscope 12.

[0113] Here, an example of calculating the oxygen saturation of the blood being observed using the B1 image signal, G2 image signal, and R2 image signal has been described. However, information indicating the oxygen saturation of the blood being observed may also be calculated using only the B1 image signal and the G2 image signal. In this case, the oxygen saturation calculated by the oxygen saturation calculation unit 75 may be a relative value such as high, medium, and low.

[0114] Furthermore, while this explanation describes an example of correcting in-plane uniformity in the B1, G2, and R2 image signals, it is also possible to correct only the in-plane uniformity in the B1 image signal, or to correct the in-plane uniformity in both the B1 and G2 image signals.

[0115] As described above, in oxygen saturation mode, the endoscope system 20 acquires a first spectral image obtained by imaging a living body irradiated with the first light using the endoscope, and a second spectral image obtained by imaging a living body irradiated with the second light, which has a different wavelength band than the first light, using the endoscope. The endoscope system 20 then corrects the brightness distribution of the first spectral image with a correction map, and outputs an oxygen saturation image, which is an image showing the distribution of oxygen saturation in the blood of the living body, from the corrected first spectral image and the second spectral image. This makes it possible to achieve oxygen saturation imaging that corrects for in-plane unevenness caused by imaging system factors of the endoscope 12.

[0116] The endoscope system 20 may correct the brightness distribution of the second spectral image with a G correction map and output an image showing the distribution of oxygen saturation in the blood in the body from the corrected first spectral image and the corrected second spectral image.

[0117] Furthermore, the endoscope system 20 may acquire a third spectral image obtained by imaging a living body irradiated with a third light having a different wavelength band than the first and second lights using an endoscope, correct the brightness distribution of the third spectral image with an R correction map, and output an image showing the distribution of oxygen saturation of blood in the living body from the corrected first spectral image, the corrected second spectral image, and the corrected third spectral image.

[0118] [Other forms of imaging optical systems] The imaging optical system may include a first image sensor and a second image sensor having a different wavelength band for photoelectric conversion from the first image sensor. As shown in Figure 19, the imaging optical system 31A includes a dichroic filter (spectroscopic element) 100, imaging optical systems 102, 104, and 106, a normal imaging sensor 108, a specific imaging sensor 110, and a notch filter 112.

[0119] The standard imaging sensor 108 is a single-chip color image sensor similar to the imaging sensor 36. The specific imaging sensor 110 is a monochrome image sensor. The dichroic filter 100 has the property of transmitting the first blue light BL and reflecting the second blue light BS, green light G, and red light R. The dichroic filter 100 reflects the second blue light BS, green light G, and red light R from the light emitted from the observation target illuminated by the illumination light, allowing the standard imaging sensor 108 to receive them, and transmits the first blue light BL, allowing the specific imaging sensor 110 to receive it. Therefore, the endoscope system 20 can acquire spectral images of green light G and red light R from the standard imaging sensor 108, and can acquire spectral images of the first blue light BL from the specific imaging sensor 110.

[0120] Thus, with the endoscopic system 20 using the imaging optical system 31A, each spectral image can be acquired simultaneously (at the same frame timing).

[0121] The imaging optical system may be configured to combine multiple dichroic filters that reflect light in a specific wavelength band and transmit light in other wavelength bands with four monochrome imaging sensors. In this case, the four monochrome imaging sensors may be configured to acquire spectral images of violet light V and second blue light BS, a spectral image of first blue light BL, a spectral image of green light G, and a spectral image of red light R, respectively.

[0122] [Computer Configuration] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0123] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0124] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a set of program modules, each of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). It is also possible that a program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may contain information, It may be connected to other code segments or hardware circuits by sending and receiving data, arguments, parameters, or memory contents.

[0125] 〔others〕 The technical scope of the present invention is not limited to the scope described in the embodiments above. The configurations and other elements in each embodiment can be appropriately combined with those in each embodiment without departing from the spirit of the present invention. [Explanation of Symbols]

[0126] 12… Endoscopy 12a... Insertion part 12b...Operation unit 12c...Mode switching switch 12d…Zoom control switch 12e... Scope ROM 13...Light source device 14…Processor unit 15…Display 16…User Interface 17…Expansion processor device 18…Extended display 20… Endoscopy System 21...Light source section 21a…V-LED 21b…BS-LED 21c…BL-LED 21d…G-LED 21e…R-LED 22…Processor for light source 23...Optical path coupling part 24...Light Guide 30...Illumination optical system 31…Imaging Optical System 31A…Imaging Optical System 32…Illumination lens 35…Objective lens 36…Imaging sensor 37…Imaging Processor 40…CDS / AGC circuit 41…A / D converter 50…Image Processing Unit 51…Image Communications Department 52...Display Control Unit 53…Central Control Unit 55...white light 55a...Curve 55b…Curve 56…First illumination light 56a...Curve 56b…Curve 57…Second illumination light 58…Third illumination light 60a...Image acquisition unit 60b...Image Processing Unit 61... Correction map generation unit 62... Oxygen saturation image generation unit 63...Display Control Unit 71... Map Memory 72...Base image generation unit 73...Calculation unit 74...In-plane unevenness correction section 75...Oxygen saturation calculation unit 76...Table for calculating oxygen saturation 76a...Table for calculating oxygen saturation 77...Color tone adjustment section 100... Dichroic filter 102…Imaging Optical System 104…Imaging Optical System 106…Imaging Optical System 108...Normal imaging sensor 110...Specific imaging sensor 112... Notch filter EL...contour lines ELH…Contour lines ELL…contour line F6A…Bandwidth of the first illumination light F6B...bandwidth of the second illumination light F6C...bandwidth of the third illumination light IS…integrating sphere NP1…White light image NP2…Image equivalent to white light OP... Oxygen saturation image S1~S6…Steps for generating the correction map S11-S16...Steps for generating oxygen saturation images

Claims

1. Equipped with a processor, The aforementioned processor, The first spectral image is obtained by imaging a living organism irradiated with the first light using an endoscope, A second spectral image is obtained by imaging the living organism, which has been irradiated with a second light having a different wavelength band than the first light, using the endoscope. The brightness distribution of the first spectral image is corrected with first correction data that has been stored in advance in association with the endoscope. From the corrected first spectral image and the second spectral image, an image showing the distribution of oxygen saturation in the blood of the living organism is output. The first correction data is calculated using a first reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope, and the brightness distribution of the reference region. Image processing device.

2. The aforementioned processor, The brightness distribution of the second spectral image is corrected with second correction data that has been previously stored in association with the endoscope. From the corrected first spectral image and the corrected second spectral image, an image showing the distribution of oxygen saturation in the blood of the living organism is output. The second correction data is calculated using a second reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope, and the brightness distribution of the reference region. The image processing apparatus according to claim 1.

3. The aforementioned processor, A third spectral image is obtained by imaging the living organism, which has been irradiated with a third light having a different wavelength band than the first and second lights, using the endoscope. The brightness distribution of the third spectral image is corrected with third correction data that has been stored in advance in association with the endoscope. From the corrected first spectral image, the corrected second spectral image, and the corrected third spectral image, an image showing the distribution of oxygen saturation in the blood of the living organism is output. The third correction data is calculated using a third reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope, and the brightness distribution of the reference region. The image processing apparatus according to claim 2.

4. The first light has a peak wavelength in the range of 460 to 480 nm. The second light has a peak wavelength in the range of 540 to 580 nm. The image processing apparatus according to claim 1.

5. The aforementioned reference region is the inner surface of the integrating sphere, The luminance distribution of the aforementioned reference region is uniform. The image processing apparatus according to any one of claims 1 to 4.

6. The first reference spectral image is an image obtained by imaging the inner surface of the integrating sphere irradiated with reference light containing light in the wavelength band of the first light from the endoscope. The image processing apparatus according to claim 5.

7. The first correction data is stored in association with the individual identifier of the endoscope. The image processing apparatus according to any one of claims 1 to 4.

8. The first correction data is stored in the memory of the endoscope. The image processing apparatus according to claim 7.

9. The endoscope has an image sensor in which a first pixel group and a second pixel group, which has a different wavelength band for photoelectric conversion from the first pixel group, are arranged in two dimensions. The first spectral image is acquired using the first pixel group, The second spectral image is captured using the second pixel group. The image processing apparatus according to any one of claims 1 to 4.

10. The endoscope has a first image sensor and a second image sensor that has a different wavelength band for photoelectric conversion from the first image sensor. The first spectral image is captured using the first image sensor. The second spectral image is captured using the second image sensor. The image processing apparatus according to any one of claims 1 to 4.

11. An endoscope equipped with an image sensor, A light source that generates multiple irradiated lights with different wavelength bands, A processor device that controls the endoscope and the light source to image a living body irradiated with the light, An image processing apparatus according to any one of claims 1 to 4, Equipped with, Endoscopic system.

12. A method of image processing performed by a processor, The aforementioned processor, The first spectral image is obtained by imaging a living organism irradiated with the first light using an endoscope, A second spectral image is obtained by imaging the living organism, which has been irradiated with a second light having a different wavelength band than the first light, using the endoscope. The brightness distribution of the first spectral image is corrected with first correction data that has been stored in advance in association with the endoscope. From the corrected first spectral image and the second spectral image, an image showing the distribution of oxygen saturation in the blood of the living organism is output. The first correction data is calculated using a first reference spectral image obtained by imaging a reference region with a known brightness distribution using the endoscope, and the brightness distribution of the reference region. Image processing methods.

13. A program that causes a computer to execute the image processing method described in claim 12.

Images