Endoscope system, image processing device, operation method of endoscope system, and program for endoscope system

Abstract

To provide an endoscope system, an image processing device, an operation method of an endoscope system, and a program for an endoscope system capable of correcting data used for calculation of a degree of oxygen saturation more reliably and quickly.SOLUTION: An endoscope system for calculating a degree of oxygen saturation acquires an image of an observation object, sets a plurality of regions of interest in the image, executes determination processing for determining whether it is possible to correct data using the regions of interest with each of the plurality of regions of interest as objects, and performs correction of the data using the regions of interest for which it is determined that correction of the data is possible.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an endoscope system, an image processing device, an operation method of an endoscope system, and a program for an endoscope system. [Background technology]

[0002] In recent years, in the medical field using endoscopes, a technique is known for obtaining biological parameters such as oxygen saturation using an image of an observation target captured by an endoscope (hereinafter referred to as an endoscopic image). Oxygen saturation imaging is a technique for generating an image showing the oxygen saturation of hemoglobin contained in the observation target (hereinafter referred to as an oxygen saturation image) using an endoscopic image.

[0003] In oxygen saturation imaging, in order to accurately calculate the oxygen saturation of an object of observation, data used to calculate the oxygen saturation is corrected using an endoscopic image. During the correction, correction of data used to calculate the oxygen saturation of the object of observation may fail due to halation occurring in the endoscopic image, or residue or residual liquid adhering to the object of observation. An endoscopic system that determines whether correction is successful or unsuccessful is known (Patent Document 1). Also, an endoscopic system having a correction mode that calculates a specific dye concentration from multiple image signals and performs a correction process on data used to calculate oxygen saturation is known (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-080246 A [Patent Document 2] International Publication No. 2023 / 119856 Summary of the Invention [Problem to be solved by the invention]

[0005] One effective method for accurately calculating the oxygen saturation of an object to be observed is to appropriately correct the data used to calculate the oxygen saturation. In addition, since this correction is often performed during endoscopic operation, it is preferable that it can be performed quickly.

[0006] The present invention aims to provide an endoscopic system, an image processing device, an operation method of an endoscopic system, and a program for an endoscopic system that can more reliably and quickly correct data used to calculate oxygen saturation. [Means for solving the problem]

[0007] The present invention is an endoscopic system that calculates the oxygen saturation of an object being observed using data used to calculate oxygen saturation, and is equipped with a processor. The processor acquires an image of the object being observed, sets multiple regions of interest in the image, and performs a determination process for each of the multiple regions of interest to determine whether data can be corrected using the region of interest. The processor corrects the data by using a region of interest for which it is determined that data correction is possible.

[0008] It is preferable that the determination process includes a region reliability calculation process that calculates a region reliability based on the region of interest for each of the multiple regions of interest, and a representative region determination process that determines a representative region by comparing the region reliability for each of the multiple regions of interest with each other.

[0009] The region reliability is preferably calculated using pixel reliability calculated for each pixel included in each of the multiple regions of interest.

[0010] The determination process preferably includes a region reliability comparison process for comparing the region reliability of the representative region determined by the representative region determination process with a preset region reliability threshold value.

[0011] Preferably, the processor modifies the pixel reliability depending on the position of each pixel in the image, and the region reliability is calculated using the modified pixel reliability.

[0012] In the region reliability calculation process, it is preferable to calculate the region reliability after performing weighting according to the area of ​​the region of interest.

[0013] In the region reliability calculation process, it is preferable to calculate the region reliability after weighting the region of interest according to its distance from the center of the image.

[0014] The processor preferably sets the plurality of regions of interest each having substantially the same shape.

[0015] The processor preferably sets the plurality of regions of interest each having substantially the same area.

[0016] The processor preferably sets regions of interest that are different shapes and / or different areas based on the location of the regions of interest in the image.

[0017] The processor preferably sets the regions of interest such that at least two of the plurality of regions of interest include the same pixel of the image.

[0018] The processor preferably sets the plurality of regions of interest such that each region of interest includes different pixels of the image.

[0019] The processor preferably performs control to notify the user when there is no region of interest determined to be capable of correcting data.

[0020] The processor preferably controls the display to show those regions of interest for which it has been determined that the data can be corrected.

[0021] It is preferable that the processor performs control so that pixels whose pixel reliability is equal to or greater than a preset pixel reliability threshold are displayed on the display in a manner that makes them distinguishable from other pixels.

[0022] If correction of the data has been performed, the processor preferably calculates the oxygen saturation of the subject by using the corrected data.

[0023] If there is no region of interest for which it has been determined that correction of data is possible, the processor preferably calculates the oxygen saturation of the subject by using uncorrected data.

[0024] The processor preferably performs the correction of the data by using image information in those regions of interest where it has been determined that correction of the data is possible.

[0025] The image information is preferably a specific dye concentration of a specific dye other than hemoglobin in blood.

[0026] The specific dye is preferably a yellow dye.

[0027] The image processing device of the present invention is an image processing device that calculates the oxygen saturation level of an object being observed, and is equipped with a processor. The processor acquires an image of the object being observed, sets multiple regions of interest in the image, and performs a determination process for each of the multiple regions of interest to determine whether data can be corrected using the region of interest. The processor corrects the data by using the region of interest for which it has been determined that data correction is possible.

[0028] The operation method of the endoscopic system of the present invention is a method for calculating the oxygen saturation of an object to be observed, and includes the steps of acquiring an image of the object to be observed, setting a plurality of regions of interest in the image, performing a determination process for each of the plurality of regions of interest to determine whether data correction using the region of interest is possible, and correcting the data by using the region of interest determined to be suitable for data correction.

[0029] The program for an endoscopic system of the present invention is a program for an endoscopic system that calculates the oxygen saturation of an object being observed, and provides a computer with the following functions: acquiring an image of the object being observed; setting multiple regions of interest in the image; performing a determination process for determining whether data can be corrected using each of the multiple regions of interest; and correcting the data by using a region of interest that has been determined to be capable of data correction. Effect of the Invention

[0030] According to the present invention, data used for calculating oxygen saturation can be corrected more reliably and quickly. [Brief description of the drawings]

[0031] [Figure 1] FIG. 2 is an explanatory diagram for explaining a region of interest. [Diagram 2] FIG. 1 is a schematic diagram of an endoscopic system using a laparoscope. [Diagram 3] FIG. 3A is an explanatory diagram for explaining the display on the display in normal mode, and FIG. 3B is an explanatory diagram for explaining the display on the extended display in normal mode. [Figure 4] FIG. 4(A) is an explanatory diagram for explaining the display on the display in the oxygen saturation mode, and FIG. 4(B) is an explanatory diagram for explaining the display on the extended display in the oxygen saturation mode. [Diagram 5] FIG. 11 is an explanatory diagram illustrating a display of a message on an extended display. [Figure 6] 2 is a block diagram showing the functions of the endoscope system. FIG. [Figure 7] 1 is a graph showing an emission spectrum of white light. [Figure 8] 8A is a graph showing the emission spectrum of the first illumination light, FIG. 8B is a graph showing the emission spectrum of the second illumination light, and FIG. 8C is a graph showing the emission spectrum of the green light G. [Figure 9] 4 is a graph showing the spectral sensitivity of an image sensor. [Figure 10]11 is a table showing illumination and acquired image signals in a normal mode. [Figure 11] 11 is a table showing illumination and acquired image signals in an oxygen saturation mode or a correction mode. [Figure 12] 11A and 11B are explanatory diagrams for explaining light emission control and display control in the oxygen saturation mode or the correction mode. [Figure 13] 1 is a graph showing the reflectance spectrum of hemoglobin that varies depending on blood concentration. [Figure 14] 1 is a graph showing the reflection spectrum of hemoglobin and the absorption spectrum of a yellow dye that vary depending on the concentration of the yellow dye. [Figure 15] 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal in the case where there is no effect of a yellow pigment. [Figure 16] 1 is a graph showing contour lines representing oxygen saturation. [Figure 17] 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of the X-axis value indicating the signal ratio ln(R2 / G2) and the Y-axis value indicating the signal ratio ln(B1 / G2). [Figure 18] 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal when there is an influence of a yellow pigment. [Figure 19] FIG. 13 is an explanatory diagram showing the oxygen saturation with and without a yellow dye when the observation subject has the same oxygen saturation. [Figure 20] 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a B3 image signal, a G2 image signal, a G3 image signal, an R2 image signal, and a B2 image signal when influenced by yellow pigment. [Figure 21] 1 is a graph showing a surface representing oxygen saturation as a function of yellow dye. [Figure 22] 1 is an explanatory diagram of the state of oxygen saturation expressed in three-dimensional coordinates of X, Y, and Z, expressed in two-dimensional coordinates of X and Y. FIG. [Figure 23] 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of the X-axis value indicating the signal ratio ln(R2 / G2), the Y-axis value indicating the signal ratio ln(B1 / G2), and the Z-axis value indicating the signal ratio ln(B3 / G3). [Figure 24] FIG. 2 is a block diagram showing the functions of the extended processor device; [Diagram 25] FIG. 2 is a block diagram showing functions of an image processing unit. [Figure 26] FIG. 11 is an explanatory diagram showing a method for calculating oxygen saturation. [Figure 27] FIG. 11 is an explanatory diagram showing a method for generating a contour line corresponding to a specific dye concentration. [Figure 28] FIG. 4 is a block diagram showing the functions of a table correction unit. [Figure 29] 4 is a block diagram showing functions of a determination processing unit. FIG. [Diagram 30] FIG. 11 is an explanatory diagram illustrating a method for performing a determination process by serial processing. [Diagram 31] This is an explanatory diagram explaining a method of performing judgment processing by serial processing, and is an explanatory diagram explaining that the region of interest to be judged changes from the region of interest shown in Figure 31(A), to the region of interest shown in Figure 31(B), to the region of interest shown in Figure 31(C), to the region of interest shown in Figure 31(D). [Diagram 32] 13 is a graph showing the relationship between the pixel value and the reliability of a G2 image signal. [Diagram 33] 13 is a graph showing reliability due to bleeding, etc. [Diagram 34] 13 is a graph showing reliability due to fat, etc.; [Diagram 35] FIG. 11 is an explanatory diagram for explaining a method for determining a representative region. [Diagram 36] FIG. 11 is an image diagram of an endoscopic image showing a representative region. [Figure 37] FIG. 4 is an explanatory diagram illustrating a function of a determination processing unit. [Figure 38] FIG. 13 is an explanatory diagram for illustrating a case where a correctable region is displayed using a threshold value. [Figure 39]FIG. 13 is an explanatory diagram of a case where a message is displayed using a threshold value. [Diagram 40] FIG. 13 is an explanatory diagram illustrating a circular region of interest. [Diagram 41] FIG. 13 is an explanatory diagram illustrating the use of different shaped regions of interest in an endoscopic image. [Diagram 42] FIG. 11 is an explanatory diagram illustrating an example of a method of determination processing. [Diagram 43] FIG. 11 is an explanatory diagram illustrating an example of a method of determination processing. [Diagram 44] 11 is a graph showing correction data for correcting calculated pixel reliability. [Diagram 45] 11 is a graph showing weighting applied to region reliability depending on the area of ​​a region of interest. [Figure 46] 11 is a graph showing weighting of region reliability according to region distance. [Figure 47] FIG. 11 is an image diagram showing a group of correction images by contours. [Figure 48] 4 is a flowchart illustrating a process of the endoscope system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] An example of an embodiment of an endoscope system of the present invention will be described. First, the process of obtaining the following example of the embodiment will be described. Endoscopic oxygen saturation imaging is a technique for calculating hemoglobin oxygen saturation from a small amount of spectral information of visible light. In order to calculate oxygen saturation, for example, an object to be observed is photographed using a plurality of illumination lights having different wavelength bands. At least, light in a wavelength band in which oxygenated hemoglobin and reduced hemoglobin have different absorption coefficients is used as illumination light to photograph the object to be observed. Then, a predetermined calculation value is calculated using pixel values ​​of the obtained image, and the oxygen saturation of the object to be observed is determined using an oxygen saturation calculation table that indicates a correlation that corresponds the calculation value to the oxygen saturation. The correlation in the oxygen saturation calculation table is data used to calculate oxygen saturation.

[0033] When yellow pigments are present in the observation subject, the yellow pigments affect the absorption of hemoglobin in the blood, resulting in a problem of deviation in the calculated oxygen saturation value. The conditions in which yellow pigments are present may vary depending on the various parts of the body (esophagus, stomach, large intestine, etc.) and individual differences in the patient (gender, age, etc.).

[0034] Even in such a case, the oxygen saturation calculation table is corrected in order to calculate the oxygen saturation more accurately (Patent Document 2). To correct the oxygen saturation calculation table, it is necessary to use an endoscopic image that has been appropriately photographed of the observation target. In this case, with regard to the endoscopic image, "appropriately photographed" means that the image has been photographed in such a way that the oxygen saturation calculation table can be corrected.

[0035] In response to this, a technique is known that notifies the user when correction of the oxygen saturation calculation table fails due to halation, movement, etc. (Patent Document 1). In particular, when yellow pigments are present in the observation subject, these yellow pigments affect the absorption of hemoglobin in the blood, causing a problem that the calculated oxygen saturation value is deviated. In response to this, a technique is known that displays this on a display (Patent Document 2).

[0036] However, with the above technology, it may be necessary to find an endoscopic image that does not fail to correct the oxygen saturation calculation table by trial and error, which may take a lot of time to correct the oxygen saturation calculation table or increase the workload of the endoscope user (hereinafter referred to as the user).

[0037] In the endoscope system, image processing device, operation method of the endoscope system, and program for the endoscope system of the present invention, multiple regions of interest are set in an endoscope image, and it is determined whether the data used for calculating oxygen saturation can be corrected for each region of interest. Therefore, even if multiple parts are captured in the endoscope image, halation occurs, or bleeding occurs in the observation subject, it is determined whether the region of interest included in the endoscope image is a region in which the oxygen saturation calculation table can be corrected, and the oxygen saturation calculation table is corrected using the region in which the oxygen saturation calculation table can be corrected, so that the oxygen saturation calculation table can be corrected more reliably. In addition, since these processes are performed automatically, the user does not need to perform excessive trial and error, and can quickly identify a region in the endoscope image where the correction of the oxygen saturation calculation table is more reliably successful and then perform the correction. Therefore, according to the present invention, the oxygen saturation calculation table, which is data used for calculating oxygen saturation, can be corrected more reliably and quickly.

[0038] The endoscope system of the present invention is an endoscope system that calculates the oxygen saturation of an observation target using data used for calculating the oxygen saturation, and includes a processor device. The processor device is specifically a computer including a processor, a memory, and the like. An extended processor device may be connected to the processor device, and the extended processor device may be configured to realize the functions of the processor device. The processor device may also be configured to realize the functions of the extended processor device. The processor device or the extended processor device includes an image processing unit, and the image processing unit includes an image acquisition unit, an area setting unit, a determination processing unit, and a correction unit.

[0039] The image acquisition unit acquires an image of an observation target. The observation target is imaged by an endoscope such as a flexible endoscope or a rigid endoscope. The observation target is, for example, an organ such as the large intestine or small intestine imaged by a laparoscope, which is a rigid endoscope, and specifically, the serosal membrane of the organ. The image acquisition unit acquires, for example, an endoscopic image captured by the laparoscope. The region setting unit sets multiple regions of interest in the image acquired by the image acquisition unit.

[0040] As shown in Fig. 1, the region setting unit sets a plurality of regions of interest 11 in an endoscopic image 10. In Fig. 1, 16 regions of interest 11 are set in the endoscopic image 10. Note that in Fig. 1, a region with diagonal lines is one region of interest 11. Also, in Fig. 1, to avoid complexity, only some of the regions of interest 11 are labeled with reference numerals. It is sufficient that a plurality of regions of interest 11 can be set in an endoscopic image, and the shape, size, etc. of the regions of interest 11 can be set arbitrarily.

[0041] The determination processing unit performs a determination process for determining whether or not data used to calculate oxygen saturation can be corrected for each of a plurality of regions of interest 11 obtained by dividing the endoscopic image 10. An example of data used to calculate oxygen saturation is an oxygen saturation calculation table. A method for correcting the oxygen saturation calculation table uses image information of the endoscopic image 10. Therefore, in the case of FIG. 1, the determination processing unit performs a determination process for each of the 16 regions of interest 11. The determination process may be performed by performing parallel processing for each of the plurality of regions of interest 11, or may be performed by performing serial processing in which one or a specific number of the plurality of regions of interest 11 are processed in sequence.

[0042] The region of interest 11 in which the oxygen saturation calculation table can be corrected means a region in which the oxygen saturation can be appropriately obtained by correcting the oxygen saturation calculation table using this region of interest 11. When the oxygen saturation is calculated after correcting the oxygen saturation calculation table using the region of interest 11 in which the oxygen saturation calculation table can be corrected, the obtained oxygen saturation is more accurate than when the oxygen saturation is calculated without correction. Even if the oxygen saturation cannot be calculated without correcting the oxygen saturation calculation table, the oxygen saturation can be obtained by using the corrected oxygen saturation calculation table.

[0043] On the other hand, the region of interest 11 in which the oxygen saturation calculation table cannot be corrected means a region in which the oxygen saturation obtained is inappropriate when the oxygen saturation calculation table is corrected based on this region of interest 11. When the oxygen saturation is calculated after correcting the oxygen saturation calculation table using the region of interest 11 in which the oxygen saturation calculation table cannot be corrected, the oxygen saturation obtained is less accurate than when the oxygen saturation is calculated without correction. In addition, in the region of interest 11 in which the oxygen saturation calculation table cannot be corrected, correction of the oxygen saturation calculation table itself may not be possible.

[0044] The process of determining whether or not the oxygen saturation calculation table can be corrected in the region of interest 11 can be a process of setting an index relating to whether an appropriate oxygen saturation can be obtained when the oxygen saturation calculation table is corrected using the region of interest 11, and making a determination based on this index. A reliability can be set as the index. The reliability is set using image information that the region of interest 11 has.

[0045] The correction unit corrects the oxygen saturation calculation table using the region of interest 11 that is determined to be capable of correcting the oxygen saturation calculation table. If there is no region of interest 11 that is capable of correcting the oxygen saturation calculation table in the determination result of the determination process, the correction of the oxygen saturation calculation table is not performed. If there are multiple regions of interest 11 that are capable of correcting the oxygen saturation calculation table, the correction of the oxygen saturation calculation table is performed using a preset method. The preset method includes a method of using the most appropriate region of interest 11 to be used for correcting the oxygen saturation calculation table selected using the reliability, displaying the region of interest 11 that is determined to be capable of correcting the oxygen saturation calculation table on the display and allowing the user to specify one region of interest 11 to be used for correcting the oxygen saturation calculation table from the displayed region of interest 11, and correcting the oxygen saturation calculation table using all the regions of interest 11.

[0046] As described above, in the correction of data used for calculating oxygen saturation, the endoscope system performs a determination process for determining whether or not the data used for calculating oxygen saturation can be corrected for each of the multiple regions of interest 11 in the endoscope image. Therefore, even if the endoscope image includes a region that is problematic for correcting the oxygen saturation calculation table, the oxygen saturation calculation table is corrected after identifying a region that is not problematic, a region that is most suitable for correction, etc. Therefore, the oxygen saturation calculation table can be corrected more reliably without excessive trial and error. In addition, these processes in the image processing unit are automatically performed for each frame image, for example, and the user does not need to perform additional operations, so the oxygen saturation calculation table can be corrected quickly. Therefore, the endoscope system having the above configuration can correct the data used for calculating oxygen saturation more reliably and quickly.

[0047] Hereinafter, an embodiment will be described with reference to the drawings. As shown in FIG. 2, an endoscope system 20 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically or electrically connected to the light source device 13, and is electrically connected to the processor device 14. The extended processor device 17 is electrically connected to the light source device 13 and the processor device 14. The connections between these devices are not limited to being wired, and may be wireless. They may also be made via a network. In this specification, the display includes the display 15 and the extended display 18.

[0048] The endoscope 12 is a rigid endoscope that is inserted into a body cavity of a subject for performing a surgical treatment and captures images of organs in the body cavity from the serous membrane side, and is a laparoscope. The endoscope 12 may be a flexible endoscope that is inserted from the nose, mouth, or anus of the subject. In this specification, the subject means a subject into which the endoscope 12 is inserted. The subject has an observation target.

[0049] The endoscope 12 includes an insertion section 12a to be inserted into the abdominal cavity of a subject, and an operation section 12b provided at the base end portion of the insertion section 12a. An illumination optical system and an imaging optical system are built into a portion near the tip of the insertion section 12a (hereinafter referred to as the tip portion). These generate an image signal. The generated image signal is output to a processor device 14. The operation section 12b includes a mode switching switch 12c and a zoom operation switch 12d for zoom operation and the like. The mode switching switch 12c is used to switch the observation mode.

[0050] The endoscope system 20 has three modes: a normal mode, an oxygen saturation mode, and a tissue color correction mode (hereinafter, referred to as a correction mode). As shown in Fig. 3, in the normal mode, a white light image NP1 with natural coloring obtained by capturing an image of an observation target using white light as illumination light is displayed on the display 15, while nothing is displayed on the extended display 18.

[0051] 4, in the oxygen saturation mode, the oxygen saturation of the observation target is calculated based on an endoscopic image of the observation target, and the calculated oxygen saturation is visualized as an oxygen saturation image OP and displayed on the extended display 18. Also, in the oxygen saturation mode, due to the difference between the illumination light in the normal mode and the illumination light in the oxygen saturation mode, a white light equivalent image NP2, which has fewer short wavelength components than the white light image, is displayed on the display 15.

[0052] In the correction mode, the white light equivalent image NP2 is displayed on the extended display 18. In the correction mode, the correction process of the oxygen saturation calculation table is performed by using the region of interest 11 that is determined to be capable of correcting the oxygen saturation calculation table. When the normal mode is switched to the oxygen saturation mode, the correction mode may be automatically switched to once, and the correction process may be completed in the correction mode before switching to the oxygen saturation mode. When the normal mode is switched to the oxygen saturation mode, a notification may be given as to whether or not to perform the correction process in the correction mode, and the user may decide whether or not to perform the correction mode. As shown in FIG. 5, when the oxygen saturation mode is switched to, a message MS1 stating "Please perform the correction process" may be displayed on the extended display 18. When the correction process is required before starting the oxygen saturation mode, the user can switch to the correction mode without forgetting by the message MS1. After the correction process is completed, the oxygen saturation image is displayed on the extended display 18.

[0053] The processor device 14 controls the entire endoscope system 20. In addition, the processor device 14 generates an endoscopic image by performing image processing and the like on the image signal transmitted from the endoscope 12. The display 15 displays the endoscopic image and the like generated by the processor device 14.

[0054] The user interface 16 is an interface for performing input operations such as device settings for the processor device 14, and is a keyboard, a mouse, a microphone, a tablet, a foot switch, etc. If the display 15 or the extended display 18 has a touch panel, the display 15 or the extended display 18 may also function as the user interface 16.

[0055] 6, the light source device 13 includes a light source section 21 and a light source processor 22 that controls the light source section 21. The light source section 21 has, for example, a plurality of semiconductor light sources, which are turned on or off, and when turned on, the amount of light emitted by each semiconductor light source is controlled to emit illumination light that illuminates an object to be observed.

[0056] In this embodiment, the light source unit 21 has five color LEDs: a V-LED (Violet Light Emitting Diode) 21a, a BS-LED (Blue Short-wavelength Light Emitting Diode) 21b, a BL-LED (Blue Long-wavelength Light Emitting Diode) 21c, a G-LED (Green Light Emitting Diode) 21d, and an R-LED (Red Light Emitting Diode) 21e.

[0057] In place of these LEDs, the light source unit 21 may use a combination of an LD (Laser Diode), a phosphor, and a band-limiting filter, or a combination of a lamp such as a xenon lamp and a band-limiting filter.

[0058] The V-LED 21a emits violet light V (hereinafter referred to as V light) of 410 nm±10 nm. The BS-LED 21b emits second blue light BS of 450 nm±10 nm. The BL-LED 21c emits first blue light BL of 470 nm±10 nm. The G-LED 21d emits green light G in the green band. The central wavelength of the green light G is preferably 540 nm. The R-LED 21e emits red light R in the red band. The central wavelength of the red light R is preferably 620 nm. The central wavelength and peak wavelength of each of the LEDs 21a to 21e may be the same or different.

[0059] Both BL-LED21c and BS-LED21b are blue light sources that emit blue light. However, as described above, the central wavelength and wavelength band of the blue light emitted by BL-LED21c (hereinafter referred to as BL light) and the blue light emitted by BS-LED21b (hereinafter referred to as BS light) are different. The central wavelength and wavelength band of the BL light are the central wavelength and wavelength band in which the difference in the absorption coefficient between oxygenated hemoglobin and reduced hemoglobin is approximately maximized in the blue wavelength band.

[0060] The G-LED 21d is a green light source that emits broadband green light (hereinafter referred to as G light) with a central wavelength of 540 nm, and the R-LED 21e is a red light source that emits broadband red light (hereinafter referred to as R light).

[0061] The light source processor 22 independently controls the turning on or off of each of the LEDs 21a to 21e, the amount of light emitted when turned on, etc., by independently inputting a control signal to each of the LEDs 21a to 21e. The control of turning on or off, etc., by the light source processor 22 differs depending on the mode, and will be described in detail later.

[0062] The light emitted from each of the LEDs 21a to 21e is incident on a light guide 24 via an optical path combining unit 23 configured with a mirror, a lens, etc. The light guide 24 is built in the endoscope 12 and a universal cord (a cord that connects the endoscope 12 with the light source device 13 and the processor device 14). The light guide 24 propagates the light from the optical path combining unit 23 to the tip of the endoscope 12.

[0063] The endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and illumination light propagated by the light guide 24 is irradiated onto the observation object via the illumination lens 32. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Light from the observation object irradiated with the illumination light enters the imaging sensor 36 via the objective lens 35. As a result, an image of the observation object is formed on the imaging sensor 36.

[0064] The imaging sensor 36 is a color imaging sensor that captures an image of an object being illuminated with illumination light. Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. Therefore, when an object is photographed with the imaging sensor 36, three types of images, a B image, a G image, and an R image, are obtained. The imaging sensor 36 is preferably a color imaging sensor with a Bayer array in which the ratio of the number of B pixels, G pixels, and R pixels is 1:2:1.

[0065] A CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used as the image sensor 36. Also, instead of the primary color image sensor 36, a complementary color image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) may be used. When a complementary color image sensor is used, image signals of four colors, CMYG, are output, and by converting the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion, image signals of each of the RGB colors similar to those of the image sensor 36 can be obtained.

[0066] The imaging sensor 36 is driven and controlled by an imaging processor 37. The control of each mode in the imaging processor 37 will be described later. A CDS / AGC circuit 40 (Correlated Double Sampling / Automatic Gain Control) performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 36. The image signal passed through the CDS / AGC circuit 40 is converted into a digital image signal by an A / D converter 41 (Analog / Digital). The digital image signal after A / D conversion is input to the processor device 14.

[0067] The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing section 50, an image communication section 51, a display control section 52, and a central control section 53. Programs relating to various processes are incorporated in a program memory (not shown) in the processor device 14. The functions of the DSP 45, the image processing section 50, the image communication section 51, the display control section 52, and the central control section 53 are realized by the central control section 53, which is constituted by a processor, executing the programs in the program memory.

[0068] The DSP 45 performs various signal processing such as defect correction, offset processing, gain correction, demosaic, linear matrix processing, white balance processing, gamma conversion, YC conversion, and noise reduction on the image signal received from the endoscope 12. In the defect correction, the signal of a defective pixel of the imaging sensor 36 is corrected. In the offset processing, dark current components are removed from the image signal that has been subjected to the defect correction processing, and an accurate zero level is set. In the gain correction, the signal level of each image signal is adjusted by multiplying the image signal of each color after the offset processing by a specific gain. The image signal of each color after the gain correction is subjected to demosaic processing and linear matrix processing to improve color reproducibility.

[0069] After linear matrix processing, white balance processing is performed, and then gamma conversion processing is performed to adjust the brightness and saturation of each image signal. Then, YC conversion processing is performed, and the luminance signal Y and the color difference signals Cb and Rc are output to the DSP 45. The DSP 45 performs noise reduction processing using, for example, a moving average method or a median filter method.

[0070] The image processing unit 50 performs various image processing on the image signal from the DSP 45. The image processing includes color conversion processing such as 3×3 matrix processing, gradation conversion processing, table processing for three-dimensional oxygen saturation calculation, color enhancement processing, structure enhancement processing such as spatial frequency enhancement, and the like. The image processing unit 50 performs image processing according to the mode. In the normal mode, the image processing unit 50 performs image processing for the normal mode to generate a white light image NP1. In the oxygen saturation mode, the image processing unit 50 generates a white light equivalent image NP2 and transmits the image signal from the DSP 45 to the extension processor device 17 via the image communication unit 51. The extension processor device 17 generates an oxygen saturation image OP based on the transmitted image signal of the endoscopic image.

[0071] The display control unit 52 performs display control for displaying on the display 15 image signals such as the white light image NP1 and the oxygen saturation image OP from the image processing unit 50, and other information.

[0072] The extended processor device 17 receives image signals from the processor device 14 and performs various image processing. The extended processor device 17 functions as an image processing device. The image processing in the extended processor device 17 is performed in a correction mode and an oxygen saturation mode, and is a process for generating an oxygen saturation image.

[0073] In the oxygen saturation mode, the extended processor device 17 calculates the oxygen saturation and generates an oxygen saturation image OP that visualizes the calculated oxygen saturation. The generated oxygen saturation image OP is displayed on the extended display 18. In the correction mode, the extended processor device 17 performs a correction process related to the calculation of the oxygen saturation. More specifically, in the correction process, a specific dye concentration is calculated according to a user operation, and the oxygen saturation calculation table is corrected based on the calculated specific dye concentration. An example of the specific dye is a yellow dye. Therefore, the specific dye concentration is the concentration of the yellow dye in the image information included in the endoscopic image 10. The oxygen saturation mode and the correction mode performed by the extended processor device 17 will be described in detail later.

[0074] The on / off control in each mode will be described. In the normal mode, the V-LED 20a, BS-LED 20b, G-LED 20d, and R-LED 20e are simultaneously turned on to emit white light 55 including purple light V with a central wavelength of 410 nm, second blue light BS with a central wavelength of 450 nm, green light G with a broadband in the green band, and red light R with a central wavelength of 620 nm, as shown in Fig. 7. The graph in the figure shows the light intensity of each wavelength band.

[0075] In the oxygen saturation mode and the correction mode, three frames of light emission with different light emission patterns are repeated. In the first frame, as shown in FIG. 8(A), the BL-LED 20c, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit the first wideband illumination light 56 ​​including the first blue light BL with a central wavelength of 470 nm, the green light G with a broadband in the green band, and the red light R with a central wavelength of 620 nm. In the second frame, as shown in FIG. 8(B), the BS-LED 20b, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit the second blue light BS with a central wavelength of 450 nm, the green light G with a broadband in the green band, and the red light R with a central wavelength of 620 nm. In the third frame, as shown in FIG. 8(C), the G-LED 20d is turned on to emit the third wideband illumination light 58 including the green light G with a broadband in the green band. In the oxygen saturation mode, since the first and second frames are necessary to obtain an image signal required for calculating the oxygen saturation, light may be emitted only in the first and second frames.

[0076] As shown in Fig. 9, the B color filter BF provided at the B pixel of the image sensor 36 mainly transmits light in the blue band, specifically, light with a wavelength band of 380 to 560 nm (blue transmission band). The peak wavelength at which the transmittance is maximum is around 460 to 470 nm. The G color filter GF provided at the G pixel of the image sensor 36 mainly transmits light in the green band, specifically, light with a wavelength band of 450 to 630 nm (green transmission band). The R color filter RF provided at the R pixel of the image sensor 36 mainly transmits light in the red band, specifically, light with a wavelength band of 580 to 760 nm (red transmission band).

[0077] 10, in the normal mode, the imaging processor 37 controls the imaging sensor 36 to capture an image of an object being observed, illuminated with white light 55 consisting of purple light V, second blue light BS, green light G, and red light R, for each frame. This causes the B pixels of the imaging sensor 36 to output Bc image signals, the G pixels to output Gc image signals, and the R pixels to output Rc image signals.

[0078] 11, in the oxygen saturation mode, when the first illumination light 56 ​​including the first blue light BL, the green light G, and the red light R is illuminated on the observation object in the first frame, the imaging processor 37 outputs a B1 image signal from the B pixel of the imaging sensor 36, a G1 image signal from the G pixel, and an R1 image signal from the R pixel as the first illumination light image. When the second illumination light 57 including the second blue light BS, the green light G, and the red light R is illuminated on the observation object in the second frame, the imaging processor 37 outputs a B2 image signal from the B pixel of the imaging sensor 36, a G2 image signal from the G pixel, and an R2 image signal from the R pixel as the second illumination light image.

[0079] In the third frame, when the third illumination light 58, which is green light G, is illuminated onto the object to be observed, the imaging processor 37 outputs a B3 image signal from the B pixel of the imaging sensor 36, a G3 image signal from the G pixel, and an R3 image signal from the R pixel as the third illumination light image.

[0080] 12, in the oxygen saturation mode, the first illumination light 56 ​​is emitted in the first frame (1stF), the second illumination light 57 is emitted in the second frame (2ndF), and the third illumination light 58 is emitted in the third frame (3rdF), after which the second illumination light 57 is emitted in the second frame and the first illumination light 56 ​​is emitted in the first frame. An image NP2 equivalent to white light obtained based on the emission of the second illumination light 57 in the second frame is displayed on the display 15. An oxygen saturation image OP obtained based on the emission of the first to third illumination lights in the first to third frames is displayed on the extended display 18.

[0081] In the oxygen saturation mode, of the image signals for the above three frames, the B1 image signal included in the first illumination light image and the G2 and R2 image signals included in the second illumination light image are used. In the correction mode, in order to measure the concentration of a specific pigment (such as a yellow pigment) that affects the calculation accuracy of the oxygen saturation, in addition to the B1, G2, and R2 image signals, the B3 and G3 image signals included in the third illumination light image are used.

[0082] The B1 image signal includes image information on at least the first blue light BL from the light transmitted through the B color filter BF in the first illumination light 56. The B1 image signal (image signal for oxygen saturation) includes image information on a wavelength band B1 whose reflection spectrum changes with a change in the oxygen saturation of blood hemoglobin as image information on the first blue light BL. As the wavelength band B1, for example, as shown in Fig. 13, it is preferable to set the wavelength band B1 to a wavelength band of 460 nm to 480 nm including 470 nm at which the difference between the reflection spectrum of oxygenated hemoglobin shown by curves 55b and 56b and the reflection spectrum of reduced hemoglobin shown by curves 55a and 56a is maximized.

[0083] 13, curve 55a represents the reflectance spectrum of reduced hemoglobin when the blood concentration is high, curve 55b represents the reflectance spectrum of oxygenated hemoglobin when the blood concentration is high, while curve 56a represents the reflectance spectrum of reduced hemoglobin when the blood concentration is low, and curve 56b represents the reflectance spectrum of oxygenated hemoglobin when the blood concentration is low.

[0084] The G2 image signal includes image information of at least a wavelength band G2 related to green light G among the light transmitted through the G color filter GF in the first illumination light 56. The wavelength band G2 is preferably a wavelength band of 500 nm to 580 nm, for example, as shown in Fig. 13. The R2 image signal includes image information of at least a wavelength band R2 related to red light R among the light transmitted through the R color filter RF in the first illumination light 56. The wavelength band R2 is preferably a wavelength band of 610 nm to 630 nm, for example, as shown in Fig. 13.

[0085] As shown in FIG. 14, the image information of the wavelength band B1 contains image information on the first blue light BL, and the image information of the wavelength band B3 contains image information on the green light G. The image information on the first blue light BL and the green light G is image information in which the absorption spectrum of a specific pigment changes due to a change in the concentration of the specific pigment such as a yellow pigment. The change in the absorption spectrum of the specific pigment also causes a change in the reflection spectrum of hemoglobin. The curve 55a represents the reflection spectrum of reduced hemoglobin when there is no influence of the yellow pigment, and the curve 55c represents the reflection spectrum of reduced hemoglobin when there is an influence of the yellow pigment. As shown in these curves 55a and 55c, the reflection spectrum of reduced hemoglobin changes depending on the presence or absence of the yellow pigment (the same applies to the reflection spectrum of oxygenated hemoglobin). Therefore, the reflection spectrum of the wavelength bands B1 and B3 changes due to a change in the oxygen saturation of blood hemoglobin under the influence of the specific pigment such as a yellow pigment.

[0086] In an ideal case where there is no influence of a specific dye such as a yellow dye in an observation target using the endoscope 12, as shown in FIG. 15, the B1 image signal (denoted as "B1"), the G2 image signal (denoted as "G2"), and the R2 image signal (denoted as "R2") are each influenced by oxygen saturation dependency, blood concentration dependency, or brightness dependency. As described above, the B1 image signal includes the wavelength band B1 in which the difference between the reflection spectrum of oxygenated hemoglobin and the reflection spectrum of reduced hemoglobin is maximized, and therefore the oxygen saturation dependency that changes depending on the oxygen saturation is about "high". In addition, the B1 image signal has a blood concentration dependency that changes depending on the blood concentration of the signal as shown by the curves 55a, 55b and the curves 56a, 56b. In addition, the B1 image signal has a brightness dependency that changes depending on the brightness of the observation target. The degrees of dependency are indicated as "large," "medium," and "small," with "large" indicating a high degree of dependency compared to other image signals, "medium" indicating a medium degree of dependency compared to other image signals, and "small" indicating a low degree of dependency compared to other image signals.

[0087] The G2 image signal has a low dependency on oxygen saturation because the magnitude relationship between the reflectance spectrum of oxygenated hemoglobin and the reflectance spectrum of reduced hemoglobin is reversed in a wide wavelength band. The G2 image signal also has a high dependency on blood concentration, as shown by curves 55a, 55b and curves 56a, 56b. The G2 image signal also has a dependency on brightness, similar to the B1 image signal.

[0088] The R2 image signal does not change as much with oxygen saturation as the B1 image signal, but has a medium degree of oxygen saturation dependency. The R2 image signal has a low degree of blood concentration dependency, as shown by curves 55a, 55b and curves 56a, 56b. The G2 image signal has a brightness dependency, similar to the B1 image signal.

[0089] As described above, since the B1, G2, and R2 image signals all have brightness dependency, the G2 image signal is used as a normalized signal to generate an oxygen saturation calculation table 73a (see FIG. 16) that is an oxygen saturation calculation table of data for calculating oxygen saturation using a signal ratio ln(B1 / G2) obtained by normalizing the B1 image signal with the G2 image signal and a signal ratio ln(R2 / G2) obtained by normalizing the R2 image signal with the G2 image signal. Note that "ln" in the signal ratio ln(B1 / G2) is a natural logarithm (similar to the signal ratio ln(R2 / G2)).

[0090] When the relationship between the signal ratio ln(B1 / G2) and the oxygen saturation level is expressed in two-dimensional coordinates with the signal ratio ln(R2 / G2) on the X-axis and the signal ratio ln(B1 / G2) on the Y-axis, the oxygen saturation level is expressed by a contour line EL along the Y-axis direction as shown in Fig. 16. The contour line ELH represents an oxygen saturation level of "100%", and the contour line ELL represents an oxygen saturation level of "0%". The contour lines are distributed so that the oxygen saturation level gradually decreases from the contour line ELH to the contour line ELL (Fig. 16 shows the contour lines of "80%, " "60%, " "40%, " and "20%").

[0091] The X-axis value (signal ratio ln(R2 / G2)) and the Y-axis value (signal ratio ln(B1 / G2)) are affected by oxygen saturation dependency and blood concentration dependency, respectively. However, as for brightness dependency, as shown in FIG. 17, the X-axis value and the Y-axis value are normalized by the G2 image signal, and therefore are considered to be "none," meaning that they are not affected. For the X-axis value (denoted as "X"), the oxygen saturation dependency is considered to be about "medium," and the blood concentration dependency is considered to be about "large." On the other hand, for the Y-axis value (denoted as "Y"), the oxygen saturation dependency is considered to be about "large," and the blood concentration dependency is considered to be about "medium."

[0092] On the other hand, in a practical case where an object observed using the endoscope 12 is affected by a specific dye such as a yellow dye, as shown in FIG. 18, the B1 image signal (denoted as "B1"), the G2 image signal (denoted as "G2"), and the R2 image signal (denoted as "R2") are each affected by oxygen saturation dependency, blood concentration dependency, yellow dye dependency, or brightness dependency. The B1 image signal contains image information in which the absorption spectrum of a specific dye such as a yellow dye changes depending on the change in concentration, and therefore has a yellow dye dependency of about "large" that changes due to the yellow dye. In contrast, the G2 image signal has a small change due to the yellow dye compared to the B1 image signal, and therefore has a yellow dye dependency of about "small to medium". The R1 image signal has a small change due to the yellow dye, and therefore has a yellow dye dependency of about "small".

[0093] In addition, when the signal ratio ln(R2 / G2) is expressed on the X-axis and the signal ratio ln(B1 / G2) is expressed on the Y-axis in two-dimensional coordinates, even if the observation subject has the same oxygen saturation, the oxygen saturation StO2A when there is no yellow pigment and the oxygen saturation StO2b when there is a yellow pigment are expressed differently, as shown in Figure 19. The oxygen saturation StO2B appears to be shifted higher than the oxygen saturation StO2A due to the presence of the yellow pigment in the image signal.

[0094] Therefore, in order to accurately calculate the oxygen saturation even in the case of the yellow pigment dependency as described above, the B3 image signal and the G3 image signal included in the third illumination light image are used when correcting the data used to calculate the oxygen saturation. The B3 image signal includes image information related to the light transmitted through the B color filter BF in the third illumination light 58. The B3 image signal (specific pigment image signal) includes image information of the wavelength band B3 having sensitivity to a specific pigment other than hemoglobin such as a yellow pigment (see FIG. 14). Although the B3 image signal is not as sensitive to the specific pigment as the B1 image signal, it has a certain degree of sensitivity to the specific pigment. Therefore, as shown in FIG. 20, the B1 image signal has a "high" dependency on the yellow pigment, while the B3 image signal has a "medium" dependency on the yellow pigment. The B3 image signal has a "low" dependency on oxygen saturation, a "high" dependency on blood concentration, and a "high" dependency on brightness.

[0095] The G3 image signal also includes an image signal in the wavelength band G3 that has a certain degree of sensitivity to the specific dye, although it is not as sensitive to the specific dye as the B3 image signal (see FIG. 14). Therefore, the yellow dye dependency of the G3 image signal is "small to medium". The G3 image signal has "small" oxygen saturation dependency, "large" blood concentration dependency, and "present" brightness dependency. The B2 image signal also has "large" yellow dye dependency, so that the B2 image signal may be used instead of the B3 image signal when correcting data used to calculate oxygen saturation. The B2 image signal has "small" oxygen saturation dependency, "large" blood concentration dependency, and "present" brightness dependency.

[0096] When the relationship between the signal ratio ln(B1 / G2) and the signal ratio ln(R2 / G2), the yellow pigment, and the oxygen saturation is expressed in three-dimensional coordinates with the signal ratio ln(R2 / G2) on the X-axis, the signal ratio ln(B1 / G2) on the Y-axis, and the signal ratio ln(B3 / G3) on the Z-axis, as shown in FIG. 21, the curved surfaces CV0 to CV4 representing the oxygen saturation are distributed in the Z-axis direction according to the pigment concentration of the yellow pigment. The curved surface CV0 represents the oxygen saturation when the yellow pigment has a concentration of "0" (no effect of the yellow pigment). The curved surfaces CV1 to CV4 represent the oxygen saturation when the yellow pigment has concentrations of "1" to "4", respectively. The higher the concentration number, the higher the concentration of the yellow pigment. Note that, as shown in the curved surfaces CV0 to CV4, the higher the concentration of the yellow pigment, the lower the Z-axis value becomes.

[0097] When the state of oxygen saturation expressed in three-dimensional coordinates of X, Y, and Z as shown in FIG. 22(A) is expressed in two-dimensional coordinates of X and Y, as shown in FIG. 22(B), the regions AR0 to AR4 representing the state of oxygen saturation are distributed at different positions according to the concentration of the yellow pigment, respectively. The regions AR0 to AR4 represent the distribution of oxygen saturation when the concentration of the yellow pigment is "0" to "4", respectively. By defining a contour line EL representing oxygen saturation for each of these regions AR0 to AR4, it is possible to obtain the oxygen saturation corresponding to the concentration of the yellow pigment (see FIG. 16). Note that, as shown in the regions AR0 to AR4, the higher the concentration of the yellow pigment, the higher the value on the X axis and the lower the value on the Y axis.

[0098] As shown in FIG. 23, the X-axis value (signal ratio ln(R2 / G2)), the Y-axis value (signal ratio ln(B1 / G2)), and the Z-axis value (signal ratio ln(B3 / G3)) are subject to yellow pigment dependency. The X-axis value (denoted as "X") has a yellow pigment dependency of "small to medium", the Y-axis value (denoted as "Y") has a yellow pigment dependency of "large", and the Z-axis value (denoted as "Z") has a yellow pigment dependency of "medium". As for the Z-axis value, the oxygen saturation dependency is "small to medium" and the blood concentration dependency is "small to medium". As for the Z-axis value, the brightness dependency is "none" since it is normalized by the G3 image signal.

[0099] In this embodiment, in a practical case where the observation target is affected by a specific dye such as a yellow dye, the data used to calculate the oxygen saturation is corrected by selecting a plurality of data representing the oxygen saturation corresponding to the concentration of the yellow dye in the endoscopic image of the observation target. Therefore, in this case, correcting the data used to calculate the oxygen saturation in the correction mode specifically means selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) representing the oxygen saturation.

[0100] 24, the extension processor device 17 includes an image acquisition unit 60a and an image processing unit 60b. The image acquisition unit 60a receives an image signal transmitted from the processor device 14 via the image communication unit 51. The image processing unit 60b performs image processing to generate an oxygen saturation image.

[0101] 25, the image processing unit 60b includes an oxygen saturation image generating unit 61, a specific dye concentration calculating unit 62, a table correcting unit 63, and a display control unit 64. In the extended processor device 17, programs related to various processes are incorporated in a program memory (not shown). A central control unit (not shown) constituted by a processor executes the programs in the program memory, thereby realizing the functions of the oxygen saturation image generating unit 61, the specific dye concentration calculating unit 62, the table correcting unit 63, and the display control unit 64.

[0102] The oxygen saturation image generating unit 61 includes a base image generating unit 70, a calculation value calculating unit 71, an oxygen saturation calculating unit 72, an oxygen saturation calculation table 73, and a color tone adjusting unit 74. The base image generating unit 70 generates a base image based on an image signal from the processor device 14. The base image is used as a base for the oxygen saturation image OP. The base image is preferably an image that allows morphological information such as the shape of the observation target to be grasped. The base image is composed of a B2 image signal, a G2 image signal, and an R2 image signal. The base image may be a narrowband light image in which blood vessels or structures (ductal structures) are highlighted by narrowband light or the like.

[0103] The calculation value calculation unit 71 calculates a calculation value by calculation processing based on the B1 image signal, the G2 image signal, and the R2 image signal included in the oxygen saturation image signal. Specifically, the calculation value calculation unit 71 calculates a signal ratio B1 / G2 between the B1 image signal and the G2 image signal, and a signal ratio R2 / G2 between the R2 image signal and the G2 image signal, as calculation values ​​used to calculate the oxygen saturation. Note that it is preferable to logarithmize (ln) the signal ratios B1 / G2 and R2 / G2, respectively. In addition, the calculation values ​​may be color difference signals Cr and Cb calculated from the B1 image signal, the G2 image signal, and the R2 image signal, or saturation S, hue H, etc.

[0104] The oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation based on the calculated value. The oxygen saturation calculation table 73 stores the correlation between the signal ratios B1 / G2 and R2 / G2, which are one of the calculated values, and the oxygen saturation. When the correlation is expressed in two-dimensional coordinates with the signal ratio ln(B1 / G2) on the vertical axis and the signal ratio ln(R2 / G2) on the horizontal axis, the state of the oxygen saturation is expressed by the contour lines EL extending in the horizontal axis direction, and the contour lines EL are distributed at different positions in the vertical axis direction when the oxygen saturation differs (oxygen saturation calculation table 73a (see FIG. 16)). The oxygen saturation calculation table 73 includes the oxygen saturation calculation table 73a expressed in two-dimensional coordinates.

[0105] The oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation corresponding to the signal ratios B1 / G2 and R2 / G2 for each pixel. For example, as shown in FIG. 26, the oxygen saturation calculation table 73a refers to the oxygen saturation calculation table 73a and calculates the oxygen saturation corresponding to the signal ratios of a specific pixel as ln(B1 * / G2 * ), ln(R2 * / G2 * ), the signal ratio is ln(B1 * / G2 * ), ln(R2 * / G2 * ) is "40%." Therefore, the oxygen saturation calculator 72 calculates the oxygen saturation of the specific pixel as "40%."

[0106] The color tone adjustment unit 74 generates an oxygen saturation image by performing composite color processing to change the color tone of the base image using the oxygen saturation calculated by the oxygen saturation calculation unit 72. The color tone adjustment unit 74 maintains the color tone of the region in the base image where the oxygen saturation exceeds a threshold, and changes the color tone of the region where the oxygen saturation is below the threshold to a color tone that changes according to the oxygen saturation. This maintains the color tone of normal areas where the oxygen saturation exceeds the threshold, while changing only the color tone of abnormal areas where the oxygen saturation is below the threshold and becomes low, making it possible to grasp the oxygen status of abnormal areas under conditions where morphological information of normal areas can be observed.

[0107] In addition, the color tone adjustment unit 74 may generate an oxygen saturation image by pseudo color processing in which a color is assigned according to the oxygen saturation level, regardless of the level of the oxygen saturation level. When the pseudo color processing is performed, the base image is not necessary.

[0108] The specific dye concentration calculation unit 62 includes a specific dye concentration calculation table 75. In the correction mode, the specific dye concentration calculation unit 62 calculates the specific dye concentration based on a specific dye image signal including image information of a wavelength band having sensitivity to a specific dye other than hemoglobin in blood among the dyes included in the observation target. The specific dye includes, for example, a yellow dye such as bilirubin. The specific dye image signal preferably includes at least a B3 image signal. Specifically, the specific dye concentration calculation unit 62 calculates the signal ratios ln(B1 / G2), ln(G2 / R2), and ln(B3 / G3). Then, the specific dye concentration calculation unit 62 refers to the specific dye concentration calculation table 75 to calculate the specific dye concentration corresponding to the signal ratios ln(B1 / G2), ln(G2 / R2), and ln(B3 / G3).

[0109] Correlations between the signal ratios ln(B1 / G2), ln(G2 / R2), and ln(B3 / G3) and the specific dye concentrations are stored in the specific dye concentration calculation table 75. For example, when the ranges of the signal ratios ln(B1 / G2), ln(G2 / R2), and ln(B3 / G3) are divided into five stages, the specific dye concentrations "0" to "4" are associated with the signal ratios ln(B1 / G2), ln(G2 / R2), and ln(B3 / G3) in the five stages and stored in the specific dye concentration calculation table 75. Note that it is preferable to logarithmize (ln) the signal ratio B3 / G3.

[0110] The table correction unit 63 performs a table correction process to correct the oxygen saturation calculation table 73 based on the specific dye concentration as a correction process performed in the correction mode. In the table correction process, the correlation between the signal ratios B1 / G2 and R2 / G2 stored in the oxygen saturation calculation table 73 and the oxygen saturation is corrected. Specifically, when the specific dye concentration is "2", the table correction unit 63 generates a contour line EL representing the state of oxygen saturation in the area AR2 corresponding to the specific dye concentration of "2" among the areas AR0 to AR4 determined according to the specific dye concentration as shown in FIG. 27. The table correction unit 63 generates a corrected oxygen saturation calculation table 73b by correcting the oxygen saturation calculation table 73 so as to become the generated contour line EL.

[0111] When correcting the oxygen saturation calculation table 73, which is data used to calculate oxygen saturation, the table correction unit 63 sets multiple regions of interest in the acquired image and performs a determination process for each of the multiple regions of interest to determine whether or not the oxygen saturation calculation table 73 can be corrected using the region of interest. Based on the result of the determination process, the oxygen saturation calculation table 73 is corrected by using the region of interest that has been determined to be correctable. In this embodiment, the oxygen saturation calculation table 73 is corrected using a specific dye concentration.

[0112] As shown in FIG. 28, the table correction unit 63 includes a region setting unit 81, a judgment processing unit 82, and a correction unit 83. The region setting unit 81 sets a plurality of regions of interest 11 in the endoscopic image 10 acquired by the image acquisition unit 60a (see FIG. 1). It is preferable to set the shape, area, etc. of the region of interest 11 in consideration of the fact that when the oxygen saturation calculation table 73 is corrected using the region of interest 11, the oxygen saturation of a desired location in the endoscopic image 10 can be calculated more accurately, and also the calculation capacity of the extension processor device 17. Since it is easier to set the region of interest 11, it is preferable to set the regions of interest 11 each having approximately the same shape, and it is also preferable to set the regions of interest 11 each having approximately the same area. Here, approximately the same includes both being the same and being almost the same.

[0113] As an example, the region setting unit 81 sets the region of interest 11 in the endoscopic image 10 in the shape of a rectangle obtained by dividing the entire endoscopic image 10 into 16 parts (see FIG. 1). Here, the 16 regions of interest 11 have substantially the same shape and substantially the same area. Note that the region of interest 11 may be set in a specific region, rather than in the entire endoscopic image 10. The specific region may be, for example, near the center of the endoscopic image 10, or a part other than the outer periphery of the endoscopic image 10, etc.

[0114] The determination processing unit 82 performs a determination process to determine whether or not the oxygen saturation calculation table 73 can be corrected for each of the multiple regions of interest 11. As shown in FIG. 29 , the determination processing unit 82 includes a reliability calculation unit 91, a representative region determination unit 92, and a correction region determination unit 93.

[0115] In the determination process, when parallel processing is performed on all of the multiple regions of interest 11, the determination process is similarly performed on all of the multiple regions of interest 11 that have been set each time an endoscopic image 10 is acquired. When serial processing is performed to sequentially process one or a specific number of the multiple regions of interest 11, the determination process is performed on one or a specific number of regions of interest 11, and when this determination process is completed, the determination process is performed on the next one or a specific number of regions, and so on, until the determination process is completed on the entire endoscopic image 10 or the entire specific region.

[0116] As shown in Fig. 30, when serial processing is performed to process a plurality of regions of interest 11 one by one in sequence, the region of interest 11 to be subjected to the determination processing is moved as shown by the arrow among the regions of interest 11 set in the endoscopic image 10, so to speak, by scanning the endoscopic image 10 with the regions of interest 11 in the order of the arrows, whereby the determination processing can be performed. The region of interest 11a has a rectangular shape obtained by dividing the endoscopic image 10 into 16 parts. In the figure, the arrows indicate the order in which the determination processing is performed using the regions of interest 11.

[0117] Specifically, a method can be adopted in which a determination process for one of the multiple regions of interest 11 is first started, and after the determination process for this region of interest 11 is completed, a determination process for another region of interest 11 is performed, and this is repeated until the determination process for the entire endoscopic image 10 is completed.

[0118] As shown in FIG. 31(A), for example, a region of interest 11a is set in the upper left of the endoscopic image 10, and a judgment process is started. Next, as shown in FIG. 31(B), after the judgment process of the region of interest 11a is completed, a judgment process is executed for a region of interest 11b on the right of the region of interest 11a. In turn, a judgment process is executed for a region of interest 11 on the right of the region of interest 11b, and when the judgment process of the region of interest 11c set at the right end of the upper stage of the endoscopic image 10 is completed as shown in FIG. 31(c), a judgment process is executed for a region of interest 11d at the left end of the second stage of the endoscopic image 10 as shown in FIG. 31(d). Thereafter, in the same manner, judgment processes are executed for the regions of interest 11 in turn, and the judgment process set for the endoscopic image 10 can be completed by completing the judgment process for all of the multiple regions of interest 11 set.

[0119] Since the determination process is a process for determining whether or not the oxygen saturation calculation table 73 can be corrected for each of the multiple regions of interest 11, an index is set regarding whether an appropriate oxygen saturation level can be obtained when the oxygen saturation calculation table 73 is corrected using the region of interest 11, and a determination is made based on this index. When a reliability level set using image information held by the region of interest 11 is used as the index, the reliability level is calculated for each region of interest 11 in the determination process.

[0120] The reliability can be determined from image information of each of a plurality of pixels included in one region of interest 11. The reliability can be calculated for each pixel included in the region of interest 11, and the reliability of the region of interest 11 can be determined by averaging the reliability of the plurality of pixels included in the region of interest 11. It is preferable that the reliability is determined to have an effect on the correction of data used to calculate oxygen saturation. One type of reliability can be calculated and used, or multiple types can be calculated and used.

[0121] The reliability calculation unit 91 calculates at least one reliability that affects the correction of data used to calculate oxygen saturation for each pixel based on the B1 image signal, G1 image signal, and R1 image signal of each pixel included in the first illumination light image, or the B2 image signal, G2 image signal, and R2 image signal of each pixel included in the second illumination light image. The reliability is expressed, for example, by a decimal between 0 and 1. The smaller the reliability value, the lower the reliability, and the larger the reliability value, the higher the reliability. When the determination processing unit 82 calculates multiple types of reliability, the final reliability of each pixel can be determined by a method such as adopting the minimum reliability of the multiple types of reliability, or adopting a reliability calculated using the multiple types of reliability.

[0122] A factor affecting the correction of data used to calculate oxygen saturation is the pixel value in the endoscopic image. Therefore, the pixel value can be used in the calculation of the reliability. As shown in FIG. 32, the pixel value of the G2 image signal can be used for the pixel value affecting the correction accuracy of data used to calculate oxygen saturation. When the pixel value of the G2 image signal is not within a certain range Rx, the reliability tends to be lower than when the pixel value of the G2 image signal is within the certain range Rx. When the pixel value is not within the certain range Rx, it includes a high pixel value such as halation, and a very small pixel value such as a dark area. When the pixel value is not within the certain range Rx, the correction accuracy of data used to calculate oxygen saturation tends to be low, and the reliability is accordingly low. The G1 image signal may be used instead of the G2 image signal to calculate the reliability using the pixel value.

[0123] Further, disturbances are cited as factors that affect the correction accuracy of data used to calculate oxygen saturation. Disturbances include at least the presence of bleeding, fat, residue, mucus, or residual fluid in the observation target. These disturbances also cause the reliability to fluctuate. For bleeding, which is one of the disturbances, the reliability is determined according to the distance from the definition line DFX on a two-dimensional plane consisting of the vertical axis ln(B2 / G2) and the horizontal axis ln(R2 / G2) as shown in FIG. 33. Here, the reliability decreases as the coordinates obtained by plotting ln(B2 / G2) and ln(R2 / G2) on the two-dimensional plane based on the B2 image signal, the G2 image signal, and the R2 image signal are located on the lower right. Furthermore, when the coordinates are located in the upper left area of ​​the definition line DFX, the reliability according to the degree of bleeding is set to a fixed value that is a high reliability. In FIG. 33, ln represents the natural logarithm. B2 / G2 represents the signal ratio between the B2 image signal and the G2 image signal, and R2 / G2 represents the signal ratio between the R2 image signal and the G2 image signal.

[0124] As shown in FIG. 34, the reliability of fat, residue, residual liquid, or mucus contained in the disturbance is determined according to the distance from the definition line DFY on a two-dimensional plane consisting of a vertical axis ln(B1 / G1) and a horizontal axis ln(R1 / G1). Here, the reliability decreases as the coordinates of ln(B1 / G1) and ln(R1 / G1) plotted on the two-dimensional plane based on the B1 image signal, the G1 image signal, and the R1 image signal are located on the lower left side. When the coordinates are located in the upper right area of ​​the definition line DFY, the reliability according to the degree of fat is set to a fixed value that is a high reliability. In FIG. 34, ln represents the natural logarithm. B1 / G1 represents the signal ratio between the B1 image signal and the G1 image signal, and R1 / G1 represents the signal ratio between the R1 image signal and the G1 image signal.

[0125] The reliability calculation unit 91 calculates the reliability for each pixel included in the region of interest 11 using one or more types of reliability. For example, when using one type of reliability, a reliability using pixel values ​​(see FIG. 32), a reliability taking bleeding into consideration (see FIG. 33), or a reliability taking fat, residue, residual fluid, or mucus into consideration (see FIG. 34). When using multiple reliability, two or more types can be selected from the multiple types of reliability. The type of reliability may be selected based on the state of the observation target, differences in the site, etc.

[0126] The determination process preferably includes a region reliability calculation process for calculating a region reliability in each of the multiple regions of interest 11. In the region reliability calculation process, the reliability of each pixel (hereinafter referred to as pixel reliability) described above is used to calculate a region reliability, which is the reliability of each region of interest 11. The region reliability is calculated using pixel reliability of pixels included in each of the multiple regions of interest 11.

[0127] The region reliability can be calculated, for example, by arithmetically averaging the pixel reliability of each pixel included in the region of interest 11. As the arithmetic average, a simple average, a weighted average, a trimmed average, etc. can be used. When a simple average is used, the region reliability Rc can be calculated by summing up the pixel reliability Pc of the pixels included in the region of interest for all pixels included in the region of interest 11 and dividing the sum by the number of pixels N included in the region of interest. This relationship is expressed by the following formula (1).

[0128] Region reliability Rc = {Σ(pixel reliability Pc for pixels included in the region of interest)} / number of pixels included in the region of interest N (1)

[0129] The representative region determination unit 92 determines a representative region 85 from the multiple regions of interest 11. After the region reliability Rc is calculated for each of the multiple regions of interest 11, the representative region determination unit 92 can perform a representative region determination process to determine a representative region by comparing the region reliability Rc for each of the multiple regions of interest 11 with each other. The representative region can be determined to be the region with the highest region reliability Rc.

[0130] As shown in Fig. 35, the calculated region reliability Rc is compared for each of the multiple regions of interest 11 to determine a representative region 85. In the example shown in Fig. 35, the value of the region reliability Rc calculated for each region of interest 11 is indicated by a number. The region reliability Rc for region of interest 11a is 0.2, and the region reliability Rc for region of interest 11b is 0.5. Note that in the figure, to avoid complexity, symbols may be attached to only some of the objects.

[0131] 35, when the region reliability Rc calculated for each of the multiple regions of interest 11 is compared with each other, the region reliability Rc for region of interest 11j is 0.9, which is the highest. Therefore, the representative region determination unit 92 determines region of interest 11j as the representative region.

[0132] In addition, when the region reliability Rc is the same value for each of the multiple regions of interest 11, or when there are multiple regions of interest 11 with a value within a preset range, all of these regions of interest 11 may be determined as the representative region, or one of these regions of interest 11 may be determined as the representative region based on a preset criterion. Examples of the preset criterion include a criterion based on the position of the region of interest 11, such as determining a region of interest 11 that is close to the center of the endoscopic image 10 as the representative region, a criterion based on a specific type of reliability such as pixel values, a criterion based on the results of performing a weighted average, a trimmed average, etc. in addition to a simple average in calculating the reliability, and a criterion based on other statistics such as the median and the mode.

[0133] After determining the representative region 85, the representative region determining unit 92 displays the representative region in the endoscopic image 10 on the extended display 18 to show the user the region for correcting the oxygen saturation calculation table. As shown in FIG. 36, the representative region 85 is displayed on the extended display 18 by a display frame 86 representing the position and shape of the region of interest 11, superimposed on the endoscopic image 10. Note that the region of interest 11 is not displayed until the representative region 85 is determined. In addition, when the above-mentioned setting and determination process of the region of interest 11 is performed for each of the endoscopic images 10 obtained for each frame, the position of the representative region 85 may change. At this time, the display frame 86 indicating the representative region 85 also changes, so that the display frame 86 may be displayed moving like an animation in the endoscopic image 10 as time passes.

[0134] By displaying the representative area 85 on the extended display 18, the user can easily and at a glance see at which position on the endoscopic image 10 a more accurate oxygen saturation level can be calculated.

[0135] Note that the representative region determination unit 92 may determine the region with the highest region reliability Rc as the representative region 85, or may determine multiple regions of interest 11 with high region reliability Rc as the representative region 85. For example, multiple regions of interest 11 previously set from top to bottom in descending order of region reliability Rc may be determined as the representative region 85. For example, three regions of interest 11 previously set from top to bottom in descending order of region reliability Rc may be determined as the representative region 85. In this case, the region reliability Rc for the multiple regions of interest 11 determined as the representative region 85 may be set to an average value of the region reliability Rc of these multiple regions of interest 11.

[0136] The correction region determining unit 93 may determine the representative region 85 as a correction region, which is the region of interest 11 in which the oxygen saturation calculation table can be corrected. The correction region is the region of interest 11 determined to be capable of correcting the oxygen saturation calculation table, and the display control unit 64 controls to display the correction region on the extended display 18. Therefore, in the case shown in FIG. 35, the correction region determining unit 93 determines the representative region 85 as a correction region, and displays the representative region 85 in the endoscopic image 10 by using a display frame 86. The correction unit 83 also corrects the oxygen saturation calculation table using the correction region.

[0137] The correction region determining unit 93 may determine that the representative region 85 is a correctable region of interest 11 after examining whether or not the oxygen saturation calculation table can be corrected using the region of interest 11. In this case, the correction region determining unit 93 compares the region reliability Rc of the representative region 85 with a preset region reliability threshold value to determine whether or not the representative region 85 is a region of interest 11 for which the oxygen saturation calculation table can be corrected.

[0138] 37, the determination processing unit 82 may further include a region reliability threshold storage unit 94. The region reliability threshold storage unit 94 stores a region reliability threshold for comparison with the region reliability Rc in the representative region 85 determined by the representative region determination process. The correction region determination unit 93 performs a process of determining the correction region using the stored region reliability threshold.

[0139] The region reliability threshold storage unit 94 includes a plurality of region reliability thresholds, and the correction region determination unit 93 pre-sets a corresponding region reliability threshold according to the type of region reliability Rc of the representative region 85 to be compared. The corresponding region reliability threshold is preferably a threshold for reliability calculated in the same manner as the region reliability Rc to be compared.

[0140] The correction region determination unit 93 compares the region reliability Rc in the region of interest 11j that is to be the representative region 85 with a set region reliability threshold, and if the region reliability Rc is equal to or greater than the region reliability threshold, determines the representative region 85 as a correction region. On the other hand, if the region reliability Rc is smaller than the region reliability threshold, determines the representative region 85 as an uncorrectable region in which correction of the oxygen saturation calculation table is not possible.

[0141] 38, the correction region determination unit 93 sets a region reliability threshold Tc corresponding to the region reliability Rc of the region of interest 11 to 0.9 in advance (Tc1). When the region reliability Rc of the region of interest 11j serving as the representative region 85 is 0.9, the region reliability Rc of the region of interest 11j is equal to or greater than the region reliability threshold Tc, so that the correction region determination unit 93 determines the region reliability Rc to be a correction region 87. The correction region 87 is displayed on the extended display 18 by being superimposed on the endoscopic image 10 by a display frame 86. The correction unit 83 also corrects the oxygen saturation calculation table 73 using the correction region 87.

[0142] 39, the correction region determination unit 93 sets the region reliability threshold Tc corresponding to the region reliability Rc in the region of interest 11 to 0.95 in advance (Tc2). When the region reliability Rc of the region of interest 11j that is the representative region 85 is 0.9, the region reliability Rc of the region of interest 11j is smaller than the region reliability threshold Tc, so that the correction region determination unit 93 determines that the region reliability Rc is an uncorrectable region. The uncorrectable region is not displayed on the extended display 18. Furthermore, when there is no correction region 87, the correction unit 83 does not correct the oxygen saturation calculation table.

[0143] Furthermore, when there is no region of interest 11 determined to be capable of correcting the oxygen saturation calculation table 73, i.e., when there is no correction region 87, the correction region determining unit 93 may perform control to notify the user that there is no region of interest 11 determined to be capable of correcting the oxygen saturation calculation table 73. The notification to the user may be in a form that the user can recognize, other than displaying text such as a display on a display, and may be displaying an icon or the like, notifying by voice, etc. As shown in FIG. 39, not displaying a display frame 86 indicating the correction region 87 is also one form of notification to the user.

[0144] As another mode of informing the user, as shown in FIG. 39, an operation guidance MS2 for generating a correction region 87 may be displayed. The operation guidance MS2 may be guidance consisting of text or the like for generating a correction region 87 in the endoscopic image 10. The content of the guidance is preferably guidance for making the reliability of any of the regions of interest 11 in the endoscopic image 10 equal to or greater than the region reliability threshold Tc. For example, if the reliability is based on pixel values, it is preferable to display guidance for improving the pixel values ​​in the endoscopic image 10. The guidance may be something other than text, such as an icon or a voice notification.

[0145] The correction unit 83 corrects the oxygen saturation calculation table 73 by using the correction region 87. The correction of the oxygen saturation calculation table 73 is performed by using the image information of the correction region 87. In this embodiment, the correction of the oxygen saturation calculation table 73 is performed by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) representing the oxygen saturation. Therefore, the correction unit 83 corrects the oxygen saturation calculation table 73 by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) representing the oxygen saturation, as described above, by using the image information of the correction region 87.

[0146] Furthermore, when there is no region of interest 11 for which it has been determined that the oxygen saturation calculation table 73 can be corrected, the oxygen saturation calculation unit 72 preferably calculates the oxygen saturation of the observation subject by using the oxygen saturation calculation table 73 that is not corrected. This makes it possible to calculate a more accurate oxygen saturation without making undesirable corrections to the oxygen saturation calculation table 73.

[0147] Since the endoscope system 20 is configured as described above, even if the endoscope image 10 includes an area that is problematic in terms of correcting the oxygen saturation calculation table, the endoscope system 20 corrects the oxygen saturation calculation table 73 after identifying an area that is not problematic, an area that is most suitable for correction, etc. Therefore, the oxygen saturation calculation table 73 can be corrected more reliably without excessive trial and error. Furthermore, these processes in the image processing unit 60b are automatically performed, for example, for each frame image, and the user does not need to perform additional operations, so that the oxygen saturation calculation table 73 can be corrected quickly. Therefore, the endoscope system 20 can correct the data used to calculate oxygen saturation more reliably and quickly.

[0148] The region setting unit 81 may set the region of interest 11 in a shape other than a rectangle in the endoscopic image 10. Also, the region of interest 11 in the endoscopic image 10 may have a variable shape and / or area. In this case, it is preferable to set the regions of interest 11 in different shapes and / or different areas based on the position of the region of interest 11 in the endoscopic image 10.

[0149] Furthermore, the region setting unit 81 may set the regions of interest 11 such that at least two of the multiple regions of interest 11 include different pixels of the endoscopic image 10. Furthermore, it is preferable to set the regions of interest 11 such that at least two of the multiple regions of interest 11 include the same pixel of the endoscopic image 10. That is, in this case, the regions of interest 11 overlap each other.

[0150] As shown in FIG. 40, the region setting unit 81 may set circular regions of interest 11a to 11d, which are shapes other than a rectangle. In order to avoid the diagram from becoming complicated, only a part of the multiple regions of interest 11 is illustrated. The circular regions of interest 11 are set so as to cover the entire endoscopic image 10. The circular regions of interest 11a are set so as to include the same pixels and different pixels in the adjacent regions of interest 11b, and the same is true for each of the regions of interest 11b to 11d. When performing serial processing to process the multiple regions of interest 11 one by one in sequence, the target of the determination process is moved with respect to the endoscopic image 10 as shown by the arrow, so to speak, the determination process can be performed by scanning the endoscopic image 10 using the regions of interest 11 in the order of the arrows. This allows the determination process to be performed efficiently while still performing the determination process on almost the entire endoscopic image 10.

[0151] As shown in FIG. 41, the region setting unit 81 may set rectangular regions of interest 11e and 11f having the same rectangular shape but different areas. In order to avoid cluttering the figure, a part of the display frame displaying the region of interest 11 is illustrated. In the case of FIG. 41, the rectangle in the center of the endoscopic image 10 is set to have a smaller area than the rectangle in the peripheral area. In the endoscopic image 10, peripheral shading may occur, in which the amount of light in the peripheral area of ​​the image is insufficient, mainly due to the relationship with the light source, etc. By setting rectangular regions of interest having different areas, it is possible to deal with cases where the center of the endoscopic image 10 can be used as the correction region 87 rather than the peripheral area.

[0152] As shown in Fig. 42(A), the region setting unit 81 may set a plurality of regions of interest 11 in a state where the regions of interest 11 overlap each other. When the judgment processing unit 82 performs serial processing, it first performs judgment processing on the region of interest 11g, and then performs judgment processing on the region of interest 11h as shown in Fig. 42(B). The regions of interest 11g and 11h overlap within a range of about 1 / 2 of the long side of the rectangle. Next, as shown in Fig. 42(C), it performs judgment processing on the region of interest 11i which overlaps with the region of interest 11h within a range of about 1 / 2 of the long side of the rectangle. In the same manner, after completing the determination process for the region of interest 11 having the right edge of the endoscopic image, as shown in Fig. 42(D), a determination process is performed for region of interest 11j that overlaps with region of interest 11g over about 1 / 2 of the short side, and then, as shown in Fig. 42(E), a determination process is performed for region of interest 11k that overlaps with regions of interest 11g and 11h over about 1 / 2 of the long side and also overlaps with region of interest 11j over about 1 / 2 of the short side. Thereafter, the determination process is similarly performed in the order shown by the arrows.

[0153] In the case of parallel processing, the judgment process is performed at once on all of the multiple regions of interest 11 set in the endoscopic image 10, such as regions of interest 11g, 11h, 11i, 11j, and 11k. In the case where the judgment processing unit 82 performs the judgment process on the regions of interest 11 by parallel processing, the judgment process is performed at once in parallel on each of the multiple regions of interest 11 set in this manner. In this way, by setting the regions of interest 11 that overlap each other, the judgment process can be performed efficiently, while the judgment process on almost the entire endoscopic image 10 can be performed more precisely.

[0154] The region setting unit 81 may set the region of interest 11 in a part of the endoscopic image 10. Even if the region setting unit 81 sets the region of interest 11 in the entire endoscopic image 10, the judgment processing unit 82 may perform judgment processing on a part of the multiple regions of interest 11 set by the region setting unit 81. The part of the multiple regions of interest 11 can be arbitrarily set depending on the type of endoscopic image 10, the type of observation target, etc. For example, it is preferable to set the part of the multiple regions of interest 11 to the center part of the endoscopic image 10. This makes it possible to perform judgment processing efficiently in consideration of cases where peripheral shading occurs in the endoscopic image 10.

[0155] As shown in Figure 43, when the region setting unit 81 sets the region of interest 11 in the same manner as in the case of Figure 41, the judgment processing unit 82 performs judgment processing on the region of interest 11 included in the central region 95 of the endoscopic image 10, and may not perform judgment processing on the regions of interest 11 other than the central region 95.

[0156] Furthermore, the determination processing unit 82 may complete the determination processing for the endoscopic image 10 by performing a determination processing once for each of the multiple regions of interest 11 set by the region setting unit 81 for the endoscopic image 10, or may complete the determination processing by performing a determination processing multiple times for some or all of the multiple regions of interest 11. Furthermore, in each of the multiple determination processings, the region setting unit 81 may set regions of interest 11 having different shapes and / or areas, or may set regions of interest 11 having the same shape and / or area. This allows the determination processing to be performed more precisely and efficiently depending on the type of the endoscopic image 10 or the observation target, the purpose of correcting the oxygen saturation calculation table, and the like.

[0157] The reliability calculation unit 91 may correct the pixel reliability according to the position of each pixel in the endoscopic image 10. The corrected pixel reliability is set as the corrected pixel reliability. When the pixel reliability is corrected, the reliability calculation unit 91 calculates the area reliability using the corrected pixel reliability. As a method of correction according to the position of each pixel in the endoscopic image 10, it is preferable to perform correction according to the distance of each pixel from the center of the endoscopic image 10. When the horizontal axis of the endoscopic image 10 is x and the vertical axis is y, the distance can be calculated by the square root of the sum of the squares of the differences between the x-axis value and the y-axis value of the coordinate of the center of the endoscopic image 10 and the coordinate of interest.

[0158] The pixels for which pixel reliability is calculated can be divided into two categories, for example, "close distance" and "long distance", depending on the distance from the center of the endoscopic image 10, and the pixel reliability can be corrected by using different correction data for each category. The correction data is set in advance by examining the relationship between the pixel reliability of a plurality of pixels at different positions in the endoscopic image 10 and the oxygen saturation. At this time, the classification criteria for "close distance" pixels and "long distance" pixels are also set together with the creation of the correction data.

[0159] As shown in Fig. 44, the modification data indicates the relationship between the calculated pixel reliability, which is the pixel reliability before modification, and the modified pixel reliability. For example, the short distance modification data 96 is data in which the pixel reliability before modification and the modified pixel reliability are in a proportional relationship, regardless of the magnitude of the pixel reliability before modification. On the other hand, the long distance modification data 97 is modification data in which the increase in the modified pixel reliability is suppressed when the pixel reliability before modification is large compared to when the pixel reliability before modification is small.

[0160] By using the corrected pixel reliability, it becomes possible to more preferably calculate the region reliability Rc, and the accuracy of the finally obtained oxygen saturation can be improved.

[0161] In the region reliability calculation process, the reliability calculation unit 91 may perform weighting according to the area of ​​the region of interest 11 before calculating the region reliability. In the region reliability calculation process, the reliability calculation unit 91 may perform calculation of the distance of the region of interest 11 from the center of the endoscopic image 10 before calculating the region reliability. The distance of the region of interest 11 from the center of the endoscopic image 10 may be the distance between the center or center of gravity of the region of interest 11 and the center of the endoscopic image 10 (hereinafter, referred to as region distance). When the horizontal axis of the endoscopic image 10 is x and the vertical axis is y, the region distance can be calculated by the square root of the sum of the squares of the x-axis value and the y-axis value between the coordinates of the center of the endoscopic image 10 and the coordinates of the center or center of gravity of the region of interest 11.

[0162] The amount of weighting is determined by weighting data indicating the relationship between the amount of weighting and the area or regional distance of the region of interest 11. The weighting data is set by investigating in advance the relationship between the area or regional distance of the region of interest 11 and the oxygen saturation level.

[0163] 45, weighting data 98 ​​relating to the area of ​​region of interest 11 indicates the relationship between the area and weight of region of interest 11. For example, weighting data 98 ​​relating to area indicates a relationship in which the amount of weighting increases as the area of ​​region of interest 11 increases, and the proportion of weighting increases in areas where region of interest 11 is small compared to areas where the area is large.

[0164] 46, the weighting data 99 for area distance indicates the relationship between area distance and weight. For example, the weighting data 99 for area distance indicates a relationship in which the amount of weighting decreases as the distance increases, and the proportion of the weighting that decreases is greater in a range where the area distance is small than in a range where the area distance is large.

[0165] In addition, it is preferable that the reliability calculation unit 91 performs both weighting according to the area of ​​the region of interest 11 and weighting according to the region distance in the region reliability calculation process. Therefore, the weighted region reliability Rc is obtained by multiplying the calculated region reliability Rc by the weighting according to the area of ​​the region of interest 11 and the weighting according to the region distance.

[0166] In addition, when the representative region determination unit 92 determines multiple regions of interest 11 with high region reliability Rc as the representative region 85, and determines the region reliability Rc of the representative region 85 as the average value of the region reliability Rc of the multiple regions of interest 11, it is preferable to weight the region reliability Rc and then calculate the average region reliability Rc.

[0167] By weighting the region reliability Rc, it becomes possible to calculate the region reliability Rc more desirably, and the accuracy of the finally obtained oxygen saturation can be improved.

[0168] The display control unit 64 controls the display of the correction region 87 on the extended display 18, and may control the display of the correction region 87 by superimposing it on the endoscopic image 10 using the display frame 86 or the like, or may control the display of pixels included in the correction region 87, whose pixel reliability is equal to or greater than a preset pixel reliability threshold, on the extended display 18 in a manner that makes them distinguishable from other pixels. This allows the area on the endoscopic image 10 where it has been determined that the oxygen saturation calculation table 73 can be corrected to be grasped in more detail, for example, on a pixel-by-pixel basis.

[0169] 47, pixels included in the correction region 87 and having an image reliability equal to or greater than a preset pixel reliability threshold may be displayed in a color that is distinguishable from other pixels. The display may be done by coloring each pixel, or, to make it easier to recognize, a correction pixel group 100 may be formed by including several surrounding pixels around a pixel having an image reliability equal to or greater than the pixel reliability threshold, and the correction pixel group 100 may be displayed by filling the region of the correction pixel group 100 with a color or by an outline 101 indicating the region of the correction pixel group 100.

[0170] A series of processing steps in an endoscope system or the like will be described using a flowchart. As shown in Fig. 48, in the endoscope system 20, a correction mode is started when calculating the oxygen saturation of an observation target (step ST110). The image acquisition unit 60a acquires an endoscopic image 10 obtained by photographing the observation target with the endoscope 12 (step ST120). The region setting unit 81 sets a region of interest 11 in the endoscopic image 10 (step ST130).

[0171] The determination processing unit 82 performs a determination process for determining whether each region of interest 11 is a correctable region of interest 11. As the determination process, a region reliability Rc of each region of interest 11 is calculated (step ST140), a representative region 85 is determined based on the calculated region reliability Rc (step ST150), and a determination process is performed to compare the region reliability Rc in the representative region 85 with a region reliability threshold (step ST160). If the region reliability Rc is equal to or greater than the region reliability threshold, the representative region 85 is determined to be a correction region 87 that is a correctable region of interest 11 (Y in step ST170).

[0172] If the area reliability Rc is smaller than the area reliability threshold (N in step ST170), operation guidance MS2 for generating a correction area 87 is displayed in text form on the extended display 18 (step ST180), prompting the user to obtain a more appropriate endoscopic image, and the procedure from obtaining the endoscopic image (step ST120) is repeated.

[0173] The correction area 87 is used to correct the oxygen saturation calculation table (step ST190). When the correction of the oxygen saturation calculation table is completed, the system transitions to the oxygen saturation mode (step ST200), and the oxygen saturation of the endoscopic image 10 acquired upon the transition is calculated, and the generated oxygen saturation image is displayed (step ST210).

[0174] The functions of the extended processor device 17 may be implemented by the processor device 14, or both the extended processor device 17 and the processor device 14 may be implemented by the functions of the extended processor device 17. Therefore, the processor device having a processor that corrects the oxygen saturation calculation table to generate an oxygen saturation image may be only the processor device 14, only the extended processor device 17, or both the processor device 14 and the extended processor device 17, etc.

[0175] In the above embodiment, the hardware structure of the processing units that execute various processes, such as the DSP 45, image processing unit 50, image communication unit 51, display control unit 52, and central control unit 53 of the processor device 14, and the image acquisition unit 60a and image processing unit 60b of the extended processor device 17, are various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing various processes.

[0176] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Also, multiple processing units may be configured with one processor. Examples of multiple processing units configured with one processor include a multi-color form in which one processor is configured with a combination of one or more CPUs and software, as represented by computers such as clients and servers, and this processor functions as multiple processing units. A mono-color form includes a form in which a processor is used to realize the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as represented by a system on chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0177] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0178] From the above description, the following additional points can be understood: [Additional note 1] An endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level, A processor is provided. The processor, Acquire an image of the object to be observed; defining a plurality of regions of interest in the image; performing a determination process for determining whether or not the data can be corrected using each of the plurality of regions of interest; An endoscope system that performs correction of the data by using the region of interest that is determined to be capable of correcting the data. [Additional note 2] The endoscopic system described in Appendix 1, wherein the judgment process includes a region reliability calculation process that calculates a region reliability based on the region of interest for each of the multiple regions of interest, and a representative region determination process that determines a representative region by comparing the region reliability for each of the multiple regions of interest with each other. [Additional note 3] The endoscope system according to claim 2, wherein the region reliability is calculated using pixel reliability calculated for each pixel contained in each of the multiple regions of interest. [Additional note 4] An endoscopic system as described in Appendix 2 or 3, wherein the judgment process includes a region reliability comparison process that compares the region reliability in the representative region determined by the representative region determination process with a predetermined region reliability threshold value. [Additional note 5] The processor modifies the pixel reliability in response to a position of each pixel in the image; The endoscope system according to claim 3 or 4, wherein the region reliability is calculated using the corrected pixel reliability. [Additional note 6] The endoscope system according to any one of claims 2 to 5, wherein the region reliability calculation process calculates the region reliability after performing weighting according to an area of ​​the region of interest. [Additional note 7] The endoscope system according to any one of claims 2 to 6, wherein the region reliability calculation process calculates the region reliability after weighting the region of interest according to its distance from the center of the image. [Additional note 8] The processor, 8. The endoscope system according to claim 1, wherein the plurality of regions of interest are set to have substantially the same shape. [Additional note 9] The processor, 9. The endoscope system according to any one of claims 1 to 8, wherein the plurality of regions of interest are set to have substantially the same area. [Additional Note 10] The processor, The endoscope system according to any one of claims 1 to 9, wherein the regions of interest having different shapes and / or different areas are set based on the positions of the regions of interest in the images. [Additional Note 11] The processor, 11. An endoscope system according to any one of claims 1 to 10, wherein the regions of interest are set so that at least two of the multiple regions of interest include the same pixel of the image. [Additional Note 12] The processor, 12. The endoscope system according to claim 1, wherein the regions of interest are set so that each of the regions of interest includes different pixels of the image. [Additional Note 13] The processor, 13. The endoscope system according to any one of claims 1 to 12, wherein, when there is no region of interest for which it is determined that the data can be corrected, control is performed to notify the user. [Additional Note 14] The processor, 14. The endoscope system according to any one of claims 1 to 13, wherein the system performs control to display on the display the region of interest for which it is determined that the data can be corrected. [Additional Note 15] The processor, An endoscope system according to any one of claims 3 to 14, which controls displaying pixels whose pixel reliability is equal to or greater than a preset pixel reliability threshold on a display in a manner that makes them distinguishable from other pixels. [Additional Note 16] The processor, An endoscope system as described in any one of appendix 1 to 15, which, when the data correction is performed, calculates the oxygen saturation of the object of observation by using the corrected data. [Additional Note 17] The processor, 17. An endoscopic system according to any one of claims 1 to 16, wherein, when there is no region of interest for which it is determined that the data can be corrected, the oxygen saturation of the object of observation is calculated by using the data that is not corrected. [Additional Note 18] The processor, 18. An endoscope system according to any one of claims 1 to 17, which performs correction of the data by using image information in the region of interest for which it has been determined that correction of the data is possible. [Additional Note 19] The endoscope system according to claim 18, wherein the image information is a specific dye concentration of a specific dye other than hemoglobin in blood. [Additional Note 20] 20. The endoscope system of claim 19, wherein the specific dye is a yellow dye. [Explanation of symbols]

[0179] 10 Endoscopic images 11, 11a-m Area of ​​interest 12 Endoscopy 12a Insertion part 12b Operation section 12c Mode Switch 12d Zoom operation switch 13 Light source device 14 Processor unit 15 Display 16 User Interface 17 Extended Processor Unit 18 Extended Display 20 Endoscope System 21 Light source section 21a V-LED 21b BS-LED 21c BL-LED 21d G-LED 21e R-LED 22 Light source processor 23 Optical path coupling section 24 Light Guide 30 Illumination optical system 31 Imaging optical system 32 Lighting Lens 35 Objective Lens 36 Image sensor 37 Imaging Processor 40 CDS / AGC circuit 41 A / D Converter 45 DSP 50 Image processing section 51 Image Communication Department 52 Display control unit 53 Central Control Unit 55 white light 56 First illumination light 57 Second illumination light 58 Third Light 55a, 55b, 56a, 56b curve 60a Image acquisition unit 60b Image processing unit 61 Oxygen saturation image generation unit 62 Specific dye concentration calculation section 63 Table correction section 64 Display control section 70 Base image generator 71 Calculation value calculation section 72 Oxygen saturation calculation unit 73, 73a Oxygen saturation calculation table 74 Color tone adjustment section 75 Specific dye concentration calculation table 81 Area setting section 82 Judgment processing unit 83 Correction section 85 Representative areas 86 Display Frame 87 Correction area 91 Reliability calculation unit 92 Representative area determination section 93 Correction area determination unit 94 Region reliability threshold memory 95 Central area 96 Close-range correction data 97 Long distance correction data 98 Area weighting data 99 Weighting data for area distance 100 Correction Pixel Group 101 Contour NP1 White light image NP2 White light equivalent image OP oxygen saturation image BF B Color Filter GF G color filter RF R color filter CV0~CV4 Curved surface AR0~AR4 area MS1, MS2 messages DFX definition line DFY definition line Tc1, Tc2 Threshold for area confidence ST110~ST210 Step

Claims

1. An endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level, A processor is provided. The processor, Acquire an image of the object to be observed; defining a plurality of regions of interest in the image; performing a determination process for determining whether or not the data can be corrected using each of the plurality of regions of interest; An endoscope system that performs correction of the data by using the region of interest that is determined to be capable of correcting the data.

2. The endoscopic system of claim 1, wherein the determination process includes a region reliability calculation process that calculates a region reliability based on the region of interest for each of the multiple regions of interest, and a representative region determination process that determines a representative region by comparing the region reliability for each of the multiple regions of interest with each other.

3. The endoscope system according to claim 2 , wherein the region reliability is calculated using pixel reliability calculated for each pixel included in each of the plurality of regions of interest.

4. The endoscope system according to claim 2 , wherein the determination process includes a region reliability comparison process that compares the region reliability in the representative region determined by the representative region determination process with a preset region reliability threshold value.

5. The processor modifies the pixel reliability in response to a position of each pixel in the image; The endoscope system according to claim 3 , wherein the region reliability is calculated using the modified pixel reliability.

6. The endoscope system according to claim 2 , wherein the region reliability calculation process calculates the region reliability after performing weighting in accordance with an area of ​​the region of interest.

7. The endoscope system according to claim 2 , wherein the region reliability calculation process calculates the region reliability after weighting the region of interest according to a distance from the center of the image.

8. The processor, The endoscope system according to claim 1 , wherein the plurality of regions of interest are set so as to have substantially the same shape.

9. The processor, The endoscope system according to claim 1 , wherein the plurality of regions of interest are set to have substantially the same area.

10. The processor, The endoscope system according to claim 1 , wherein the regions of interest having different shapes and / or different areas are set based on the positions of the regions of interest in the images.

11. The processor, The endoscope system according to claim 1 , wherein the regions of interest are set such that at least two of the plurality of regions of interest include the same pixel of the image.

12. The processor, The endoscope system according to claim 1 , wherein the regions of interest are set so that each of the regions of interest includes pixels of the image that are different from each other.

13. The processor, The endoscope system according to claim 1 , wherein, when there is no region of interest for which it is determined that the data can be corrected, control is performed to notify the user.

14. The processor, The endoscope system according to claim 1 , further comprising control for displaying, on the display, the region of interest for which it is determined that the data can be corrected.

15. The processor, The endoscope system according to claim 3 , wherein a pixel having a pixel reliability equal to or greater than a preset pixel reliability threshold is displayed on a display in a manner that makes it possible to distinguish the pixel from other pixels.

16. The processor, The endoscope system according to claim 1 , wherein when the data is corrected, the oxygen saturation of the object to be observed is calculated by using the corrected data.

17. The processor, The endoscope system according to claim 1 , wherein if there is no region of interest for which it is determined that the data can be corrected, the oxygen saturation of the object of observation is calculated by using the data that has not been corrected.

18. The processor, The endoscope system according to claim 1 , wherein the data is corrected by using image information in the region of interest for which it has been determined that the data can be corrected.

19. The endoscope system according to claim 18, wherein the image information is a specific dye concentration of a specific dye other than hemoglobin in blood.

20. The endoscope system according to claim 19 , wherein the specific dye is a yellow dye.

21. An image processing device for calculating an oxygen saturation level of an observation object, A processor is provided. The processor, Acquire an image of the object to be observed; defining a plurality of regions of interest in the image; performing a determination process for determining whether or not the data can be corrected using each of the plurality of regions of interest; An image processing device that corrects the data by using the region of interest that has been determined to be capable of correcting the data.

22. A method for operating an endoscope system for calculating an oxygen saturation level of an observation object, comprising: acquiring an image of the object to be observed; defining a plurality of regions of interest in the image; performing a determination process for determining whether or not the data can be corrected using each of the plurality of regions of interest; and performing correction of the data by using the region of interest determined to be capable of correcting the data.

23. A program for an endoscope system that calculates an oxygen saturation level of an observation object, On the computer, A function of acquiring an image of the object to be observed; defining a plurality of regions of interest in the image; a function of performing a determination process for determining whether or not the data can be corrected using each of the plurality of regions of interest; and a function of correcting the data by using the region of interest determined to be capable of correcting the data.

Images