Endoscopic system and its operating method

Abstract

Provided are: a processor device that makes it possible to calculate oxygen saturation accurately even when observing a plurality of tissues having different spectroscopic characteristics; an operation method therefor; and an endoscope system.　Referring to an oxygen saturation calculation table TBL, oxygen saturation is calculated on the basis of a computed value. On the basis of a B3 image signal and a G1 image signal, a specific pigment concentration of a specific pigment other than the hemoglobin is calculated. Whether or not a first difference D1 between the specific pigment concentration and a reference concentration is within a specific range is monitored. On the basis of the results of monitoring the specific pigment concentration, a correction notification relating to correction of the oxygen saturation calculation table TBL is made.

Description

Technical Field

[0001] The present invention relates to a processor device for calculating the oxygen saturation of an observation target, an operating method thereof, and an endoscope system.

Background Art

[0002] In recent years, in the medical field using endoscopes, oxygen saturation imaging is known. In oxygen saturation imaging, hemoglobin oxygen saturation is calculated from a small amount of spectroscopic information of visible light. In the calculation of oxygen saturation, when a specific dye such as a yellow dye is present in the observation tissue in addition to hemoglobin, the spectroscopic signal is affected by the light absorption of the specific dye, so there is a problem that the calculated oxygen saturation deviates. To solve this problem, before observing the oxygen saturation, correction imaging for acquiring the spectroscopic characteristics of the observation tissue is performed, and based on the signal obtained by the correction imaging, the algorithm for calculating the oxygen saturation is corrected and used for subsequent oxygen saturation calculations (see Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, in the correction method performed before calculating the oxygen saturation, it is assumed that the same location on the tissue is imaged during the correction imaging and during subsequent observations. However, in actual clinical use, after performing the correction imaging once, it is assumed that other locations will also be observed. In that case, if a location where the spectroscopic characteristics of the tissue are different from those at the first correction is observed, the calculated oxygen saturation value may deviate from the true value.

[0005] The present invention aims to provide a processor device, a method for operating the same, and an endoscope system that can accurately calculate oxygen saturation even when observing multiple tissues with different spectral characteristics. [Means for solving the problem]

[0006] The processor device of the present invention comprises a processor, which acquires a first image signal corresponding to a first wavelength band sensitive to the specific pigment concentration of a specific pigment other than hemoglobin in the blood among the pigments contained in the object of observation, a second image signal corresponding to a second wavelength band sensitive to the oxygen saturation of hemoglobin in the blood, a third image signal corresponding to a third wavelength band sensitive to the blood volume, and a fourth image signal corresponding to a fourth wavelength band with a longer wavelength than the first, second, and third wavelength bands, calculates a calculated value by performing calculation processing based on the second, third, and fourth image signals, calculates the oxygen saturation based on the calculated value by referring to an oxygen saturation calculation table, calculates the specific pigment concentration for calculating the pigment concentration based on the first and third image signals, monitors the specific pigment concentration during the calculation of oxygen saturation, and provides correction notification regarding the correction of the oxygen saturation calculation table based on the monitoring result of the specific pigment concentration.

[0007] Preferably, the processor calculates a first difference or first ratio between the sequentially calculated specific dye concentrations and a predetermined reference concentration for the specific dye concentrations, and issues a correction notification if the first difference or first ratio falls outside a specified range. Preferably, the processor calculates an average specific dye concentration, which is the average value of the specific dye concentrations calculated within a certain time, and calculates a second difference or second ratio between the average specific dye concentration and a predetermined reference concentration for the specific dye concentrations, and issues a correction notification if the second difference or second ratio falls outside a specified range.

[0008] The reference concentration is preferably the concentration of the specific dye at the time the oxygen saturation calculation table was corrected. The reference concentration is preferably predetermined for each patient or site. The specific dye is preferably a yellow dye. The first wavelength band is preferably 450±10nm, the second wavelength band is preferably 470±10nm, the third wavelength band is the green band, and the fourth wavelength band is the red band.

[0009] The present invention comprises a processor device as described above, a light source unit having a first semiconductor light source emitting first blue light, a second semiconductor light source emitting second blue light having a longer wavelength than the first blue light, a third semiconductor light source emitting green light, and a fourth semiconductor light source emitting red light, and a light source processor that controls the lighting and extinguishing of the first semiconductor light source, the second semiconductor light source, the third semiconductor light source, and the fourth semiconductor light source, and an endoscope having an imaging sensor provided with a B color filter having a blue transmission band, a G color filter having a green transmission band, and an R color filter having a red transmission band, wherein the first wavelength band is the wavelength band of green light transmitted through the B color filter, the second wavelength band is the wavelength band of second blue light transmitted through the B color filter, the third wavelength band is the wavelength band of green light transmitted through the G color filter, and the fourth wavelength band is the wavelength band of red light transmitted through the R color filter.

[0010] Preferably, the blue transmission band is 380-560 nm, the green transmission band is 450-630 nm, and the red transmission band is 580-760 nm.

[0011] The method of operating the processor device of the present invention includes the steps of: the processor acquiring a first image signal corresponding to a first wavelength band sensitive to the specific pigment concentration of a specific pigment other than hemoglobin in the blood among the pigments contained in the object of observation; a second image signal corresponding to a second wavelength band sensitive to the oxygen saturation of hemoglobin in the blood; a third image signal corresponding to a third wavelength band sensitive to the blood volume; and a fourth image signal corresponding to a fourth wavelength band with a longer wavelength than the first, second, and third wavelength bands; calculating calculated values ​​by calculation processing based on the second, third, and fourth image signals, referring to an oxygen saturation calculation table, calculating oxygen saturation based on the calculated values, and calculating the specific pigment concentration based on the first and third image signals; monitoring the specific pigment concentration during the calculation of oxygen saturation; and providing correction notification regarding the correction of the oxygen saturation calculation table based on the monitoring results of the specific pigment concentration. [Effects of the Invention]

[0012] According to the present invention, even when observing multiple tissues with different spectral characteristics, oxygen saturation can be calculated with high accuracy. [Brief explanation of the drawing]

[0013] [Figure 1] This is an external view of the endoscope system. [Figure 2] Block diagram showing the functions of the endoscopic system according to the first embodiment. [Figure 3] This graph shows the spectral sensitivity of the imaging sensor. [Figure 4] This is an explanatory diagram showing the emission of illumination light and imaging of the object being observed in normal mode. [Figure 5] This is an explanatory diagram showing the emission of illumination light and imaging of the observed object in oxygen saturation mode. [Figure 6] This is a block diagram showing the functions of the oxygen saturation image processing unit. [Figure 7]It is a graph showing the position of the isopleth of oxygen saturation in a two-dimensional space where the vertical axis is ln(B2 / G1) and the horizontal axis is ln(R1 / G1). [Figure 8] It is an explanatory diagram showing a correction method for an oxygen saturation calculation table. [Figure 9] It is a graph showing the absorption coefficients of oxyhemoglobin and deoxyhemoglobin. [Figure 10] It is an explanatory diagram showing a method for calculating oxygen saturation. [Figure 11] It is a graph showing the absorption coefficient of yellow pigment. [Figure 12] It is an explanatory diagram regarding correction notification. [Figure 13] It is an explanatory diagram showing the display of a message box, which is one of the correction notifications. [Figure 14] It is an explanatory diagram showing a specific pigment concentration calculated at a fixed time TL. [Figure 15] It is a flowchart showing a series of processes in the oxygen saturation mode. [Figure 16] It is a block diagram showing the functions of the endoscope system of the second embodiment. [Figure 17] It is a plan view of a rotary filter.

Embodiments for Carrying Out the Invention

[0014] [First Embodiment] As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The light source device 13 supplies illumination light to the endoscope 12.

[0015] The endoscope 12 is used to illuminate the object to be observed with illumination light and to image the object to acquire an endoscopic image. The endoscope 12 has an insertion section 12a that is inserted into the body of the object to be observed, an operating section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip end of the insertion section 12a. The bending section 12c bends when the operating section 12b is operated. The tip section 12d irradiates illumination light toward the object to be observed and receives reflected light from the object to image the object. The tip section 12d is directed in a desired direction by the bending movement of the bending section 12c. The operating section 12b is provided with a mode switching switch 12f used for switching modes, a still image acquisition instruction switch 12g used for instructing the acquisition of a still image of the object to be observed, and a zoom operation section 12h used for operating the zoom lens 21b.

[0016] The processor unit 14 is electrically connected to the display 15 and the user interface 16. The processor unit 14 receives image signals from the endoscope 12 and performs various processing based on the image signals. The display 15 outputs and displays images or information of the object being observed that have been processed by the processor unit 14. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has the function of accepting input operations such as function settings.

[0017] The endoscope system 10 has two modes: normal mode and oxygen saturation mode. These three modes can be switched by the user operating the mode switching switch 12f. In normal mode, a normal image with natural colors obtained by imaging the observation target using white light as illumination is displayed on the display 15. In oxygen saturation mode, the oxygen saturation of the observation target is calculated based on the image signal from the endoscope 12 by referring to the oxygen saturation calculation table TBL. The calculated oxygen saturation is then visualized using pseudo-color or similar methods and displayed on the display 15 as an oxygen saturation image.

[0018] If the tissue color correction button 12j is operated while the oxygen saturation mode is running, the tissue color correction mode will be executed. In tissue color correction mode, the oxygen saturation calculation table TBL is corrected to calculate the oxygen saturation that matches the tissue color being observed. Then, in oxygen saturation mode, the oxygen saturation is calculated using the corrected oxygen saturation calculation table TBL. In addition, the concentration of a specific dye is monitored during the oxygen saturation calculation, and correction notifications regarding the correction of the oxygen saturation calculation table TBL are provided according to the monitoring results.

[0019] As shown in Figure 2, the light source device 13 comprises a light source unit 20 and a light source processor 21 that controls the light source unit 20. The light source unit 20 has, for example, a plurality of semiconductor light sources, and emits illumination light to illuminate the object of observation by turning each of them on or off, and controlling the amount of light emitted from each semiconductor light source when they are on. In this embodiment, the light source unit 20 has four color LEDs: BS-LED (Blue Short-wavelength Light Emitting Diode) 20a, BL-LED (Blue Long-wavelength Light Emitting Diode) 20b, G-LED (Green Light Emitting Diode) 20c, and R-LED (Red Light Emitting Diode) 20d.

[0020] BS-LED20a (first semiconductor light source) emits first blue light BS with a wavelength of 450 nm ± 10 nm. BL-LED20b (second semiconductor light source) emits second blue light BL with a wavelength of 470 nm ± 10 nm. G-LED20c (third semiconductor light source) emits green light G in the green band. The center wavelength of green light G is preferably 540 nm. R-LED20d (fourth semiconductor light source) emits red light R in the red band. The center wavelength of red light R is preferably 620 nm. Note that the center wavelength and peak wavelength of each LED20a to 20d may be the same or different.

[0021] The light source processor 21 independently controls the on / off state and the amount of light emitted when each LED 20a to 20d is lit by inputting control signals to each LED 20a to 20d independently. The on / off control in the light source processor 21 differs depending on the mode. In normal mode, the BS-LED20a, G-LED20c, and R-LED20d are lit simultaneously, causing the first blue light BS, green light G, and red light R to be emitted simultaneously (see diagram).

[0022] In oxygen saturation mode or tissue color correction mode, three frames with different emission patterns are repeatedly emitted (see diagram). In the first frame, BS-LED20a, G-LED20c, and R-LED20d are lit simultaneously to emit the first blue light BS, green light G, and red light R simultaneously. In the second frame, BL-LED20b, G-LED20c, and R-LED20d are lit simultaneously to emit the second blue light BL, green light G, and red light R simultaneously. In the third frame, G-LED20c is lit to emit the green light G.

[0023] The light emitted from each LED 20a to 20d is incident on the light guide 25 via an optical path coupling section 23, which is composed of mirrors and lenses. The light guide 25 is built into the endoscope 12 and the universal cord (a cord that connects the endoscope 12 to the light source device 13 and the processor device 14). The light guide 25 propagates the light from the optical path coupling section 23 to the tip 12d of the endoscope 12.

[0024] The tip 12d of the endoscope 12 is equipped with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and illumination light propagated by the light guide 25 is irradiated onto the object to be observed through the illumination lens 32. The imaging optical system 31 has an objective lens 42 and an imaging sensor 44. Light from the object to be observed, which is irradiated with illumination light, enters the imaging sensor 44 through the objective lens 42. As a result, an image of the object to be observed is formed on the imaging sensor 44.

[0025] The imaging sensor 44 is a color imaging sensor that captures an object being observed while illuminated by illumination light. Each pixel of the imaging sensor 44 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. For example, it is preferable that the imaging sensor 44 is a Bayer array color imaging sensor in which the ratio of the number of B pixels, G pixels, and R pixels is 1:2:1.

[0026] As shown in Figure 3, the B color filter BF primarily transmits light in the blue band, specifically light with a wavelength range of 380-560 nm (blue transmission band). The peak wavelength at which transmittance is maximum is around 460-470 nm. The G color filter GF primarily transmits light in the green band, specifically light with a wavelength range of 450-630 nm (green transmission band). The R color filter RF primarily transmits light in the red band, specifically light with a wavelength range of 580-760 nm (red transmission band).

[0027] As the image sensor 44, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. Alternatively, instead of the primary color image sensor 44, a complementary color image sensor equipped with complementary color filters for C (cyan), M (magenta), Y (yellow), and G (green) may be used. When using a complementary color image sensor, a four-color image signal of CMYG is output. By converting the four-color image signal of CMYG to a three-color image signal of RGB through complementary-to-primary color conversion, image signals for each RGB color similar to those of the image sensor 44 can be obtained.

[0028] The imaging sensor 44 is driven and controlled by the imaging control unit 45. The control of each mode in the imaging control unit 45 will be described later. The CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 44. The image signal that has passed through the CDS / AGC circuit 46 is converted into a digital image signal by the A / D (Analog / Digital) converter 48. The digital image signal after A / D conversion is input to the processor device 14.

[0029] The processor unit 14 comprises an image signal acquisition unit 50, a DSP (Digital Signal Processor) 52, a noise reduction unit 54, an image processing switching unit 56, a normal image processing unit 58, an oxygen saturation image processing unit 60, and a video signal generation unit 64. Programs for each process are stored in a program memory (not shown) of the processor unit 14. The central control unit (not shown), which is composed of the processor, executes the programs in the program memory, thereby realizing the functions of the image signal acquisition unit 50, the DSP 52, the noise reduction unit 54, the image processing switching unit 56, the normal image processing unit 58, the oxygen saturation image processing unit 60, and the video signal generation unit 64. Accordingly, the functions of the calculation value calculation unit 70, the oxygen saturation calculation unit 71, the image generation unit 72, the specific dye concentration calculation unit 73, the concentration monitoring unit 74, and the notification unit 75, which are included in the oxygen saturation image processing unit 60, are realized.

[0030] The image signal acquisition unit 50 receives the image signal input from the endoscope 12 and transmits the received image signal to the DSP 52. The DSP 52 performs various signal processing on the received image signal, including defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaicing processing, and YC conversion processing. In the defect correction processing, the signals of defective pixels in the imaging sensor 44 are corrected. In the offset processing, the dark current component is removed from the image signal after defect correction processing, and an accurate zero level is set. Gain correction processing adjusts the signal level of each image signal by multiplying the image signal of each color after offset processing by a specific gain. After gain correction processing, the image signal of each color is subjected to linear matrix processing to improve color reproduction.

[0031] Subsequently, gamma conversion processing adjusts the brightness and saturation of each image signal. After linear matrix processing, the image signal is subjected to demosaicing (also called isotropization or simulcasting), and interpolation generates signals for the missing colors of each pixel. Demosaicing ensures that all pixels have signals for each RGB color. DSP52 applies YC conversion processing to each image signal after demosaicing and outputs the luminance signal Y, chrominance signal Cb, and chrominance signal Cr to the noise reduction unit 54.

[0032] The noise reduction unit 54 applies noise reduction processing, such as the moving average method or the median filter method, to the image signal that has undergone demosaicing or other processing by the DSP 56. The noise-reduced image signal is input to the image processing switching unit 56.

[0033] The image processing switching unit 56 switches the destination of the image signal from the noise reduction unit 54 to either the normal image processing unit 58 or the oxygen saturation image processing unit 60, depending on the set mode. Specifically, when set to normal mode, the image signal from the noise reduction unit 54 is input to the normal image processing unit 58. When set to oxygen saturation mode, the image signal from the noise reduction unit 54 is input to the oxygen saturation image processing unit 60.

[0034] The normal image processing unit 58 applies further color conversion processing, such as 3x3 matrix processing, grayscale conversion processing, and 3D LUT (Look Up Table) processing, to the Rc image signal, Gc image signal, and Bc image signal for one frame that has been input. Then, various color enhancement processing is applied to the RGB image data that has undergone color conversion processing. Structural enhancement processing, such as spatial frequency enhancement, is applied to this RGB image data that has undergone color enhancement processing. The RGB image data that has undergone structural enhancement processing is input to the video signal generation unit 64 as a normal image.

[0035] The oxygen saturation image processing unit 60 calculates the oxygen saturation using the image signal obtained in oxygen saturation mode. The method for calculating oxygen saturation will be described later. An oxygen saturation image is generated by imaging the calculated oxygen saturation using pseudo-color or similar methods. This oxygen saturation image is input to the video signal generation unit 64.

[0036] The video signal generation unit 64 converts a normal image from the normal image processing unit 58 or an oxygen saturation image from the oxygen saturation image processing unit 60 into a video signal that enables full-color display on the display 15. The converted video signal is input to the display 15. As a result, the normal image or the oxygen saturation image is displayed on the display 15.

[0037] The following describes the imaging control for each mode in the imaging control unit 45. As shown in Figure 4, in normal mode, the imaging control unit 45 controls the imaging sensor 44 to capture an image of the object being observed while it is illuminated with the first blue light BS, green light G, and red light R, one frame at a time. As a result, a Bc image signal is output from the B pixel of the imaging sensor 44, a Gc image signal is output from the G pixel, and an Rc image signal is output from the R pixel.

[0038] As shown in Figure 5, in oxygen saturation mode, in the first frame, when the first blue light BS, green light G, and red light R illuminate the object being observed, the imaging control unit 45 outputs a B1 image signal from the B pixel, a G1 image signal from the G pixel, and an R1 image signal from the R pixel of the imaging sensor 44. In the second frame, when the second blue light BL, green light G, and red light R illuminate the object being observed, the imaging control unit 45 outputs a B2 image signal from the B pixel, a G2 image signal from the G pixel, and an R2 image signal from the R pixel of the imaging sensor 44. In the third frame, when the green light G illuminates the object being observed, the imaging control unit 45 outputs a B3 image signal from the B pixel, a G3 image signal from the G pixel, and an R3 image signal from the R pixel of the imaging sensor 44.

[0039] In oxygen saturation mode, the B3, B2, G1, and R1 image signals from the three frames of image signals mentioned above are used to calculate oxygen saturation and specific dye concentrations. The B3 image signal (first image signal) contains image information related to the wavelength band (first wavelength band) of light that passed through the B color filter BF of the green light emitted in the third frame. The B2 image signal (second image signal) contains image information related to the wavelength band of light that passed through the B color filter BF, at least within the second blue light BL, of the light emitted in the second frame. The G1 image signal (third image signal) contains image information related to the wavelength band of light that passed through the G color filter GF, at least within the green light G, of the light emitted in the first frame. The R1 image signal (fourth image signal) contains image information related to the wavelength band of light that passed through the R color filter RF, at least within the red light R, of the light emitted in the first frame.

[0040] The method for calculating oxygen saturation is described below. As shown in Figure 6, the oxygen saturation image processing unit 60 includes a calculated value unit 70, an oxygen saturation calculation unit 71, an image generation unit 72, a specific dye density calculation unit 73, a density monitoring unit 74, and a notification unit 75. The signal ratio calculation unit 70 calculates calculated values ​​by calculation processing based on the B2 image signal, G1 image signal, and R1 image signal. Specifically, the calculated value calculation unit 70 calculates the signal ratio B2 / G1 of the B2 image signal and the G1 image signal, and the signal ratio R1 / G1 of the R1 image signal, as calculated values ​​used to calculate oxygen saturation. It is preferable to logarithmically scale (ln) the signal ratio B2 / G1 and the signal ratio R1 / G1. In addition, color difference signals Cr, Cb, or saturation S, hue H, etc., calculated from the B2 image signal, G1 image signal, and R1 image signal may be used as calculated values.

[0041] The oxygen saturation calculation unit 71 refers to the oxygen saturation calculation table TBL and calculates the oxygen saturation based on the calculated value. The oxygen saturation calculation table TBL stores the correlation between the signal ratios B2 / G1 and R1 / G1 and the oxygen saturation. When the correlation is represented in a two-dimensional space formed by the vertical axis Ln(B2 / G1) and the horizontal axis Ln(R1 / G1), as shown in Figure 7, contour lines connecting the parts with the same oxygen saturation are formed almost along the horizontal axis. Furthermore, the contour lines are located further down relative to the vertical axis as the oxygen saturation increases. For example, the contour line 77 for 100% oxygen saturation is located below the contour line 78 for 0% oxygen saturation.

[0042] The oxygen saturation calculation table TBL can be modified to change the correlation between the signal ratios B2 / G1 and R1 / G1 and oxygen saturation, according to the concentration of a specific pigment (such as yellow pigment) that affects the calculation of oxygen saturation. The modification of the oxygen saturation calculation table TBL is performed according to the operation of the tissue color correction button 12j. The change in correlation corresponds to the adjustment of the spacing and position of the contour lines shown in Figure 7. For example, when the specific pigment concentration is concentration CP, the correlation of the oxygen saturation calculation table TBL is changed to the first correlation, as shown in Figure 8(A). Also, when the specific pigment concentration is concentration CQ, which is different from concentration CP, the correlation of the oxygen saturation calculation table TBL is changed to the second correlation, which is different from the first correlation, as shown in Figure 8(B).

[0043] The correlation described above is closely related to the absorbance and light scattering characteristics of oxyhemoglobin (Graph 80) and deoxyhemoglobin (Graph 81) shown in Figure 9. For example, in wavelength bands where the difference in extinction coefficients between oxyhemoglobin and deoxyhemoglobin is large, such as the wavelength band of the second blue light BL at 470 ± 10 nm, the amount of absorbance changes depending on the oxygen saturation of hemoglobin, making it easy to handle oxygen saturation information. Therefore, by using the signal ratio B2 / G1, which includes the B2 image signal corresponding to the light of the second blue light BL with a central wavelength of 470 nm, it is possible to calculate oxygen saturation. However, the signal ratio B2 / G1 is highly dependent not only on oxygen saturation but also on blood volume. Therefore, by using the signal ratio R1 / G1, which changes mainly depending on blood volume, in addition to the signal ratio B2 / G1, it is possible to accurately determine oxygen saturation without being affected by blood volume. Furthermore, the 540±20nm wavelength band of green light included in the G1 image signal is a wavelength band where the amount of absorbed light tends to change depending on the blood volume, because hemoglobin has a relatively high absorption coefficient.

[0044] The oxygen saturation calculation unit 74 refers to the oxygen saturation calculation table TBL and calculates the oxygen saturation corresponding to the signal ratio B2 / G1, R1 / G1 for each pixel. For example, as shown in Figure 10, the signal ratio B2 of a specific pixel * / G1 * ,R1* / G1 * The corresponding oxygen saturation level is "40%". Therefore, the oxygen saturation calculation unit 74 calculates the oxygen saturation level of the specific pixel as "40%".

[0045] Furthermore, the signal ratios B2 / G1 and R1 / G1 rarely become extremely large or extremely small. In other words, the combinations of values ​​for the signal ratios B2 / G1 and R1 / G1 rarely fall below the upper limit contour line 77 (see Figure 7) representing 100% oxygen saturation, or conversely, above the lower limit contour line 78 (see Figure 7) representing 0% oxygen saturation. However, if the distribution falls below the upper limit contour line 77, the oxygen saturation is set to 100%, and if the distribution falls above the lower limit contour line 78, the oxygen saturation calculation unit 71 sets the oxygen saturation to 0%. In addition, if the points corresponding to the signal ratios B2 / G1 and R1 / G1 do not fall between the upper limit contour line 77 and the lower limit contour line 78, it may be indicated that the reliability of the oxygen saturation at that pixel is low, and the oxygen saturation may not be calculated.

[0046] The image generation unit 72 generates an oxygen saturation image, which visualizes oxygen saturation, using the oxygen saturation calculated by the oxygen saturation calculation unit 71. Specifically, the image generation unit 76 acquires the B1 image signal, G1 image signal, and R1 image signal (corresponding to the normal image), and applies a gain to each pixel of these image signals according to the oxygen saturation. Then, it generates RGB image data using the B1 image signal, G1 image signal, and R1 image signal to which the gain has been applied.

[0047] For example, the image generation unit 76 multiplies the B1 image signal, G1 image signal, and R1 image signal by the same gain of "1" for pixels with an oxygen saturation of 60% or more. In contrast, for pixels with an oxygen saturation of less than 60%, the B1 image signal is multiplied by a gain of less than "1", while the G1 image signal and R1 image signal are multiplied by a gain of "1" or more. The RGB image data generated using the B1 image signal, G1 image signal, and R1 image signal after this gain processing is the oxygen saturation image.

[0048] In the oxygen saturation image generated by the image generation unit 72, the high-oxygen region (oxygen saturation of 60-100%) is represented in the same colors as in a normal observation image. On the other hand, the low-oxygen region (oxygen saturation of 0-60%), where the oxygen saturation falls below a specific value, is represented in different colors (pseudocolor) than in a normal observation image.

[0049] In this embodiment, the image generation unit 72 applies a gain to pseudo-colorize only the low-oxygen region, but it is also possible to apply a gain according to the oxygen saturation to the high-oxygen region as well, and pseudo-colorize the entire oxygen saturation image. Furthermore, although the low-oxygen and high-oxygen regions are separated at an oxygen saturation of 60%, this boundary is also arbitrary.

[0050] The specific pigment concentration calculation unit 73 calculates the specific pigment concentration of a specific pigment based on the B3 image signal and the G1 image signal. The specific pigment concentration calculation unit 73 continuously calculates the specific pigment concentration during the oxygen saturation mode. The specific pigment is a pigment other than hemoglobin in the blood that is included in the observed object and affects the calculation of oxygen saturation.

[0051] Examples of specific pigments include yellow pigment. As shown in Figure 11, the absorption coefficient of yellow pigment has a peak that is highest around a wavelength of 450 ± 10 nm. Therefore, the first wavelength band around 450 ± 10 nm is a wavelength band in which the amount of absorbance changes easily depending on the concentration of yellow pigment. Image information related to the first wavelength band is included in the B3 image signal. Therefore, the specific pigment concentration calculation unit 73 calculates the signal ratio B3 / G1 as the pigment concentration of yellow pigment when the specific pigment is yellow pigment. The signal ratio B3 / G1 does not change with oxygen saturation, but the signal value changes with the concentration of yellow pigment and the amount of blood.

[0052] The concentration monitoring unit 74 monitors the concentration of a specific dye while calculating oxygen saturation. The notification unit 75 provides correction notifications regarding the correction of the oxygen saturation calculation table TBL based on the monitoring results of the specific dye concentration. Specifically, it is preferable that the concentration monitoring unit 74 calculates a first difference D1 (=|CX-CS|) or a first ratio P1 (CX / CS) between the specific dye concentration CX, which is sequentially calculated by the specific dye concentration calculation unit 73, and a predetermined reference concentration CS for the specific dye concentration. As shown in Figure 12, the notification unit 75 does not provide correction notifications as long as the first difference D1 or the first ratio P1 is within a specified range. On the other hand, the notification unit 75 provides correction notifications when the first difference D1 or the first ratio P1 falls outside the specified range.

[0053] Correction notifications are preferably given by sound or by display on the display 15. For example, when correction notifications are given by display on the display 15, as shown in Figure 13, it is preferable to display a small message box MB as a correction notification on the display 15 in the peripheral area of ​​the image display area RIN where the oxygen saturation image is displayed, or in the area ROUT outside the image display area, so as not to interfere with the observation of the oxygen saturation image (in Figure 13, the message box MB is displayed in the image display area RIN). The content of the message box MB is preferably something that prompts the user to correct tissue color (for example, "Please operate the tissue color correction button 12j").

[0054] Furthermore, the concentration monitoring unit 74 may calculate a second difference D2 (=|Cave-CS|) or a second ratio P2 (Cave / CS) between the average value Cave of the specific dye concentration calculated by the specific dye concentration calculation unit 73 within a certain period of time and a predetermined reference concentration CS for the specific dye concentration. The notification unit 75 determines whether the second difference D2 or second ratio P2 obtained at regular intervals falls within a specific range. As a result, correction notifications are only issued when the monitored specific dye concentration remains outside the specific range, preventing frequent correction notifications due to fluctuations caused by noise or temporary changes in observation location.

[0055] Furthermore, the notification unit 75 does not perform a correction notification as long as the second difference D2 or second ratio P2 is within the specified range, similar to the case of the first difference or first ratio P1 described above. On the other hand, the notification unit 75 performs a correction notification when the second difference D2 or second ratio P2 falls outside the specified range. It is preferable that the average value Cave of the specific dye concentration is the sum of the N specific dye concentrations C1, C2, ..., CN (N is a natural number) calculated by the specific dye concentration calculation unit 73 within a certain time TL (C1 + C2 + ... + CN) divided by N, as shown in Figure 14.

[0056] Preferably, the reference concentration is the specific dye concentration at the time the oxygen saturation calculation table TBL is corrected. Specifically, it is preferable that the specific dye concentration calculated by the specific concentration calculation unit 73 at the time the tissue color correction button 12j is operated be used as the reference concentration. Furthermore, it is preferable to pre-determine the reference concentration for each patient or for each body part. For example, the pre-treatment conditions (residual state of yellow dye) before endoscopic diagnosis may differ depending on the patient, so in this case, it is preferable to change the reference concentration for each patient. Also, the situation in which yellow dye is present in the observed target may differ when observing the upper digestive tract such as the esophagus or stomach, and when observing the lower digestive tract such as the large intestine, so in this case, it is preferable to change the reference concentration for each body part. The reference concentration is changed by the user operating the user interface 16.

[0057] Next, the sequence of operations in oxygen saturation mode will be explained following the flowchart in Figure 15. The mode switch 12f is operated to set the system to oxygen saturation mode. This illuminates the object of observation for three frames, each with a different emission pattern. The specific dye concentration is calculated from the image signals obtained from these three frames of illumination. Before actually observing the oxygen saturation, the user places the area for which they want to observe the oxygen saturation in their field of view and operates the tissue color correction button 12j. The specific dye concentration calculated at the time the tissue color correction button 12j is operated is used as the reference concentration, and the oxygen saturation calculation table TBL is corrected to match the reference concentration. Then, referring to the corrected oxygen saturation calculation table TBL, the oxygen saturation is calculated based on the image signals obtained from the three frames of illumination. The calculated oxygen saturation is visualized as an oxygen saturation image and displayed on the display 15.

[0058] During the calculation of oxygen saturation, monitoring is performed to determine whether the first difference D1 between the specific dye concentration and the reference concentration falls within a specified range. If the first difference D1 falls within the specified range, no correction notification is issued, and the oxygen saturation calculation continues. On the other hand, if the first difference D1 does not fall within the specified range, a correction notification is issued. If the user believes that correction of the oxygen saturation calculation table TBL is necessary, they can operate the tissue color correction button 12j to recorrect the oxygen saturation calculation table TBL. This entire process is repeated as long as the oxygen saturation mode is maintained.

[0059] [Second Embodiment] In the second embodiment, instead of the four-color LEDs 20a to 20d shown in the first embodiment, a broadband light source such as a xenon lamp and a rotating filter are used to illuminate the object to be observed. Also, instead of the color imaging sensor 44, a monochrome imaging sensor is used to image the object to be observed. Otherwise, it is the same as the first embodiment.

[0060] As shown in Figure 16, in the endoscope system 100 of the second embodiment, the light source device 13 is provided with a broadband light source 102, a rotating filter 104, and a filter switching unit 105 instead of the four-color LEDs 20a to 20d. In addition, the imaging optical system 31 is provided with a monochrome imaging sensor 106 without a color filter instead of a color imaging sensor 44.

[0061] The broadband light source 102 is a xenon lamp, a white LED, etc., and emits white light with a wavelength range from blue to red. The rotating filter 104 includes an inner filter 108 located on the inside and an outer filter 109 located on the outside (see Figure 17). The filter switching unit 105 moves the rotating filter 104 radially. When the mode switching SW12f is set to normal mode, the inner filter 108 of the rotating filter 104 is inserted into the optical path of the white light, and when the mode is set to oxygen saturation mode, the outer filter 109 of the rotating filter 104 is inserted into the optical path of the white light.

[0062] As shown in Figure 17, the inner filter 108 is provided with a B1 filter 108a that transmits the first blue light BS from the white light, a G filter 108b that transmits the green light G from the white light, and an R filter 108c that transmits the red light R from the white light, along the circumferential direction. Therefore, in normal mode, the rotation of the rotating filter 104 alternately irradiates the object of observation with the first blue light BS, green light G, and red light R.

[0063] The outer filter 109 is provided with a B1 filter 109a that transmits the first blue light BS of the white light, a B2 filter 109b that transmits the second blue light BL of the white light, a G filter 109c that transmits the green light G of the white light, and an R filter 109d that transmits the red light R of the white light, all arranged circumferentially. Therefore, in oxygen saturation mode, the rotating filter 104 rotates, causing the first blue light BS, second blue light BL, green light G, and red light R to alternately irradiate the object of observation.

[0064] In the endoscope system 100, during normal mode, the observation target is imaged by the monochrome imaging sensor 106 each time it is illuminated with the first blue light BS, green light G, and red light R. This yields Bc image signals, Gc image signals, and Rc image signals. Based on these three color image signals, a normal image is generated in the same manner as in the first embodiment described above.

[0065] On the other hand, in oxygen saturation mode, the monochromatic imaging sensor 106 captures an image of the object being observed each time it is illuminated with the first blue light BS, second blue light BL, green light G, and red light R. This yields a B3 image signal, a B2 image signal, a G1 image signal, and a R1 image signal. Based on these four color image signals, an oxygen saturation image is generated in the same manner as in the first embodiment.

[0066] In the above embodiment, the hardware structure of the processing unit that performs various processes such as the image signal acquisition unit 50, noise reduction unit 54, image signal switching unit 56, normal image processing unit 58, oxygen saturation image processing unit 60, video signal generation unit 64, calculation value calculation unit 70, oxygen saturation calculation unit 71, image generation unit 72, specific dye concentration calculation unit 73, concentration monitoring unit 74, and notification unit 75 is the following type of processor. The types of processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units), GPUs (Graphical Processing Units), and FPGAs (Field Programmable Gate Arrays), as well as programmable logic devices (PLDs) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits which are processors with circuit configurations specifically designed to perform various processes.

[0067] A single processing unit may be composed of one of these various processors, or it may be composed of a combination of two or more processors of the same or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as clients and servers. Secondly, a configuration where a processor is used that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of a System on a Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0068] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit formed by combining circuit elements such as semiconductor devices. The hardware structure of the memory unit is a storage device such as an HDD (hard disk drive) or SSD (solid state drive). [Explanation of Symbols]

[0069] 10 Endoscopy Systems 12 Endoscopes 12a Insertion section 12b Operation section 12c curved section 12d Tip 12f Mode Selector Switch 12g still image acquisition instruction switch 12h Zoom control section 12j Tissue Color Correction Button 13 Light source device 14 Processor Unit 15 displays 16 User Interface 20 Light source section 20a BS-LED 20b BL-ELD 20c G-LED 20d R-LED 21 Light source processor 23 Optical Path Integration Unit 25 Light Guide 30 Illumination optical system 31 Imaging optical system 32 Illumination Lenses 42 Objective lens 44 Imaging sensors 45 Imaging Control Unit 46 CDS / AGC circuit 48 A / D converters 50 Image signal acquisition unit 52 DSP 54 Noise Reduction Section 56 Specific signal switching section 58 Normal Image Processing Unit 60 Oxygen Saturation Image Processing Unit 64 Video signal generation unit 70 Calculation unit 71 Oxygen saturation calculation unit 72 Image generation unit 73 Specific dye concentration calculation section 74 Concentration Monitoring Unit 75 Hochi Department 77th and 78th contour lines 80, 81 Graphs 100 Endoscopy Systems 102 Broadband light source 104 Rotation Filter 105 Filter switching section 106 Imaging Sensor 108 Internal Filter 108a B1 filter 108b G filter 108c R filter 109 External filter 109a B1 filter 109b B2 filter 109c G filter 109d R filter BF B Color Filter GF G color filter MB Message Box RF R color filter RIN Image Display Area ROUT (Outside the image display area) TBL Table for Calculating Oxygen Saturation

Claims

1. Equipped with a processor, The aforementioned processor, The system acquires a first image signal corresponding to a first wavelength band sensitive to the specific pigment concentration of a specific pigment other than hemoglobin in the blood among the pigments contained in the observed object, a second image signal corresponding to a second wavelength band sensitive to the oxygen saturation of hemoglobin in the blood, a third image signal corresponding to a third wavelength band sensitive to blood volume, and a fourth image signal corresponding to a fourth wavelength band with a longer wavelength than the first, second, and third wavelength bands. A calculated value is obtained by performing calculations based on the second image signal, the third image signal, and the fourth image signal. Refer to the oxygen saturation calculation table and calculate the oxygen saturation based on the calculated value. Based on the first image signal and the third image signal, the specific dye concentration is calculated to determine the specific dye concentration. During the calculation of the oxygen saturation, the concentration of the specific dye is monitored. Based on the monitoring results of the specific dye concentration, a correction notification is made regarding the correction of the oxygen saturation calculation table. A light source unit having a first semiconductor light source that emits first blue light, a second semiconductor light source that emits second blue light with a longer wavelength than the first blue light, a third semiconductor light source that emits green light, and a fourth semiconductor light source that emits red light; and a light source processor that controls the lighting and extinguishing of the first semiconductor light source, the second semiconductor light source, the third semiconductor light source, and the fourth semiconductor light source; The endoscope comprises an imaging sensor equipped with a B color filter having a blue transmission band, a G color filter having a green transmission band, and an R color filter having a red transmission band. The first wavelength band is the wavelength band of the green light that has passed through the B color filter, The second wavelength band is the wavelength band of the second blue light that has passed through the B color filter. The third wavelength band is the wavelength band of the green light that has passed through the G color filter, The fourth wavelength band is the wavelength band of the red light that has passed through the R color filter in the endoscope system.

2. The aforementioned processor, The first difference or first ratio is calculated between the sequentially calculated specific dye concentrations and a predetermined standard concentration for the said specific dye concentrations. The endoscope system according to claim 1, wherein the correction notification is performed when the first difference or first ratio falls outside a specific range.

3. The aforementioned processor, The average value of the specific pigment concentration, which is the average value of the specific pigment concentration calculated within a certain period of time, is calculated. The second difference or second ratio is calculated between the average value of the specified dye concentration and a predetermined standard concentration for the specified dye concentration. The endoscope system according to claim 1, wherein the correction notification is performed when the second difference or second ratio falls outside a specific range.

4. The endoscope system according to claim 1 or 2, wherein the reference concentration is the specific dye concentration at the time when the oxygen saturation calculation table is corrected.

5. The endoscopic system according to claim 1 or 2, wherein the aforementioned standard concentration is predetermined for each patient or for each body part.

6. The endoscope system according to claim 1, wherein the specified dye is a yellow dye.

7. The endoscope system according to claim 1, wherein the first wavelength band is 450 ± 10 nm, the second wavelength band is 470 ± 10 nm, the third wavelength band is the green band, and the fourth wavelength band is the red band.

8. The endoscope system according to claim 1, wherein the blue transmission band is 380 to 560 nm, the green transmission band is 450 to 630 nm, and the red transmission band is 580 to 760 nm.

9. The processor, The steps include acquiring a first image signal corresponding to a first wavelength band sensitive to the specific pigment concentration of a specific pigment other than hemoglobin in the blood among the pigments contained in the object of observation, a second image signal corresponding to a second wavelength band sensitive to the oxygen saturation of hemoglobin in the blood, a third image signal corresponding to a third wavelength band sensitive to blood volume, and a fourth image signal corresponding to a fourth wavelength band with a longer wavelength than the first, second, and third wavelength bands. A step of calculating a calculated value by performing calculation processing based on the second image signal, the third image signal, and the fourth image signal, A step of calculating the oxygen saturation based on the calculated value by referring to the oxygen saturation calculation table, A step of calculating the specific dye concentration based on the first image signal and the third image signal, The steps include monitoring the concentration of the specific dye while calculating the oxygen saturation, The system includes a step of providing correction notification regarding the correction of the oxygen saturation calculation table based on the monitoring results of the specific dye concentration, A light source processor controls the on / off switching of a light source unit having a first semiconductor light source that emits first blue light, a second semiconductor light source that emits second blue light with a longer wavelength than the first blue light, a third semiconductor light source that emits green light, and a fourth semiconductor light source that emits red light. An endoscope having an imaging sensor equipped with a B color filter having a blue transmission band, a G color filter having a green transmission band, and an R color filter having a red transmission band performs imaging. The first wavelength band is the wavelength band of the green light that has passed through the B color filter, The second wavelength band is the wavelength band of the second blue light that has passed through the B color filter. The third wavelength band is the wavelength band of the green light that has passed through the G color filter, The fourth wavelength band is the wavelength band of the red light that has passed through the R color filter. A method for operating an endoscope system.

Images