Endoscope system and operation method therefor

Abstract

The purpose of the present invention is to provide an endoscope system and an operation method therefor, with which it is possible to calculate biological parameters, including oxygen saturation, that are more accurate as compared with the computation of oxygen saturation from a small number of spectral signals.　On the basis of a first spectral signal and a reference spectral signal determined on the basis of the first spectral signal, a biological parameter calculation unit (61) calculates biological parameters that include oxygen saturation. An extended display (18) displays an oxygen saturation image based on the oxygen saturation.

Description

Endoscope system and method of operation thereof

[0001] The present invention relates to an endoscope system for calculating biological parameters such as oxygen saturation and an operating method thereof.

[0002] In recent years, oxygen saturation imaging has become known in the medical field using endoscopes. Oxygen saturation imaging is a technique for calculating hemoglobin oxygen saturation from a small amount of spectral information of visible light. Conventionally, as shown in Patent Document 1, an oxygen saturation image is created and displayed from spectral signals acquired by switching the illumination light for two or three frames of a video.

[0003] Patent No. 6039639

[0004] There was a need to realize more accurate and robust calculations by taking into account the diversity of spectroscopic characteristics of biological tissues, thereby making it possible to calculate biological parameters including oxygen saturation more accurately than when oxygen saturation is calculated from a small number of spectroscopic signals, as in Patent Document 1.

[0005] The present invention aims to provide an endoscope system and an operating method thereof that can calculate biological parameters including oxygen saturation more accurately than when oxygen saturation is calculated from a small number of spectroscopic signals.

[0006] The endoscopic system of the present invention includes a light source device that emits illumination light, an endoscope that captures an image of an object to be observed, and one or more processors that acquire a first spectral signal captured by the endoscope while the illumination light is being irradiated onto the object to be observed. The one or more processors calculate biological parameters including oxygen saturation based on the first spectral signal and a reference spectral signal determined in accordance with the first spectral signal, and perform control to display an oxygen saturation image based on the oxygen saturation.

[0007] The one or more processors preferably calculate the biological parameters using N or more reference spectroscopic signals at different wavelengths determined by N-1 (N is an integer greater than 3) biological parameters including oxygen saturation, and N or more first spectroscopic signals corresponding to the wavelengths of the N or more reference signals. The biological parameters preferably include at least one of hemoglobin concentration, bilirubin concentration, and a scattering wavelength-dependent parameter.

[0008] Preferably, the reference spectroscopic signal is set from the emission spectrum of the illumination light, the spectral reflectance of the living body determined from a spectroscopic model, and the spectral sensitivity of the imaging sensor provided in the endoscope, the spectroscopic model has a biological parameter as an argument, and the one or more processors calculate the biological parameter so that a difference calculation value based on the difference between the reference spectroscopic signal and the first spectroscopic signal falls within a specific range.

[0009] It is preferable that M sets of reference spectroscopic signals are set for each spectroscopic model based on the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectroscopic model, and the spectral sensitivity of the imaging sensor provided in the endoscope, and that one or more processors select, from the M sets of reference spectroscopic signals, a specific reference spectroscopic signal whose difference calculation value based on the difference from the first spectroscopic signal is the smallest, and calculate, as the biological parameter, a specific biological parameter determined by the spectroscopic model corresponding to the specific reference spectroscopic signal.

[0010] It is preferable that M sets of reference spectroscopic signals are set for each phantom based on the emission spectrum of the illumination light, the spectral reflectance of the living body determined by spectroscopic measurement of a phantom simulating a spectroscopic model, and the spectral sensitivity of the imaging sensor provided in the endoscope, and that one or more processors select, from the M sets of reference spectroscopic signals, a specific reference spectroscopic signal whose difference calculation value based on the difference from the first spectroscopic signal is the smallest, and calculate, as the biological parameter, a specific biological parameter determined in the phantom corresponding to the specific reference spectroscopic signal.

[0011] It is preferable that M sets of reference spectroscopic signals are obtained by illuminating a phantom simulating a spectroscopic model with illumination light and imaging each phantom, and that a specific reference spectroscopic signal having the smallest difference calculation value based on the difference from the first spectroscopic signal is selected from the M sets of reference spectroscopic signals, and that a specific biological parameter defined for the phantom corresponding to the specific reference spectroscopic signal is calculated as the biological parameter.

[0012] Preferably, the reference spectroscopic signal is normalized by a reference emission value corresponding to the emission spectrum of the illumination light, and the first spectroscopic signal is normalized by an average value of the plurality of first spectroscopic signals. When the emission spectrum of the illumination light is changed, the one or more processors preferably reset the reference spectroscopic signal. When the endoscope has an image sensor and the image sensor is changed to one with a different spectral sensitivity, the one or more processors preferably reset the reference spectroscopic signal.

[0013] It is preferable that the light source device emits, as illumination light, a first illumination light and a second illumination light having a broader bandwidth than the first illumination light, and that the one or more processors acquire, as a first spectral signal, a signal captured while the first illumination light is being irradiated, and acquire, as a second spectral signal, a signal captured while the second illumination light is being irradiated, and display an image generated from the second spectral signal.

[0014] Preferably, the one or more processors align the first spectroscopic signals based on the amount of movement between the second spectroscopic signals. Preferably, the biological parameter includes a hemoglobin concentration, and the display controls to display a hemoglobin concentration image based on the hemoglobin concentration on the display.

[0015] Preferably, the light source device emits, as illumination light, either first illumination light or second illumination light having a broader bandwidth than the first illumination light, the one or more processors control the illumination to alternate between the first illumination light and the second illumination light, and the first spectral signal is obtained by capturing an image of the object illuminated with the first illumination light using an endoscope. Preferably, the first illumination light is emitted in sequence, with center wavelengths of 450 nm, 470 nm, 540 nm, 620 nm, 690 nm, and 850 nm, and the second illumination light is emitted between each emission of the first illumination light.

[0016] The imaging sensor provided in the endoscope is preferably a spectroscopic imaging sensor that splits the illumination light into first illumination lights, and the spectroscopic imaging sensor obtains the first spectral signal by capturing an image of an observation target illuminated with the illumination light. The spectroscopic imaging sensor preferably splits the illumination light into first illumination lights having center wavelengths of 450 nm, 470 nm, 500 nm, 540 nm, 620 nm, 690 nm, and 850 nm.

[0017] In a method for operating an endoscopic system including a light source device that emits illumination light, an endoscope that captures an image of an object to be observed, and one or more processors that acquire a first spectral signal captured by the endoscope while the illumination light is being irradiated onto the object to be observed, the one or more processors calculate biological parameters including oxygen saturation based on the first spectral signal and a reference spectral signal determined according to the first spectral signal, and perform control to display an oxygen saturation image based on the oxygen saturation.

[0018] According to the present invention, it is possible to calculate biological parameters including oxygen saturation more accurately than when oxygen saturation is calculated from a small number of spectroscopic signals.

[0019] 1 is a schematic diagram of an endoscope system for the digestive tract. FIG. 1 is an explanatory diagram showing the display aspects on the display and the extended display in normal mode. FIG. 2 is an explanatory diagram showing the display aspects on the display and the extended display in oxygen saturation mode. (A) is an image diagram of the extended display displaying an internal digestive tract oxygen saturation image, and (B) is an image diagram of the extended display displaying a serosal side oxygen saturation image. FIG. 2 is a block diagram showing the functions of the endoscope system. FIG. 3 is a graph showing the emission spectra of monochromatic light b, sb, g, a, r, and ir. FIG. 4 is an explanatory diagram showing an emission pattern in oxygen saturation mode. FIG. 5 is a graph showing the spectral sensitivity of an imaging sensor. FIG. 6 is a table showing image signals obtained in normal mode. FIG. 7 is a table showing image signals obtained in oxygen saturation mode. FIG. 8 is a graph showing the relationship between the center wavelengths of monochromatic light b, sb, g, a, r, and ir and the reflection spectrum of hemoglobin. FIG. 9 is a block diagram showing the functions of an extension processor device of the first embodiment. FIG. 10 is an explanatory diagram showing alignment. FIG. 11 is a graph showing the relationship between reflectance R and ratio μa / μs'. FIG. 12 is a graph showing an increase in the light intensity of monochromatic light sb. FIG. 13 is an explanatory diagram showing replacement of an endoscope with respect to a light source device. 1 is a flowchart showing a series of steps in the oxygen saturation mode.

[0033] FIG. 1 is an explanatory diagram showing the oxygen saturation, hemoglobin concentration, bilirubin concentration, and scattering wavelength-dependent parameters of the spectroscopic model required as a minimum for calculating biological parameters.

[0034] FIG. 1 is an explanatory diagram showing the oxygen saturation, hemoglobin concentration, bilirubin concentration, and scattering wavelength-dependent parameters of the spectroscopic model required for improving the accuracy of calculating biological parameters.

[0035] FIG. 1 is an explanatory diagram showing the oxygen saturation, hemoglobin concentration, bilirubin concentration, and scattering wavelength-dependent parameters of the spectroscopic model required for further improving the accuracy of calculating biological parameters.

[0036] FIG. 1 is a graph showing the spectral reflectance determined by the spectroscopic model.

[0037] FIG. 1 is a block diagram showing the functions of the extension processor device of the second embodiment.

[0038] FIG. 2 is a table showing reference spectral signals for each spectroscopic model.

[0039] FIG. 2 is a graph showing the spectral reflectance determined for a phantom.

[0039] FIG. 3 is a block diagram showing the functions of the extension processor device of the third embodiment.

[0039] FIG. 4 is a table showing reference spectral signals for each phantom.

[0039] FIG. 4 is a block diagram showing the functions of the extension processor device of the fourth embodiment.

[0039] FIG. 5 is an explanatory diagram showing a method of acquiring a reference spectral signal in the fourth embodiment.

[0039] FIG. 6 is a schematic diagram of an endoscope system for a laparoscope.

[0039] FIG. 7 is an explanatory diagram showing an imaging unit having three monochrome image sensors.1 is a diagram illustrating an imaging unit having one color image sensor; FIG. 2 is a graph illustrating an emission spectrum of broadband illumination light; FIG. 3 is a diagram illustrating a pixel arrangement in a spectroscopic image sensor; and FIG. 4 is a diagram illustrating another emission pattern in an oxygen saturation mode.

[0020] 1 , an endoscopic system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a processor-side user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically or electrically connected to the light source device 13 and electrically connected to the processor device 14. The extended processor device 17 is electrically connected to the light source device 13 and the processor device 14. Note that the term "display" in the claims includes the extended display 18 in addition to the display 15.

[0021] The endoscope 12 has an insertion section 12a, an operating section 12b, a bending section 12c, and a tip section 12d. The insertion section 12a is inserted into the body of the subject. The operating section 12b is provided at the base end of the insertion section 12a. The bending section 12c and the tip section 12d are provided on the tip side of the insertion section 12a. The bending section 12c is bent by operating the angle knob 12e of the operating section 12b. The tip section 12d is directed in a desired direction by the bending of the bending section 12c. A forceps channel (not shown) is provided from the insertion section 12a to the tip section 12d for inserting a treatment tool or the like. The treatment tool is inserted into the forceps channel through the forceps port 12j.

[0022] The endoscope 12 is provided with an optical system for forming an image of a subject and an optical system for irradiating the subject with illumination light. The operation unit 12b is provided with an angle knob 12e, a mode selector switch 12f, a still image acquisition instruction switch 12h, and a zoom operation unit 12i. The mode selector switch 12f is used to switch observation modes. The still image acquisition instruction switch 12h is used to instruct acquisition of a still image of the subject. The zoom operation unit 12i is used to enlarge or reduce the observation target. The mode selector switch 12f and the still image acquisition instruction switch 12h are included in a scope-side user interface 19 that is used to perform various operations on the processor device 14.

[0023] The light source device 13 generates illumination light. The processor device 14 performs system control of the endoscope system 10 and generates endoscopic images by performing image processing on image signals transmitted from the endoscope 12. The display 15 displays medical images transmitted from the processor device 14. The processor-side user interface 16 has a keyboard, mouse, microphone, tablet, foot switch, touch pen, etc., and accepts input operations such as function settings.

[0024] The endoscope system 10 has a normal mode and an oxygen saturation mode, and the user can switch between these modes by operating the mode selector switch 12f. As shown in Fig. 2, in the normal mode, a white light image with natural coloring obtained by capturing an image of an object to be observed using white light as illumination light is displayed on the display 15, while nothing is displayed on the extended display 18.

[0025] As shown in Figure 3, in the oxygen saturation mode, the oxygen saturation of the subject is calculated, and an oxygen saturation image that visualizes the calculated oxygen saturation is displayed on the extended display 18. Also, in the oxygen saturation mode, a white light equivalent image, which has fewer short wavelength components than a white light image, is displayed on the display 15. Note that the "normal light image" in the claims includes a white light equivalent image. The oxygen saturation image may also be displayed on the display 15.

[0026] The endoscope system 10 is a flexible endoscope type for the digestive tract, such as the stomach and large intestine, and in the oxygen saturation mode, as shown in Fig. 4A, displays an internal digestive tract oxygen saturation image, which visualizes the oxygen saturation state inside the digestive tract, on the extended display 18. In addition, if the endoscope system described below is a rigid endoscope type for the abdominal cavity, such as the serosa, in the oxygen saturation mode, a serosal-side oxygen saturation image, which visualizes the oxygen saturation state on the serosal side, is displayed on the extended display 18 as shown in Fig. 4B.

[0027] In the oxygen saturation mode, it is possible to accurately calculate oxygen saturation in the following cases: - When observing a predetermined target region (for example, the esophagus, stomach, or large intestine) - In a non-corporeal environment with ambient lighting - When there is no residue, residual fluid, mucus, blood, or fat remaining on the mucosa or serosa - When dye is not sprayed onto the mucosa - When the endoscope 12 is more than 7 mm away from the observation region - When the endoscope is observed at an appropriate distance without being too far away from the observation region - In an area sufficiently illuminated by illumination light - When there is little specular reflection light from the observation region - In an area within two-thirds of the oxygen saturation image - When there is little movement of the endoscope, or little movement of the patient, such as pulsation or breathing - When blood vessels deep in the gastrointestinal mucosa are not observed

[0028] As shown in FIG. 5 , the light source device 13 includes a light source unit 20 and a light source processor 21 that controls the light source unit 20. The light source unit 20 emits at least one of polychromatic light and monochromatic light as illumination light. The light source unit 20, for example, has multiple semiconductor light sources, which are turned on or off, and when turned on, the light emission amount of each semiconductor light source is controlled to emit illumination light that illuminates the observation target. The light source unit 20 includes a v-LED 20g (Violet-Light Emitting Diode), a b-LED 20a (Blue-Light Emitting Diode), a sb-LED 20b (Sky Blue-Light Emitting Diode), a g-LED 20c (Green-Light Emitting Diode), an a-LED 20d (Amber-Light Emitting Diode), an r-LED 20e (Red-Light Emitting Diode), and an ir-LED 20f (Infra Red-Light Emitting Diode).

[0029] Light emitted from each of the LEDs 20a to 20f is incident on a light guide 25 via an optical path coupling unit 23 composed of a mirror, a lens, etc. The light guide 25 is built into the endoscope 12 and a universal cord (a cord that connects the endoscope 12 with the light source device 13 and the processor device 14). The light guide 25 propagates the light from the optical path coupling unit 23 to the tip 12d of the endoscope 12.

[0030] The distal end 12d of the endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and illumination light propagated by the light guide 25 is irradiated onto the observation object via the illumination lens 32. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Light from the observation object irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, an image of the observation object is formed on the imaging sensor 36.

[0031] The imaging sensor 36 captures an image of the object to be observed. The imaging sensor 36 is preferably a color imaging sensor. Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. The spectral transmittances of the B color filter, G color filter, and R color filter that determine the spectral sensitivity of the imaging sensor 36 will be described later. Note that the imaging sensor 36 is preferably a color imaging sensor with a Bayer array in which the ratio of the number of B pixels to G pixels to R pixels is 1:2:1, for example.

[0032] A charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor can be used as the image sensor 36. Instead of the primary-color image sensor 36, a complementary-color image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. When a complementary-color image sensor is used, image signals of four colors (CMYG) are output. Therefore, by converting the four CMYG image signals into three RGB image signals by complementary-color-primary-color conversion, it is possible to obtain image signals of each of the RGB colors similar to those of the image sensor 36.

[0033] The imaging sensor 36 is driven and controlled by an imaging processor 37. Control of each mode by the imaging processor 37 will be described later. A CDS / AGC circuit 40 (Correlated Double Sampling / Automatic Gain Control) performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 36. The image signal that passes through the CDS / AGC circuit 40 is converted into a digital image signal by an A / D converter 41 (Analog / Digital). The digital image signal after A / D conversion is input to the processor device 14.

[0034] The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. The processor device 14 has programs related to various processes stored in a program memory (not shown). The central control unit 53, which is made up of a processor, executes the programs in the program memory to realize the functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53. It is preferable to use one or more processors.

[0035] The DSP 45 performs various signal processing on the image signals received from the endoscope 12, such as defect correction, offset processing, gain correction, linear matrix processing, gamma conversion, demosaic processing, white balance processing, YC conversion, and noise reduction. In the defect correction processing, signals from defective pixels of the imaging sensor 36 are corrected. In the offset processing, dark current components are removed from the image signals that have been subjected to defect correction processing, and an accurate zero level is set. In the gain correction processing, the signal level of each image signal is adjusted by multiplying the image signals of each color after offset processing by a specific gain. The image signals of each color after gain correction are subjected to linear matrix processing to improve color reproducibility.

[0036] Then, the brightness and saturation of each image signal are adjusted by gamma conversion. The image signals after linear matrix processing are subjected to demosaic processing (also known as isotropy processing or synchronization processing), and signals of missing colors for each pixel are generated by interpolation. The demosaic processing ensures that all pixels have signals for each of the RGB colors. The DSP 45 performs YC conversion processing on each image signal after demosaic processing, and outputs a luminance signal Y and color difference signals Cb and Cr to the DSP 45. The DSP 45 then performs noise reduction processing, such as using a moving average method or a median filter method, on the image signals that have been subjected to demosaic processing and other processing.

[0037] The image processing unit 50 performs various image processing on the image signal from the DSP 45. Image processing includes color conversion processing such as 3x3 matrix processing, tone conversion processing, and 3D LUT (Look Up Table) processing, color enhancement processing, and structure enhancement processing such as spatial frequency enhancement. The image processing unit 50 performs image processing according to the mode. In the normal mode, the image processing unit 50 performs image processing for the normal mode to generate a white light image. In the oxygen saturation mode, the image processing unit 50 generates a white light equivalent image. The image processing unit 50 also transmits the image signal from the DSP 45 to the extension processor device 17 via the image communication unit 51.

[0038] The display control unit 52 performs display control to display the white light image or image information such as the oxygen saturation image from the image processing unit 50, and other information, on the display 15. In accordance with the display control, the white light image or the white light equivalent image is displayed on the display 15.

[0039] The extended processor unit 17 receives image signals from the processor unit 14 and performs various image processing. In the oxygen saturation mode, the extended processor unit 17 calculates the oxygen saturation level and generates an oxygen saturation image by visualizing the calculated oxygen saturation level. Details of the extended processor unit 17 will be described later. The generated oxygen saturation image is displayed on the extended display 18.

[0040] The on / off control in each mode will be described in detail below. As shown in FIG. 6, the v-LED 20g emits monochromatic light v with a center wavelength of 410 nm. The b-LED 20a emits monochromatic light b with a center wavelength of 450 nm. The sb-LED 20b emits monochromatic light sb with a center wavelength of 470 nm. The g-LED 20c emits monochromatic light g with a center wavelength of 540 nm. The a-LED 20d emits monochromatic light a with a center wavelength of 620 nm. The r-LED 20e emits monochromatic light r with a center wavelength of 690 nm. The ir-LED 20f emits monochromatic light ir with a center wavelength of 850 nm. The center wavelength of each monochromatic light may be the same as or different from the peak wavelength.

[0041] In the normal mode, the v-LED 20g, b-LED 20a, g-LED 20c, and a-LED 20d are simultaneously turned on to emit polychromatic light including monochromatic light v with a central wavelength of 410 nm, monochromatic light b with a central wavelength of 450 nm, monochromatic light g with a central wavelength of 540 nm, and monochromatic light a with a central wavelength of 620 nm. In the oxygen saturation mode, monochromatic light with central wavelengths of 450 nm, 470 nm, 540 nm, 620 nm, 690 nm, and 850 nm is emitted in sequence as the first illumination light, and polychromatic light having a broader bandwidth than the first illumination light is emitted between the emissions of each monochromatic light. Specifically, as shown in FIG. 7 , when light emission is repeatedly performed for 10 frames from frame 1 to frame 10, monochromatic light sb, g, a, r, and ir having central wavelengths of 470 nm, 540 nm, 620 nm, 690 nm, and 850 nm are emitted in even-numbered frames 2, 4, 6, 8, and 10, respectively, and polychromatic light including monochromatic light b having a central wavelength of 450 nm, monochromatic light g having a central wavelength of 540 nm, and monochromatic light r having a central wavelength of 690 nm is emitted in odd-numbered frames 1, 3, 5, 7, and 9 between the even-numbered frames.

[0042] The spectral transmittance of the B, G, and R color filters will now be described in detail. As shown in FIG. 8 , the B color filter BF provided in the B pixel of the image sensor 36 transmits mainly light in the blue band, specifically light with a wavelength band of 380 to 560 nm (blue transmission band). The peak wavelength at which transmittance is maximum is around 460 to 470 nm. The G color filter GF provided in the G pixel of the image sensor 36 transmits mainly light in the green band, specifically light with a wavelength band of 450 to 630 nm (green transmission band). The R color filter RF provided in the R pixel of the image sensor 36 transmits mainly light in the red band, specifically light with a wavelength band of 580 to 900 nm (red transmission band).

[0043] Control of each mode by the imaging processor 37 will be described in detail below. As shown in Fig. 9, in normal mode, the imaging processor 37 controls the imaging sensor 36 to capture an image of an object being observed frame by frame while it is illuminated with polychromatic light including monochromatic light b, monochromatic light g, and monochromatic light r. As a result, Bc image signals are output from the B pixels of the imaging sensor 36, Gc image signals are output from the G pixels, and Rc image signals are output from the R pixels. A white light image is generated based on the Bc image signals, Gc image signals, and Rc image signals.

[0044] 10 , in the oxygen saturation mode, in odd-numbered frames 1, 3, 5, 7, and 9, the imaging processor 37 controls the imaging of an object being observed while being illuminated with polychromatic light including monochromatic light b, monochromatic light g, and monochromatic light r. As a result, in frames 1, 3, 5, 7, and 9, Bc image signals are output from the B pixels of the imaging sensor 36, Gc image signals are output from the G pixels, and Rc image signals are output from the R pixels. A white light equivalent image is generated based on the Bc image signals, Gc image signals, and Rc image signals.

[0045] Meanwhile, in frame 2 of the even-numbered frames, the imaging processor 37 controls the imaging of the object illuminated with monochromatic light sb. As a result, a B1 image signal is output from the B pixel of the imaging sensor 36, a G1 image signal is output from the G pixel, and an R1 image signal is output from the R pixel. Similarly, in frames 4, 6, 8, and 10 of the even-numbered frames, the imaging processor 37 controls the imaging of the object illuminated with monochromatic light g, a, r, and ir. As a result, B2, B3, B4, and B5 image signals are output from the B pixel of the imaging sensor 36, G2, G3, G4, and G5 image signals are output from the G pixel, and R2, R3, R4, and R5 image signals are output from the R pixel.

[0046] Of the image signals obtained in frames 2, 4, 6, 8, and 10 of the even-numbered frames, the B1, B2, G2, R3, R4, B5, G5, and R5 image signals are used to calculate biological parameters such as oxygen saturation. The B1 image signal contains image information related to monochromatic light sb among the light transmitted through the B color filter BF. The B2 image signal contains image information related to monochromatic light g among the light transmitted through the B color filter BF. The G2 image signal contains image information related to monochromatic light g among the light transmitted through the G color filter GF. The R3 image signal contains image information related to monochromatic light a among the light transmitted through the R color filter RF. The R4 image signal contains image information related to monochromatic light r among the light transmitted through the R color filter. The B5 image signal contains image information related to monochromatic light ir among the light transmitted through the B color filter. The G5 image signal contains image information related to the monochromatic light ir among the light transmitted through the G color filter, and the R5 image signal contains image information related to the monochromatic light ir among the light transmitted through the R color filter.

[0047] 11, the image information related to the monochromatic light sb includes image information of the wavelength band sb whose reflection spectrum changes with changes in the oxygen saturation of blood hemoglobin. Similarly, the image information related to the monochromatic light g, a, r, and ir includes image information of the wavelength bands g, a, r, and ir whose reflection spectrum changes with changes in the oxygen saturation of blood hemoglobin. Note that curve 55a represents the reflection spectrum of reduced hemoglobin, and curve 55b represents the reflection spectrum of oxygenated hemoglobin.

[0048] The extension processor device 17 will be described in detail. As shown in Fig. 12, the extension processor device 17 includes a measured first spectral signal acquisition unit 60, a biological parameter calculation unit 61, a reference spectral signal setting unit 62, a display control unit 63, and an alignment unit 64. The extension processor device 17 has programs related to various processes stored in a program memory (not shown). A central control unit (not shown) configured by a processor executes the programs stored in the program memory to realize the functions of the measured first spectral signal acquisition unit 60, the biological parameter calculation unit 61, the reference spectral signal setting unit 62, the display control unit 63, and the alignment unit 64. It is preferable to use one or more processors.

[0049] The measured first spectral signal acquisition unit 60 acquires measured first spectral signals (first spectral signals) captured by the imaging sensor 36 while the object to be observed is irradiated with the first illumination light. The processor performs processing based on the measured first spectral signals. The measured first spectral signals include the B1 image signal, B2 image signal, G2 image signal, R3 image signal, R4 image signal, B5 image signal, G5 image signal, and R5 image signal obtained in the oxygen saturation mode. In this embodiment, the B1 image signal, B2 image signal, G2 image signal, R3 image signal, and R4 image signal are used as the measured first spectral signals, and the B5 image signal, G5 image signal, and R5 image signal are used as the measured first spectral signal by combining the image signals of these three colors. Hereinafter, the B1 image signal, B2 image signal, G2 image signal, R3 image signal, R4 image signal, and MR5 image signal of the measured first spectral signals will be referred to as the B1 image signal. ｉｍｇ Image signal, B2 ｉｍｇ Image signal, G2 ｉｍｇ Image signal, R3 ｉｍｇ Image signal, R4 ｉｍｇ Image signal and MR5 ｉｍｇ It is expressed as an image signal.

[0050] In this embodiment, the measured first spectral signals are preferably normalized by the average value of the measured first spectral signals in order to eliminate instability in the illuminance of the illumination light. ｉｍｇ Image signal, B2ｉｍｇ Image signal, G2 ｉｍｇ Image signal, R3 ｉｍｇ Image signal, R4 ｉｍｇ Image signal and MR5 ｉｍｇ When the average value of the image signal is Ave, the logarithmic B1 ｉｍｇ By subtracting the logarithmic average value Ave from the image signal, B1 ｉｍｇ The image signal is normalized (Log(B1 ｉｍｇ Image signal) - Log (Ave)) B2 ｉｍｇ Image signal, G2 ｉｍｇ Image signal, R3 ｉｍｇ Image signal, R4 ｉｍｇ Image signal and MR5 ｉｍｇ For the image signal, B1 ｉｍｇ Normalization is performed in the same manner as for image signals.

[0051] The biological parameter calculation unit 61 calculates biological parameters including oxygen saturation based on the measured first spectral signal and a reference spectral signal predetermined according to the measured first spectral signal. Specifically, in the first embodiment, the biological parameter calculation unit 61 calculates biological parameters including oxygen saturation based on the reference spectral signal set from the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectral model, and the spectral sensitivity of the imaging sensor 36, and the measured first spectral signal. The display control unit 63 displays an oxygen saturation image based on the oxygen saturation image on the extended display 18. Note that the reference spectral signal is preferably predetermined so that the wavelength range or number of wavelengths of the reference spectral signal matches that of the measured first spectral signal.

[0052] The biological parameter calculation unit 61 calculates the biological parameters so that a difference calculation value based on the difference between the reference spectral signal and the measured first spectral signal falls within a specific range (fitting process). In this embodiment, the reference spectral signal uses an error calculation value based on the error between the reference spectral signal and the measured first spectral signal as the difference calculation value based on the difference between the reference spectral signal and the measured first spectral signal. Since the reference spectral signal is determined based on a spectroscopic model that uses biological parameters as arguments, it is defined as a function of the biological parameters. In this embodiment, four biological parameters are used: oxygen saturation S, hemoglobin concentration Hb, bilirubin concentration Bb, and scattering wavelength-dependent parameter b.

[0053] The reference spectroscopic signal is determined for each measured first spectroscopic signal. ｉｍｇ Image signal, B2 ｉｍｇ Image signal, G2 ｉｍｇ Image signal, R3 ｉｍｇ Image signal, R4 ｉｍｇ Image signal and MR5 ｉｍｇ As a reference spectroscopic signal corresponding to the image signal, B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ The image signal is defined.

[0054] The calculated error value Diff is preferably calculated by the square error calculation formula based on (Formula 1): Diff=W Ｂ１ (B1 ｓｔｄ (S, Hb, Bb, b) - B1 ｉｍｇ ) ２ +W Ｂ２ (B2 ｓｔｄ (S, Hb, Bb, b) - B2 ｉｍｇ ) ２ +W Ｇ１ (G2 ｓｔｄ (S, Hb, Bb, b) - G2 ｉｍｇ ) ２ +W Ｒ３ (R3 ｓｔｄ (S, Hb, Bb, b)-R3 ｉｍｇ ) ２ +W Ｒ４ (R4 ｃａｌ (S, Hb, Bb, b) - R4 ｉｍｇ ) ２ +W ＭＲ５ (MR5 ｓｔｄ (S, Hb, Bb, b)-MR5 ｉｍｇ ) ２ ... (Equation 1) Note that B1 ｓｔｄ (S, Hb, Bb, b), B2 ｓｔｄ (S, Hb, Bb, b), G2 ｓｔｄ (S, Hb, Bb, b), R3 ｓｔｄ (S, Hb, Bb, b), R4 ｓｔｄ(S, Hb, Bb, b) and MR5 ｓｔｄ (S, Hb, Bb, b) are B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ It represents an image signal, and is a function of the oxygen saturation S, the hemoglobin concentration Hb, the bilirubin concentration Bb, and the scattering wavelength-dependent parameter b. ｉｍｇ , B2 ｉｍｇ , G2 ｉｍｇ , R3 ｉｍｇ , R4 ｉｍｇ , and MR5 ｉｍｇ are B1 ｉｍｇ Image signal, B2 ｉｍｇ Image signal, G2 ｉｍｇ Image signal, R3 ｉｍｇ Image signal, R4 ｉｍｇ Image signal and MR5 ｉｍｇ It represents an image signal. Ｂ１ , W Ｂ２ , W Ｇ２ , W Ｒ３ , W Ｒ４ , W ＭＲ５ represents a weighting factor for the squared difference value of each term.

[0055] When the calculated error value Diff is calculated by the above (Equation 1), the biological parameter calculation unit 61 calculates the biological parameters so that the calculated error value Diff is minimized. As a result, the oxygen saturation level S, the hemoglobin concentration Hb, the bilirubin concentration Bb, and the scattering wavelength-dependent parameter b are obtained as biological parameters. The biological parameters may be calculated for each pixel, or may be calculated for each pixel region having multiple pixels.

[0056] In this embodiment, it is preferable to calculate the biological parameters using N or more reference spectroscopic signals at different wavelengths determined by N-1 (N is an integer greater than 3) biological parameters including oxygen saturation, and N or more first spectroscopic signals corresponding to the wavelengths of the N or more reference signals. By calculating the biological parameters using N first spectroscopic signals that are greater than N-1, which is the number of biological parameters, the N-1 biological parameters can be calculated with high accuracy.

[0057] In this embodiment, in order to accurately calculate the four biological parameters, namely, the oxygen saturation level S, the hemoglobin concentration Hb, the bilirubin concentration Bb, and the scattering wavelength-dependent parameter b, six image signals, namely, the B1 image signal, the B2 image signal, the G2 image signal, the R3 image signal, the R4 image signal, and the MR5 image signal, are used as the measured first spectral image signals.

[0058] The calculated oxygen saturation level is displayed on the extended display 18 as an oxygen saturation image in various display modes. For example, it is preferable to display a still image of the oxygen saturation level alongside a moving image of the white light equivalent image on the extended display 18. Furthermore, when displaying the moving image of the white light equivalent image and the moving image of the oxygen saturation image alongside each other, it is preferable to display the moving image of the oxygen saturation image at a lower frame rate. The white light equivalent image is preferably generated based on the measured second spectral signal (second spectral signal) captured by the imaging sensor 36 while the object of observation is irradiated with the second illumination light. The measured second spectral signal is preferably the Bc image signal, Gc image signal, and Rc image signal obtained in odd-numbered frames (frames 1, 3, 5, 7, and 9) in the oxygen saturation mode.

[0059] In addition to oxygen saturation, hemoglobin concentration, bilirubin concentration, and scattering wavelength-dependent parameters may also be calculated as biological parameters, and these may be displayed as images on the extended display 18. For example, a hemoglobin concentration image based on hemoglobin concentration may be displayed on the extended display 18. Because a combination of hemoglobin concentration and oxygen saturation can be used as an index of congestion, it is preferable to display an oxygen saturation image and a hemoglobin concentration image on the extended display 18 alongside a white light image. Furthermore, an index combining oxygen saturation and hemoglobin concentration may be visualized and displayed on the extended display 18. For example, a pseudocolor image in which oxygen saturation is assigned to color differences Cr and Cb and hemoglobin concentration is assigned to luminance Y is displayed on the extended display 18. In this case, the higher the hemoglobin concentration in the pseudocolor image, the darker it appears.

[0060] The alignment unit 64 aligns the measured first spectral signals based on the amount of movement between the measured second spectral signals. The alignment unit 64 calculates the amount of movement of the object from the measured second spectral signals obtained in odd-numbered frames (frames 1, 3, 5, 7, and 9) in the oxygen saturation mode, and uses the calculated amount of movement to align the B1 image signal, B2 image signal, G2 image signal, R3 image signal, R4 image signal, B5 image signal, G5 image signal, and R5 image signal of the measured first spectral signals. For example, as shown in FIG. 13 , the alignment unit 64 calculates the amount of movement from the Bc image signal, Gc image signal, and Rc image signal of frames 1, 3, and 5. The calculated amount of movement is used to align the B1 image signal of frame 2 with the B2 image signal and G2 image signal of frame 4. In FIG. 12, the B1 image signal of frame 2 is aligned to match the positions of the B2 image signal and G2 image signal of frame 4, but conversely, the B2 image signal and G2 image signal of frame 4 may be aligned to match the position of the B1 image signal of frame 2.

[0061] The reference spectral signal setting unit 62 sets the reference spectral signal from the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectral model, and the spectral sensitivity of the image sensor 36. The reference spectral signal is preferably normalized by a reference emission value corresponding to the emission spectrum.

[0062] In this embodiment, B1 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 2). In (Equation 2), "Lsb(λ)" represents the luminance of the monochromatic light sb. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of the living body determined from the spectral model. "Sb(λ)" represents the sensitivity of the B pixel of the image sensor 36. The denominator of (Equation 2) is the reference light emission value corresponding to the emission spectrum of the monochromatic light sb. At this reference light emission value, B1 ｃａｌ (S, Hb, Bb, b) are normalized.

[0063] In this embodiment, B2 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 3). In (Equation 3), "Lg(λ)" represents the luminance of monochromatic light g. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of the living body determined from the spectral model. "Sb(λ)" represents the sensitivity of the B pixel of the image sensor 36. The denominator of (Equation 3) is the reference light emission value corresponding to the emission spectrum of monochromatic light g. At this reference light emission value, B2 ｓｔｄ (S, Hb, Bb, b) are normalized.

[0064] In this embodiment, G2 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 4). In (Equation 4), "Lg(λ)" represents the luminance of monochromatic light g. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of the living body determined from the spectral model. "Sg(λ)" represents the sensitivity of the G pixel of the image sensor 36. The denominator of (Equation 4) is the reference light emission value corresponding to the emission spectrum of monochromatic light g. At this reference light emission value, G2 ｓｔｄ (S, Hb, Bb, b) are normalized.

[0065] In this embodiment, R3 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 5). In (Equation 5), "La(λ)" represents the luminance of monochromatic light a. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of a living body determined from a spectral model. "Sr(λ)" represents the sensitivity of the R pixel of the image sensor 36. The denominator of (Equation 5) is the reference light emission value corresponding to the emission spectrum of monochromatic light a. At this reference light emission value, R3 ｓｔｄ (S, Hb, Bb, b) are normalized.

[0066] In this embodiment, R4 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 6). In (Equation 6), "Lr(λ)" represents the luminance of monochromatic light r. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of a living body determined from a spectral model. "Sr(λ)" represents the sensitivity of the R pixel of the image sensor 36. The denominator of (Equation 6) is the reference light emission value corresponding to the emission spectrum of monochromatic light r. At this reference light emission value, R4 ｓｔｄ (S, Hb, Bb, b) are normalized.

[0067] In this embodiment, MR5 ｓｔｄ (S, Hb, Bb, b) are set according to (Equation 7). In (Equation 7), "Lir(λ)" represents the luminance of monochromatic light r. "R(λ; S, Hb, Bb, b)" represents the spectral reflectance of a living body determined from a spectroscopic model. "Sr(λ)", "Sg(λ)", and "Sb(λ)" represent the sensitivities of the R, G, and B pixels of the image sensor 36, respectively. The denominator of (Equation 6) is the reference light emission value corresponding to the emission spectrum of monochromatic light ir, and R5 is determined at this reference light emission value. ｓｔｄ (S, Hb, Bb, b) are normalized.

[0068] The spectral reflectance R(λ; S, Hb, Bb, b) of a living body is calculated from the ratio μa / μs' between the absorption coefficient μa(λ; Hb, S, Bb) and the scattering coefficient μs'(λ; b). The relationship between the spectral reflectance R and the ratio μa / μs' is shown in FIG. 14. The functional form of the spectral reflectance R (ratio μa / μs') may be stored in a table, or may be approximated and stored using a simple function such as a polynomial. In FIG. 14, both the spectral reflectance R and the ratio μa / μs' are logarithmic.

[0069] The absorption coefficient μa(λ; Hb, S, Bb) is expressed by (Equation 8). μa(λ; Hb, S, Bb) = Hb × {S / 100 × μaHbO2(λ) + (1-S / 100) × μaHb(λ)} + Bb × μaBb(λ) ... (Equation 8) In (Equation 8), μaHbO2(λ) represents the absorption coefficient of oxygenated hemoglobin, and Hb(λ) represents the absorption coefficient of reduced hemoglobin. μaBb(λ) represents the absorption coefficient of bilirubin. λ represents the wavelength (similarly to Equation (9)).

[0070] The scattering coefficient μs'(λ;b) is expressed by (Equation 9): μs'(λ;b)=14 cm －１ × (λ / 500 nm) －ｂ ...(Formula 9)

[0071] When the emission spectrum of the illumination light is changed, the reference spectral signal setting unit 62 resets the reference spectral signal. For example, as shown in FIG. 15, when the light intensity of the monochromatic light sb emitted in frame 2 is increased from Lx to Ly, the reference spectral signal is reset according to (Equation 2). ｓｔｄ The reference spectral signal setting unit 62 also resets the reference spectral signal when the image sensor 36 is changed to one with a different spectral sensitivity. For example, as shown in FIG. 16, when the endoscope 12A having the image sensor 36 with the spectral sensitivity X is removed from the light source device 13 and the endoscope 12B having the image sensor 36 with the spectral sensitivity Y different from the spectral sensitivity X is attached to the light source device 13, the reference spectral signal setting unit 62 resets the reference spectral signal according to (Equation 2) to (Equation 7). ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄImage signal and MR5 ｓｔｄ Reset the image signal.

[0072] Next, the sequence of operations in the oxygen saturation mode will be described with reference to the flowchart in FIG. 17 . When the mode selector switch 12f is operated to switch to the oxygen saturation mode, polychromatic light and monochromatic light are alternately used as illumination light. The measured first spectral signal acquirer 60 acquires a measured first spectral signal by capturing an image of the object illuminated with monochromatic light using the image sensor 36. The biological parameter calculator 61 calculates biological parameters, including oxygen saturation, based on the measured first spectral signal and a reference spectral signal set based on the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectral model, and the spectral sensitivity of the image sensor 36. The extended display 18 displays an oxygen saturation image based on the oxygen saturation.

[0073] [Second Embodiment] In the second embodiment, when calculating a biological parameter, a spectroscopic model does not use the biological parameter as an argument (variable), but instead predefines a plurality of fixed values, and biological parameters that satisfy a condition are calculated from among the fixed values. In the second embodiment, M sets of reference spectral signals used to calculate the biological parameter are set for each spectroscopic model from the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectroscopic model, and the spectral sensitivity of the image sensor 36. Note that, apart from the setting of the reference spectral signals, the second embodiment is the same as the first embodiment.

[0074] Specifically, 16 spectroscopic models are used as the minimum number of spectroscopic models required to calculate biological parameters (M=16). In this case, as shown in Fig. 18 , a total of 16 spectroscopic models SP1 to SP16 are used for two cases of oxygen saturation, "high oxygen saturation" and "low oxygen saturation," two cases of hemoglobin concentration, "high hemoglobin concentration" and "low hemoglobin concentration," two cases of bilirubin concentration, "high bilirubin concentration" and "low bilirubin concentration," and two cases of scattering wavelength-dependent parameters, "high scattering wavelength-dependent parameter" and "low scattering wavelength-dependent parameter." In this case, 16 sets of reference spectral signals are set. In Figure 18, "S" represents oxygen saturation, "Hb" represents hemoglobin concentration, "Bb" represents bilirubin concentration, and "b" represents a scattering wavelength-dependent parameter, with "high" representing a high concentration or a high parameter and "low" representing a low concentration or a low parameter (the same applies to Figures 19 and 20 (in the case of Figures 19 and 20, "medium" represents a medium concentration or an intermediate parameter)).

[0075] Furthermore, 81 spectroscopic models (M=81) are used as spectroscopic models necessary for improving the calculation accuracy of biological parameters. In this case, as shown in Fig. 19 , a total of 81 spectroscopic models SP1 to SP81 are used for three cases of oxygen saturation: "high oxygen saturation," "medium oxygen saturation," and "low oxygen saturation," three cases of hemoglobin concentration: "high hemoglobin concentration," "medium hemoglobin concentration," and "low hemoglobin concentration," three cases of bilirubin concentration: "high bilirubin concentration," "medium bilirubin concentration," and "low bilirubin concentration," and three cases of scattering wavelength-dependent parameters: "high scattering wavelength-dependent parameter," "medium scattering wavelength-dependent parameter," and "low scattering wavelength-dependent parameter." In this case, 81 sets of reference spectral signals are set.

[0076] Furthermore, as shown in FIG. 20 , as a spectroscopic model required to further improve the accuracy of biological parameter calculation, an intermediate concentration may be set for oxygen saturation between "high oxygen saturation" and "medium oxygen saturation," and an intermediate concentration may be set between "medium oxygen saturation" and "low oxygen saturation." Similar intermediate concentrations may also be set for other parameters, such as hemoglobin concentration, bilirubin concentration, and scattering wavelength-dependent parameters. In this case, a total of M spectroscopic models SP1 to SPM are used. In this case, M sets of reference spectroscopic signals are set.

[0077] In the following, for the sake of simplicity, the following description will be given using some of the spectral models SPX1 to SPX4 used to calculate the biological parameters. As shown in Fig. 21, the spectral reflectance RLX1 determined by the spectral model SPX1 is determined by the biological parameter LPX1. The biological parameter LP1 includes an oxygen saturation SX1 for low oxygen saturation, a hemoglobin concentration HbX1 for high hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1.

[0078] The spectral reflectance RL2X determined by the spectral model SPX2 is determined by the biological parameter LPX2. The biological parameter LP2 includes the oxygen saturation SX2 for high oxygen saturation, the hemoglobin concentration HbX2 for high hemoglobin concentration, the medium bilirubin concentration BbX1, and the medium scattering wavelength-dependent parameter bX1. The spectral reflectance RLX3 determined by the spectral model SPX3 is determined by the biological parameter LPX3. The biological parameter LPX3 includes the oxygen saturation SX3 for low oxygen saturation, the hemoglobin concentration HbX3 for low hemoglobin concentration, the medium bilirubin concentration BbX1, and the medium scattering wavelength-dependent parameter bX1. The spectral reflectance RLX4 determined by the spectral model SPX4 is determined by the biological parameter LPX4. The biological parameter LPX4 includes an oxygen saturation SX4 for high oxygen saturation, a hemoglobin concentration HbX4 for low hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1.

[0079] As shown in Fig. 22, the reference spectral signal setting unit 71 of the second embodiment sets a reference spectral signal for the spectral model SPX1 from the emission spectrum of the illumination light, the spectral reflectance RLX1, and the spectral sensitivity of the image sensor 36. The method of setting the reference spectral signal is the same as in the first embodiment. As shown in Fig. 23, the reference spectral signal for the spectral model SPX1 obtained as described above includes six B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ The reference spectral signals for the spectral models SPX2, SPX3, and SPX4 are also included in the six B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ In the same manner, reference spectral signals for the spectral models SPX2, SPX3, and SPX4 are set.

[0080] The biological parameter calculation unit 70 of the second embodiment selects, from among M sets of reference spectral signals, a specific reference spectral signal that has the smallest calculated error value based on the error from the measured first spectral signal. Then, a specific biological parameter defined by a spectroscopic model corresponding to the specific reference spectral signal is calculated as the biological parameter. Specifically, when four sets of reference spectral signals for spectroscopic models SPX1, SPX2, SPX3, and SPX4 are used, if the reference spectral signal for spectroscopic model SP1 has the smallest calculated error value based on the error from the measured first spectral signal, the reference spectral signal for spectroscopic model SPX1 is selected as the specific reference spectral signal. In this case, a biological parameter LP1 defined by spectroscopic model SPX1 is calculated as the biological parameter. That is, an oxygen saturation level S1 and a hemoglobin concentration Hb1 are calculated as the biological parameters. Note that, as in the first embodiment, the calculated error value is preferably a value obtained by square error.

[0081] [Third Embodiment] In the third embodiment, when calculating biological parameters, instead of a spectroscopic model, multiple phantoms that mimic the spectroscopic model are used, and biological parameters that satisfy conditions are calculated from the biological parameters determined by the phantoms. In the third embodiment, M sets of reference spectral signals used to calculate the biological parameters are set for each phantom based on the emission spectrum of the illumination light, the spectral reflectance of the biological body determined by spectroscopic measurement of the phantom, and the spectral sensitivity of the image sensor 36. Note that, apart from the setting of the reference spectral signals, the third embodiment is similar to the first embodiment.

[0082] Specifically, for the phantoms required at a minimum for calculating the biological parameters, it is preferable to use phantoms corresponding to the 16 spectral models shown in FIG. 18 . In this case, 16 sets of reference spectral signals are set. Furthermore, for the phantoms required to improve the accuracy of calculating the biological parameters, it is preferable to use phantoms corresponding to the 81 spectral models shown in FIG. 19 . In this case, 81 sets of reference spectral signals are set. Furthermore, for the phantoms required to further improve the accuracy of calculating the biological parameters, it is preferable to use phantoms corresponding to the M spectral models shown in FIG. 20 . In this case, M sets of reference spectral signals are set.

[0083] For simplicity of explanation, the following description will be given using phantoms FHX1 to FHX4, which are some of the phantoms used to calculate the biological parameters. As shown in Fig. 24, phantom FHX1 has a spectral reflectance RLX1 determined by a biological parameter LPX1. The biological parameter LPX1 includes an oxygen saturation level SX1 for low oxygen saturation, a hemoglobin concentration HbX1 for high hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1.

[0084] Phantom FHX2 has a spectral reflectance RLX2 determined by a biological parameter LPX2. The biological parameter LPX2 includes an oxygen saturation level SX2 for high oxygen saturation, a hemoglobin concentration HbX2 for high hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1. Phantom FHX3 has a spectral reflectance RLX3 determined by a biological parameter LPX3. The biological parameter LPX3 includes an oxygen saturation level SX3 for low oxygen saturation, a hemoglobin concentration HbX3 for low hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1. Phantom FH4 has a spectral reflectance RLX4 determined by a biological parameter LPX4. The biological parameter LPX4 includes an oxygen saturation level SX4 for high oxygen saturation, a hemoglobin concentration HbX4 for low hemoglobin concentration, a medium bilirubin concentration BbX1, and a medium scattering wavelength-dependent parameter bX1. The above spectral reflectances RLX1 to RLX4 are calculated in advance by measuring a phantom with a spectroscopic measurement device. The calculated spectral reflectances RLX1 to RLX4 are stored in advance in the extension processor device 17.

[0085] As shown in Fig. 25, the reference spectral signal setting unit 81 of the third embodiment sets a reference spectral signal for the phantom FHX1 from the emission spectrum of the illumination light, the spectral reflectance RLX1, and the spectral sensitivity of the image sensor 36. The method for setting the reference spectral signal is the same as in the first embodiment. As shown in Fig. 26, the reference spectral signal for the phantom FHX1 obtained as described above includes six B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ The reference spectral signals for phantoms FHX2 to FHX4 are also included in the six B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄIn the same manner, reference spectral signals for the phantoms FHX2, FHX3, and FHX4 are set.

[0086] The biological parameter calculation unit 80 of the third embodiment selects, from among M sets of reference spectral signals, a specific reference spectral signal that has the smallest calculated error value based on the error from the measured first spectral signal. Then, a specific biological parameter defined by a spectroscopic model corresponding to the specific reference spectral signal is calculated as the biological parameter. Specifically, when four sets of reference spectral signals for phantoms FHX1 to FHX4 are used, if the reference spectral signal for phantom FHX1 has the smallest calculated error value based on the error from the measured first spectral signal, the reference spectral signal for phantom FHX1 is selected as the specific reference spectral signal. In this case, a biological parameter LPX1 defined for phantom FHX1 is calculated as the biological parameter. That is, oxygen saturation S1 and hemoglobin concentration Hb1 are calculated as the biological parameters. Note that the calculated error value is preferably a value obtained by square error, as in the first embodiment.

[0087] [Fourth Embodiment] In the fourth embodiment, when calculating biological parameters, instead of a spectral model, multiple phantoms that mimic the spectral models are used, and each phantom is illuminated with monochromatic light and imaged to obtain a reference spectral signal. Then, among the reference spectral signals, biological parameters determined by the phantom that correspond to a reference spectral signal that satisfies a condition are calculated as biological parameters. Therefore, in the fourth embodiment, the reference spectral signal acquisition unit 90 shown in FIG. 27 acquires M sets of reference spectral signals in advance for each phantom by illuminating the phantom with monochromatic light and imaging it. Note that the fourth embodiment is similar to the first embodiment except for the acquisition of the reference spectral signals.

[0088] Specifically, for the minimum number of phantoms required for calculating the biological parameters, it is preferable to use phantoms corresponding to the 16 spectral models shown in FIG. 18 . In this case, 16 sets of reference spectral signals are acquired. Furthermore, for phantoms required to improve the accuracy of calculating the biological parameters, it is preferable to use 81 phantoms corresponding to the spectral models shown in FIG. 19 . In this case, 81 sets of reference spectral signals are acquired. Furthermore, for phantoms required to further improve the accuracy of calculating the biological parameters, it is preferable to use phantoms corresponding to the M spectral models shown in FIG. 20 . In this case, M sets of reference spectral signals are acquired.

[0089] For simplicity's sake, the following description will be given using phantoms FHX1 to FHX4, which are some of the phantoms used to calculate biological parameters. As shown in FIG. 28 , phantom FHX1 is illuminated with monochromatic light beams sb, g, a, r, and ir, and an image is captured by the image sensor 36 for each monochromatic light beam. This capture results in B1, G1, and R1 image signals, B2, G2, and R2 image signals, B3, G3, and R3 image signals, B4, G4, and R4 image signals, and B5, G5, and R5 image signals. Of these image signals, the B1 image signal, B2 image signal, G2 image signal, R3 image signal, and R4 image signal, and the MR5 image signal, which is a combination of the B5, G5, and R5 image signals, are each used as a reference spectral signal.

[0090] That is, as shown in FIG. 26, in the case of the phantom FHX1, the reference spectral signal is B1 ｓｔｄ Image signal, B2 ｓｔｄ Image signal, G2 ｓｔｄ Image signal, R3 ｓｔｄ Image signal, R4 ｓｔｄ Image signal and MR5 ｓｔｄ Similarly, for the phantoms FHX2 to FHX4, reference spectral signals are obtained for each phantom by illuminating the phantoms FHX2 to FHX4 with monochromatic light and capturing an image, just as in the case of the phantom FHX1.

[0091] The biological parameter calculation unit 91 of the fourth embodiment selects, from among M sets of reference spectral signals, a specific reference spectral signal that has the smallest calculated error value based on the error from the measured first spectral signal. Then, a specific biological parameter determined for the phantom corresponding to the specific reference spectral signal is calculated as the biological parameter. Specifically, when four sets of reference spectral signals for phantoms FHX1 to FHX4 are used, if the reference spectral signal for phantom FHX1 has the smallest calculated error value based on the error from the measured first spectral signal, the reference spectral signal for phantom FHX1 is selected as the specific reference spectral signal. In this case, a biological parameter LPX1 determined for phantom FHX1 is calculated as the biological parameter. That is, oxygen saturation S1 and hemoglobin concentration Hb1 are calculated as the biological parameters. It is preferable that the calculated error value be a value obtained by square error, as in the first embodiment.

[0092] As the endoscope, an endoscope 12, which is a flexible endoscope for the digestive tract, or an endoscope that is a rigid endoscope for laparoscopy may be used. When a rigid endoscope is used, an endoscope system 200 shown in Fig. 29 is used. The endoscope system 200 includes an endoscope 101, a light source device 13, a processor device 14, a display 15, a processor-side user interface 16, an extended processor device 17, and an extended display 18. In the following description of the endoscope system 100, parts that are common to the endoscope system 10 will be omitted, and only the differences will be described.

[0093] 30 , the imaging unit 103 separates light from the endoscope 101 into light of multiple wavelength bands and acquires image signals based on the separated wavelength bands. The imaging unit 103 includes dichroic mirrors 105, 106, and 107 and monochrome image sensors 110, 111, 112, and 113. The dichroic mirror 105 reflects light in the blue band among the light reflected from the endoscope 101 and transmits light with wavelengths longer than that of the blue band. The light in the blue band reflected by the dichroic mirror 105 is incident on the imaging sensor 110. The monochromatic light sb, the blue band portion of the monochromatic light g, and the monochromatic light ir are incident on the imaging sensor 110.

[0094] The dichroic mirror 106 reflects light in the green wavelength range out of the light that has passed through the dichroic mirror 105, and transmits light with longer wavelengths than the light in the green wavelength range. The light in the green wavelength range reflected by the dichroic mirror 106 is incident on the image sensor 111. The light in the green wavelength range out of the monochromatic light g and the monochromatic light ir are incident on the image sensor 111. On the other hand, the light that has passed through the dichroic mirror 106 is incident on the image sensor 112. The monochromatic light r and the monochromatic light ir are incident on the image sensor 112.

[0095] In normal mode, the image sensors 110, 111, and 112 output Bc, Gc, and Rc image signals in response to the incidence of monochromatic light b, monochromatic light g, and monochromatic light r, respectively. In oxygen saturation mode, in odd-numbered frames (frames 1, 3, 5, 7, and 9), the image sensors 110, 111, and 112 output Bc, Gc, and Rc image signals in response to the incidence of monochromatic light b, monochromatic light g, and monochromatic light r, respectively. Meanwhile, in even-numbered frames in oxygen saturation mode, in frame 2, the image sensor 110 outputs a B1 image in response to the incidence of monochromatic light sb. In frame 4, the image sensor 110 outputs a B2 image signal in response to the incidence of light in the blue band of the monochromatic light g, and the image sensor 111 outputs a G2 image signal in response to the incidence of monochromatic light g.

[0096] In frame 6, the image sensor 112 outputs an R3 image signal in response to the incidence of monochromatic light a. In frame 8, the image sensor 112 outputs an R4 image signal in response to the incidence of monochromatic light r. In frame 10, the image sensor 110 outputs a B5 image signal in response to the incidence of monochromatic light ir, the image sensor 111 outputs a G5 image signal in response to the incidence of monochromatic light ir, and the image sensor 112 outputs an R5 image signal in response to the incidence of monochromatic light ir. These three image signals, B5, G5, and R5, are combined to generate an MR5 image signal.

[0097] When using an endoscope that is a rigid endoscope for laparoscopy, an endoscope system 200 that images the object of observation using another imaging method may be used instead of the endoscope system 100 that images the object of observation using the three monochrome image sensors 110-112. As shown in FIG. 31 , the endoscope system 200 uses a one-sensor type abdominal endoscope 201 that has one color image sensor 203. The image sensor 203 is provided in an imaging unit 205 of the endoscope 201. The spectral sensitivity of the image sensor 203 is the same as that of the image sensor 36. The rest is the same as that of the endoscope system 100.

[0098] In the above embodiment, the measured first spectral signal is acquired by sequentially switching between monochromatic lights and capturing images with the image sensor 36. However, instead, the measured first spectral signal may be acquired by illuminating with broadband illumination light and capturing images with a spectroscopic imaging sensor that splits the broadband illumination light into first illumination lights. Specifically, as shown in FIG. 32 , illumination light having a wavelength range of 300 nm to 1000 nm is used as the broadband illumination light. As the spectroscopic imaging sensor, a snapshot mosaic hyperspectral imaging sensor is preferably used.

[0099] In this case, as shown in Fig. 33 , the spectral image sensor 300 has a color filter array, each consisting of nine pixels (3 pixels vertically and 3 pixels horizontally), arranged in a tile pattern. In the color filter array, pixels are arranged with color filters having transmission bands with center wavelengths of 450 nm, 470 nm, 500 nm, 540 nm, 620 nm, 690 nm, and 850 nm. In Fig. 33 , "450 nm, 470 nm, 500 nm, 540 nm, 620 nm, 690 nm, and 850 nm" represent pixels with color filters having the respective transmission band center wavelengths. By capturing an image of an object illuminated with broadband illumination light using the spectroscopic imaging sensor 300, the spectroscopic imaging sensor 300 outputs a b image signal from the 450 nm pixel, an sb image signal from the 470 nm pixel, an sg image signal from the 500 nm pixel, an lg image signal from the 540 nm pixel, an a image signal from the 620 nm pixel, an r image signal from the 60 nm pixel, and an ir image signal from the 850 nm pixel.

[0100] As described above, when image signals are output from the spectroscopic imaging sensor 300, each pixel has a discrete distribution of image signals, with only the image signal of that pixel present and no image signals output from other pixels present. To address this, demosaic processing, which is an interpolation process between pixels, is performed to compensate for the pixel signals of other pixels in each pixel. As a result, each pixel has image signals for all wavelength ranges (b image signal, sb image signal, sg image signal, sl image signal, a image signal, r image signal, and ir image signal). It is preferable to use the method described in JP-A-2023-529189 for the demosaic processing, for example.

[0101] Regarding the image signal after demosaic processing, the b image signal is B1 ｉｍｇ The sb image signal is treated as a B2 ｉｍｇ The image signal obtained by combining the sg and sl image signals is treated as a G2 image signal. ｉｍｇ The image signal is treated as an R3 image signal. ｉｍｇ The r image signal is treated as R4 ｉｍｇ The IR image signal is treated as an MR5ｉｍｇ The image signal is treated as an image signal, and the innate parameters are calculated in the same manner as in the above embodiment.

[0102] In the above embodiment, monochromatic light is emitted as the first illumination light while polychromatic light is emitted as the second illumination light in the oxygen saturation mode, but other light may be emitted instead. For example, as shown in Fig. 34 , when monochromatic light b, monochromatic light g, and monochromatic light a are simultaneously emitted as the second illumination light in frames 1, 3, and 5, monochromatic light sb and monochromatic light a may be simultaneously emitted as the first illumination light in frame 2, and monochromatic light g and monochromatic light r may be simultaneously emitted as the first illumination light in frame 4. Frame 6 emits only monochromatic light ir as the first illumination light.

[0103] In this case, in frame 2, the image signal output from the B pixel of the image sensor 36 is B1 ｉｍｇ the image signal output from the R pixel is R3 ｉｍｇ In frame 4, the image signal output from the B pixel of the image sensor is designated as B2 ｉｍｇ image signal output from the G pixel is G2 ｉｍｇ image signal, and the image signal output from the R pixel is R2 ｉｍｇ For frame 6, similarly to frame 10 in the above embodiment, the image signals output from the B pixels, G pixels, and R pixels of the image sensor 36 are respectively designated as B5 ｉｍｇ Image signal, G5 ｉｍｇ Image signal, R5 ｉｍｇ As described above, when the first illumination light is emitted, the number of frames can be reduced to six frames (frames 1 to 6).

[0104] In the above embodiment, the hardware structure of processing units that perform various processes, such as the actual measurement first spectral signal acquirer 60, the biological parameter calculator 61, the reference spectral signal setting unit 62, the display controller 63, and the alignment unit 64, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to function as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration specifically designed to perform various processes.

[0105] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Multiple processing units may also be configured with a single processor. Examples of multiple processing units configured with a single processor include: a first configuration, as typified by client or server computers, in which a single processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units; and a second configuration, as typified by system-on-chip (SoC), in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0106] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0107] 10 Endoscope system 12, 12A, 12B Endoscope 12a Insertion section 12b Operation section 12c Bending section 12d Tip section 12e Angle knob 12f Mode changeover switch 12h Still image acquisition instruction switch 12i Zoom operation section 12j Forceps port 13 Light source device 14 Processor device 15 Display 16 Processor-side user interface 17 Extended processor device 18 Extended display 19 Scope-side user interface 20 Light source section 21 Light source processor 20a b-LED 20b sb-LED 20c g-LED 20d a-LED 20e r-LED 20f ir-LED 20g v-LED 23 Optical path coupling section 25 Light guide 30 Illumination optical system 31 Imaging optical system 32 Illumination lens 35 Objective lens 36 Image sensor 37 Image capture processor 40 CDS / AGC circuit 41 A / D converter 45 DSP 50 Image processing unit 52 Display control unit 53 Central control unit 55a, 55b Curve 60 Measured first spectral signal acquisition unit 61 Biological parameter calculation unit 62 Reference spectral signal setting unit 63 Display control unit 64 Positioning unit 70 Biological parameter calculation unit 71 Reference spectral signal setting unit 80 Biological parameter calculation unit 81 Reference spectral signal setting unit 90 Reference spectral signal acquisition unit 91 Biological parameter calculation unit 100 Endoscope system 101 Endoscope 103 Imaging unit 105, 106 Dichroic mirror 110, 111, 112 Image sensor 200 Endoscope system 201 Endoscope 203 Image sensor 205 Image capture unit 300 Spectroscopic image sensor

Claims

1. An endoscope system comprising: a light source device that emits illumination light; an endoscope that captures an image of an object to be observed; and one or more processors that acquire a first spectral signal captured by the endoscope while the illumination light is irradiating the object to be observed, wherein the one or more processors calculate biological parameters including oxygen saturation based on the first spectral signal and a reference spectral signal determined according to the first spectral signal, and perform control to display an oxygen saturation image based on the oxygen saturation.

2. The endoscope system of claim 1, wherein the one or more processors calculate the biological parameters using N or more reference spectroscopic signals at different wavelengths determined by N-1 (N is an integer greater than 3) biological parameters including the oxygen saturation, and N or more first spectroscopic signals corresponding to the wavelengths of the N or more reference signals.

3. An endoscope system according to claim 2, wherein the biological parameters include at least one of hemoglobin concentration, bilirubin concentration, and scattering wavelength dependent parameters.

4. The endoscope system of claim 1, wherein the reference spectral signal is set from the emission spectrum of the illumination light, the spectral reflectance of the living body determined from a spectral model, and the spectral sensitivity of an imaging sensor provided in the endoscope, the spectral model has the biological parameter as an argument, and the one or more processors calculate the biological parameter so that a difference calculation value based on the difference between the reference spectral signal and the first spectral signal falls within a specific range.

5. The endoscope system according to claim 1, wherein M sets of the reference spectral signals are set for each of the spectral models based on the emission spectrum of the illumination light, the spectral reflectance of the living body determined from the spectral model, and the spectral sensitivity of the imaging sensor provided in the endoscope, and the one or more processors select, from the M sets of reference spectral signals, a specific reference spectral signal that has the smallest calculated difference value based on the difference from the first spectral signal, and calculate, as the biological parameter, a specific biological parameter determined by the spectral model that corresponds to the specific reference spectral signal.

6. The endoscope system of claim 1, wherein M sets of the reference spectral signals are set for each phantom based on the emission spectrum of the illumination light, the spectral reflectance of a living body determined by spectroscopic measurement of a phantom simulating a spectroscopic model, and the spectral sensitivity of an imaging sensor provided in the endoscope, and the one or more processors select, from the M sets of reference spectral signals, a specific reference spectral signal that has the smallest calculated difference value based on the difference from the first spectral signal, and calculate, as the biological parameter, a specific biological parameter determined in the phantom that corresponds to the specific reference spectral signal.

7. The endoscope system according to claim 1, wherein M sets of the reference spectral signals are obtained by illuminating a phantom simulating a spectral model with the illumination light for each phantom and capturing an image of the phantom, and a specific reference spectral signal is selected from the M sets of reference spectral signals, the specific reference spectral signal having the smallest calculated difference value based on the difference from the first spectral signal, and a specific biological parameter determined for the phantom corresponding to the specific reference spectral signal is calculated as the biological parameter.

8. An endoscope system according to claim 1, wherein the reference spectroscopic signal is normalized by a reference emission value corresponding to the emission spectrum of the illumination light, and the first spectroscopic signal is normalized by an average value of a plurality of the first spectroscopic signals.

9. The endoscope system according to claim 1, wherein the one or more processors reset the reference spectral signal when the emission spectrum of the illumination light is changed.

10. An endoscope system according to claim 1, wherein the endoscope has an imaging sensor, and when the imaging sensor is changed to one with a different spectral sensitivity, the one or more processors reset the reference spectral signal.

11. The endoscope system according to claim 1, wherein the light source device emits, as the illumination light, a first illumination light and a second illumination light having a broader bandwidth than the first illumination light, and the one or more processors acquire, as the first spectral signal, a signal captured while the first illumination light is being irradiated, and acquire, as the second spectral signal, a signal captured while the second illumination light is being irradiated, and perform control to display an image generated from the second spectral signal.

12. The endoscope system according to claim 9, wherein the one or more processors align the first spectroscopic signals based on the amount of movement between the second spectroscopic signals.

13. An endoscope system according to claim 1, wherein the biological parameters include hemoglobin concentration, and the display controls to display a hemoglobin concentration image based on the hemoglobin concentration.

14. An endoscope system as described in claim 11, wherein the light source device emits either the first illumination light or a second illumination light having a broader bandwidth than the first illumination light as the illumination light, the one or more processors control the illumination to alternately use the first illumination light and the second illumination light, and the first spectral signal is obtained by imaging the object of observation illuminated with the first illumination light using the endoscope.

15. An endoscope system according to claim 14, wherein the first illumination light having center wavelengths of 450 nm, 470 nm, 540 nm, 620 nm, 690 nm, and 850 nm is emitted in sequence, and the second illumination light is emitted between each emission of the first illumination light.

16. An endoscope system according to claim 11, wherein the imaging sensor provided in the endoscope is a spectroscopic imaging sensor that separates the illumination light into the first illumination light, and the spectroscopic imaging sensor obtains the first spectral signal by imaging the object illuminated by the illumination light.

17. An endoscope system according to claim 16, wherein the spectroscopic image sensor splits the illumination light into the first illumination light having center wavelengths of 450 nm, 470 nm, 500 nm, 540 nm, 620 nm, 690 nm, and 850 nm.

18. A method for operating an endoscopic system comprising a light source device that emits illumination light, an endoscope that captures an image of an object to be observed, and one or more processors that acquire a first spectral signal captured by the endoscope while the illumination light is irradiating the object to be observed, wherein the one or more processors calculate biological parameters including oxygen saturation based on the first spectral signal and a reference spectral signal determined according to the first spectral signal, and perform control to display an oxygen saturation image based on the oxygen saturation.

Images