Endoscope system, processor device, and method of operating an endoscope system

The endoscope system addresses the challenge of distinguishing between stagnant and ischemic regions by calculating and displaying relative hemoglobin and oxygen saturation values, providing clear indicators for suture failure prevention in surgical sites.

JP7590607B2Active Publication Date: 2024-11-26FUJIFILM CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2024015386
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-02-05
Publication Date
2024-11-26
Estimated Expiration
2039-05-09

AI Technical Summary

Technical Problem

Existing endoscopic oxygen saturation imaging methods struggle to distinguish between stagnant and ischemic regions, making it difficult to determine resection or anastomosis sites where suture failure is unlikely to occur, as information on oxygen saturation alone may not adequately differentiate between these areas.

Method used

An endoscope system that calculates relative values of hemoglobin concentration and oxygen saturation using multiple spectral images, generating images that indicate ischemia or congestion, allowing for stable determination of suitable resection or anastomosis sites by displaying relative value images.

Benefits of technology

The system provides clear indicators for determining ischemia or congestion, enabling surgeons to identify resection or anastomosis sites where suture failure is less likely, without the need for fluorescent drugs and allowing for real-time visualization of tissue oxygenation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007590607000001
    Figure 0007590607000001
  • Figure 0007590607000002
    Figure 0007590607000002
  • Figure 0007590607000003
    Figure 0007590607000003
Patent Text Reader

Abstract

To provide an endoscope system capable of displaying a determination index of ischemia or congestion of blood easily and stably, and to provide a processor device and an operation method of the endoscope system.SOLUTION: A measurement value calculation unit 73 calculates an actual measurement value of hemoglobin concentration and an actual measurement value of oxygen saturation of an observation object on the basis of a plurality of first spectral images. A reference value calculation unit 84 calculates a reference value of hemoglobin concentration and a reference value of oxygen saturation on the basis of a plurality of second spectral images in a reference value calculation mode. A relative value calculation unit 75 calculates a relative value of hemoglobin concentration and a relative value of oxygen saturation in an ischemia determination mode. A mode switching unit 28a receives an instruction for switching to the ischemia determination mode. When the mode switching unit 28a receives the instruction for switching to the ischemia determination mode, the mode is automatically switched to the reference value calculation mode, and switched to the ischemia determination mode after the reference value is calculated.SELECTED DRAWING: Figure 9
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an endoscope system, a processor device, and an operation method of an endoscope system that are used during or after surgery to remove a tumor such as a cancer. [Background technology]

[0002] In recent years, laparoscopic surgery, which uses a laparoscope to perform surgery, has been attracting attention. For example, in the case of colon cancer resection using a laparoscope, a laparoscope and surgical treatment tools are inserted into the abdominal cavity through a trocar inserted into the patient's abdomen. The abdominal cavity is then inflated with aeration gas made of carbon dioxide, and an image of the abdominal cavity is displayed on a monitor. The surgeon then resects the part of the large intestine where the primary lesion occurred while observing the image of the abdominal cavity displayed on the monitor. The part of the large intestine separated by the resection of the primary lesion is sutured up using an automatic stapler or the like.

[0003] Normally, in the normal wound healing process after surgery, fibroblasts in the tissue are activated 2 to 3 days after suturing, and healing is completed in about 7 days. However, if some factors inhibit healing between the tissues during this healing process, the tissues may not heal sufficiently, and a suture failure may occur in which the sutured area is partially or entirely separated.

[0004] In surgical procedures such as intestinal anastomosis after colon cancer resection, it is important to connect the intestine at a part with blood flow as much as possible to prevent postoperative suture failure. The level of blood flow is basically judged by visual inspection based on the color tone of the intestine, but angiography using fluorescent drugs such as ICG (indocyanine green) is also widely used to visualize the border of ischemia, which is difficult to see visually. The method using ICG administration requires waiting several minutes after intravenous injection and requires a dedicated light source and camera for fluorescent image observation. In addition, since the drug remains in the blood after one administration, it is difficult to re-administer and re-observe. Therefore, the use of oxygen saturation imaging, which is the visualization or imaging of the oxygen saturation of hemoglobin contained in the object of observation by endoscopy, is expected.

[0005] Patent Document 1 discloses an endoscope system equipped with a setting change means for changing reference information that defines the correlation between pixel values ​​obtained by imaging an observation site and oxygen saturation in endoscopic oxygen saturation imaging according to the characteristics of the observation site, thereby making it possible to obtain highly reliable oxygen saturation information. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2014-76375 A Summary of the Invention [Problem to be solved by the invention]

[0007] When using endoscopic oxygen saturation imaging, information on the high and low levels of oxygen saturation can be obtained from the imaged oxygen saturation information. However, in areas with low oxygen saturation, there are areas of congestion where blood is pooled in the blood vessels and areas of ischemia where there is little blood in the blood vessels. Therefore, with only oxygen saturation information, for example, it may be difficult to distinguish the boundaries between congested and ischemic areas, or between normal and ischemic areas, and this information is not necessarily sufficient as a basis for preventing postoperative suture failure.

[0008] The present invention aims to provide an endoscopic system, a processor device, and an operating method of an endoscopic system that can simply and stably display indicators for determining ischemia or congestion, which can be used to determine resection sites or anastomosis sites where suture failure is less likely to occur. [Means for solving the problem]

[0009] The endoscopic system of the present invention includes a measurement value calculation unit that calculates a first actual measured value of a concentration of hemoglobin contained in an object of observation and a first actual measured value of oxygen saturation of hemoglobin contained in an object of observation based on a plurality of first spectral images; a reference value calculation unit that calculates a reference value of the concentration and a reference value of the oxygen saturation based on a plurality of second spectral images in a reference value calculation mode; a relative value calculation unit that calculates a relative value of the concentration and a relative value of the oxygen saturation based on the first actual measured value of the concentration and the reference value of the concentration, as well as the first actual measured value of the oxygen saturation and the reference value of the oxygen saturation in an ischemia determination mode; an image generation unit that generates a relative value image that visualizes the relative value of the concentration and / or the relative value of the oxygen saturation; a display unit that displays the relative value image; and a mode switching unit that accepts an instruction to switch to the ischemia determination mode.When the mode switching unit accepts the instruction to switch to the ischemia determination mode, it automatically switches to the reference value calculation mode and switches to the ischemia determination mode after calculating the reference value.

[0010] It is preferable that the reference value calculation unit calculates the reference value of concentration or the reference value of oxygen saturation by averaging the second actual measured values ​​of concentration calculated for each pixel of the multiple second spectral images or the second actual measured values ​​of oxygen saturation calculated for each pixel of the multiple second spectral images, respectively.

[0011] It is preferable that the measurement value calculation unit includes a signal ratio calculation unit that calculates a signal ratio dependent on concentration based on a plurality of first spectral images or a plurality of second spectral images, a correlation storage unit that stores a correlation between the concentration and the signal ratio, and an actual value calculation unit that calculates a first actual value or a second actual value of the concentration corresponding to the signal ratio based on the correlation.

[0012] It is preferable that the first spectral image is an image obtained by imaging an observation object for evaluating a congestion or ischemic state, and the second spectral image is an image obtained by imaging an observation object different from the observation object for evaluating a congestion or ischemic state.

[0013] It is preferable that the relative value image includes a hemoglobin concentration image that visualizes the relative values ​​of concentration, an oxygen saturation image that visualizes the relative values ​​of oxygen saturation, and a judgment index image that visualizes the relative values ​​of concentration and the relative values ​​of oxygen saturation.

[0014] It is preferable that the image generation unit generates a hemoglobin concentration image by pseudo-color imaging the relative values ​​of concentration using a color table for concentration that stores pseudo-color information that changes depending on concentration, and generates an oxygen saturation image by pseudo-color imaging the relative values ​​of oxygen saturation using a color table for oxygen saturation that stores pseudo-color information that changes depending on oxygen saturation.

[0015] It is preferable that the image generating unit generates the determination index image by allocating the relative value of the concentration to a luminance channel and the relative value of the oxygen saturation to two color difference channels.

[0016] It is preferable that the display unit includes a display image selection unit that accepts a selection of a display image, and the display unit displays at least one of the hemoglobin concentration image, the oxygen saturation image, and the determination index image in accordance with the selection accepted by the display image selection unit. It is preferable that the display unit displays both the hemoglobin concentration image and the oxygen saturation image on the same screen.

[0017] It is preferable to include an index value calculation unit that calculates a congestion index value indicating the degree of congestion in the subject and / or an ischemia index value indicating the degree of ischemia in the subject based on the relative value of the concentration and / or the relative value of the oxygen saturation.

[0018] The processor device of the present invention includes an image acquisition unit that receives a plurality of first spectral images or a plurality of second spectral images from an endoscope device that acquires a plurality of first spectral images or a plurality of second spectral images; a measurement value calculation unit that calculates a first actual measurement value of a concentration of hemoglobin contained in an object of observation and a first actual measurement value of oxygen saturation of hemoglobin contained in the object of observation based on the plurality of first spectral images; a reference value calculation unit that calculates a reference value of the concentration and a reference value of the oxygen saturation based on the plurality of second spectral images in a reference value calculation mode; the first actual measured value of oxygen saturation and the reference value of oxygen saturation; an image generating unit generating a relative value image that visualizes the relative value of the concentration and / or the relative value of the oxygen saturation; a display unit displaying the relative value image; and a mode switching unit receiving an instruction to switch to the ischemia determination mode, wherein when the mode switching unit receives an instruction to switch to the ischemia determination mode, it automatically switches to the reference value calculation mode, and switches to the ischemia determination mode after calculating the reference value.

[0019] The operation method of the endoscope system of the present invention includes an actual measurement value calculation step in which the measurement value calculation unit calculates a first actual measurement value of the concentration of hemoglobin contained in the observation target and a first actual measurement value of the oxygen saturation of the hemoglobin contained in the observation target based on a plurality of first spectral images, a reference value calculation step in which the reference value calculation unit calculates a reference value of the concentration and a reference value of the oxygen saturation based on a plurality of second spectral images in a reference value calculation mode, and a relative value calculation step in which the relative value calculation unit calculates the first actual measurement value of the concentration and the reference value of the concentration, as well as the first actual measurement value of the oxygen saturation and the reference value of the oxygen saturation in an ischemia determination mode. the relative value calculation step of calculating a relative value of concentration and a relative value of oxygen saturation based on the value; an image generation step of generating a relative value image in which the image generation unit visualizes the relative value of concentration and / or the relative value of oxygen saturation; a display step of displaying the relative value image in which the display unit displays; and a mode switching step of receiving an instruction to switch to the ischemia assessment mode in which, when the mode switching unit receives the instruction to switch to the ischemia assessment mode, it automatically switches to the reference value calculation mode, and switches to the ischemia assessment mode after calculating the reference value. Effect of the Invention

[0020] According to the present invention, it is possible to simply and stably provide an indicator for determining ischemia or congestion, which can serve as a basis for determining resection sites or anastomosis sites where suture failure is unlikely to occur. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is an external view of a medical device system. [Diagram 2] FIG. 1 is an external view of a abdominal endoscope system. [Diagram 3] FIG. 1 is a block diagram of a luminal endoscope system. [Figure 4] FIG. 2 is an explanatory diagram of a pixel of an image sensor. [Diagram 5] FIG. 2 is an explanatory diagram of a color filter. [Figure 6] FIG. 11 is an explanatory diagram of a light emission pattern in a normal mode. [Figure 7] FIG. 13 is an explanatory diagram of a light emission pattern etc. in an ischemia evaluation mode. [Figure 8] 11A to 11C are explanatory diagrams of light emission patterns etc. in a reference value calculation mode. [Figure 9] FIG. 2 is a block diagram showing functions of an image processing unit. [Figure 10] FIG. 4 is a block diagram showing functions of a measurement value calculation unit. [Figure 11] 1 is a graph showing the correlation between hemoglobin concentration and signal ratio. [Figure 12] 1 is a graph showing the correlation between oxygen saturation and signal ratio. [Figure 13] 1 is a graph showing the extinction coefficient of hemoglobin. [Figure 14] FIG. 13 is an explanatory diagram for explaining a method for determining an actual measurement value of oxygen saturation from a signal ratio in a graph 86. [Figure 15] FIG. 13 is an explanatory diagram for explaining a method for determining a relative value of oxygen saturation from a signal ratio in a graph 86. [Figure 16]FIG. 8 is an explanatory diagram for explaining a method for determining a relative value of hemoglobin concentration from a signal ratio in a graph 85. [Figure 17] FIG. 4 is a block diagram showing a procedure for creating a hemoglobin concentration image and an oxygen saturation image. [Figure 18] 11 is a graph showing the relationship between the relative value of the hemoglobin concentration and the color difference signal. [Figure 19] 11 is a graph showing the relationship between the relative value of oxygen saturation and color difference signals. [Figure 20] FIG. 13 is a block diagram showing a procedure for creating a relative value image. [Figure 21] 11 is a graph showing the relationship between the relative value of the hemoglobin concentration for the determination index image and the luminance signal. [Figure 22] 11 is a graph showing the relationship between the relative value of oxygen saturation for a determination index image and a color difference signal. [Figure 23] FIG. 13 is an image diagram of a display device that displays a hemoglobin concentration image and an oxygen saturation image side by side. [Figure 24] FIG. 2 is an image diagram of a display device displaying either a hemoglobin concentration image or an oxygen saturation image. [Diagram 25] FIG. 13 is an image diagram of a display device that displays a determination index image. [Figure 26] 3 is a flowchart showing the operation of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] As shown in FIG. 1, in this embodiment, the endoscope system 10 is composed of a luminal endoscope system 20 and a laparoscopic endoscope system 30, and is used to resect a tumor occurring in the large intestine. First, before resecting the tumor, the luminal endoscope system 20 is used to detect the tumor in the large intestine, and a certain range (resection range) including the tumor is marked. Next, the laparoscopic endoscope system 30 is used to resect the marked resection range of the large intestine, and the large intestine separated by the resection of the tumor is sutured. Finally, the luminal endoscope system 20 is used to check whether the sutured portion has healed tissue-wise.

[0023] The luminal endoscope system 20 includes a luminal light source device 21 that generates light for illuminating the inside of the lumen, a luminal endoscope device 22 that irradiates the inside of the lumen with light from the luminal light source device 21 and captures a reflected image, and a luminal processor device 23 that processes image signals obtained by imaging with the luminal endoscope device 22. The luminal processor device 23 is connected to a display device 12 (display unit) that displays an endoscopic image, etc. obtained by image processing, and an input device 14 composed of a keyboard, etc. The display device 12 is, for example, a monitor.

[0024] The luminal endoscope device 22 captures an image of an observation target. The luminal endoscope device 22 is a flexible endoscope, and includes a luminal endoscope operation section 24a, a flexible luminal endoscope insertion section 24b, and a scope tip section 25 that is provided at the tip of the insertion section, irradiates light toward the inside of the lumen, and captures a reflected image of the inside of the lumen. The luminal endoscope device 22 has a bending section 27 provided at the tip side of the luminal endoscope insertion section 24b. The luminal endoscope operation section 24a has an angle knob 29 used for operating the bending section 27. The bending section 27 is bent based on the operation of the angle knob 29, thereby directing the scope tip section 25 in a desired direction. The scope tip section 25 is provided with an injection port (not shown) that injects a cleaning liquid toward the observation target.

[0025] In addition to the angle knob 29, the luminal endoscope operation section 24a is provided with a mode switching section 28a used for switching between observation modes, and a reference value calculation instruction section 28b for receiving an instruction to calculate a reference value.

[0026] The luminal endoscope device 22 is also provided with a forceps channel 26 for inserting a treatment tool such as a hemostatic probe. The treatment tool is inserted into the forceps channel 26 from a forceps inlet 26a provided in the operation section, and the treatment tool inserted into the forceps channel 26 protrudes from a forceps outlet 26b at the tip.

[0027] The luminal endoscope system 20 has at least three modes: a normal mode, an ischemia evaluation mode, and a reference value calculation mode. In the normal mode, an image with natural colors obtained by imaging the observation area using white light as illumination light (hereinafter referred to as a normal image) is displayed on the display device 12. In the ischemia evaluation mode, a relative value of the hemoglobin concentration (cHb, hereinafter referred to as hemoglobin concentration) contained in the observation object and the oxygen saturation (StO 2 The relative value of oxygen saturation (hereinafter referred to as oxygen saturation) is calculated, and a relative value image in which the relative value of hemoglobin concentration, the relative value of oxygen saturation, or the relative value of hemoglobin concentration and the relative value of oxygen saturation are visualized is displayed on the display device 12. The relative value image is an index for determining ischemia or congestion that can be used to determine the resection site or anastomosis site where suture failure is unlikely to occur. The surgeon or the like can use the relative value image as an index for evaluation of the ischemic state of the observation subject.

[0028] In the reference value calculation mode, the hemoglobin concentration and oxygen saturation of the reference observation target are calculated, and based on the calculated values, the previously set reference values ​​of the hemoglobin concentration and the oxygen saturation are updated. The normal mode and the ischemia evaluation mode can be switched by an instruction from the mode switching unit 28a or the like provided in the luminal endoscope device 22.

[0029] In the ischemia evaluation mode, the relative value of the hemoglobin concentration and the relative value of the oxygen saturation are calculated based on a first actual measurement value of the hemoglobin concentration and a first actual measurement value of the oxygen saturation, which are based on a plurality of spectroscopic images (first spectroscopic images) obtained by imaging an area to be evaluated for an ischemic state as an observation target, and a reference value of the hemoglobin concentration and a reference value of the oxygen saturation, which are based on a plurality of spectroscopic images (second spectroscopic images) obtained by imaging an area to be used as a reference for an ischemic state as an observation target. In the ischemia evaluation mode, the hemoglobin concentration and / or the oxygen saturation can be relatively evaluated. Therefore, the ischemia evaluation mode does not mean a mode for evaluating only ischemia, but can also evaluate conditions such as ischemia, congestion, or hyperemia in addition to oxygen saturation.

[0030] The ischemic state is a state evaluated from both the oxygen saturation and blood volume, and is a state in which the oxygen saturation is relatively low and the blood volume is relatively low.Similarly, the congestive state is a state evaluated from both the oxygen saturation and blood volume, and is a state in which the oxygen saturation is relatively low and the blood volume is relatively high.

[0031] The object to be evaluated for an ischemic state is usually an object (first observation object) that includes a lesion, and therefore the first spectral image is an image of the first observation object that includes a lesion. On the other hand, the reference value of the hemoglobin concentration and the reference value of the oxygen saturation are obtained by using a plurality of spectral images (second spectral images) obtained by imaging an object other than the object to be evaluated for an ischemic state as the observation object. Therefore, the object to be used as the reference value is usually an object (second observation object) that does not include a lesion, and therefore the second spectral image is an image of the second observation object that does not include a lesion.

[0032] The reference value calculation mode is a mode that is automatically switched to when an instruction to calculate a reference value is given by the reference value calculation instruction unit 28b or the like provided in the luminal endoscope device 22. For example, the surgeon issues an instruction to calculate a reference value by the reference value calculation instruction unit 28b or the like while pointing the scope of the luminal endoscope device 22 at a region for which a reference value is desired to be set as the observation target. This instruction switches to the reference value calculation mode. After the reference value is updated, the mode automatically returns to the mode before switching to the reference value calculation mode.

[0033] The abdominal endoscope system 30 includes an abdominal light source device 31 that generates light for illuminating the abdominal cavity, an abdominal endoscope device 32 that irradiates the abdominal cavity with the light from the abdominal light source device 31 and captures a reflected image, and an abdominal processor device 33 that processes an image signal obtained by imaging with the abdominal endoscope device 32. The abdominal processor device 33 is connected to the display device 12 and the input device 14. The abdominal endoscope device 32 captures an image of an observation target. The abdominal endoscope device 32 is a rigid endoscope, and includes a rigid abdominal endoscope insertion section 34 and a abdominal endoscope tip section 35 that is provided at the tip of the abdominal endoscope insertion section, irradiates light toward the abdominal cavity, and captures a reflected image within the abdominal cavity.

[0034] Similarly to the luminal endoscope system 20, the abdominal endoscope system 30 has at least three modes: a normal mode, an ischemia evaluation mode, and a reference value calculation mode. The two modes, the normal mode and the ischemia evaluation mode, can be switched by an instruction from a foot switch 36 or the like connected to the abdominal processor device 33. The reference value calculation mode is a mode that is automatically switched to when an instruction to calculate a reference value is given by an input device 14 or the like provided in the abdominal endoscope system 30. These three modes are the same as those described for the luminal endoscope system 20.

[0035] As shown in Fig. 2, in the abdominal endoscope system 30, in order to observe the abdominal cavity and perform a tumor removal operation, an abdominal light source device 31, an abdominal endoscope device 32, an abdominal processor device 33, an insufflation device 37, a treatment tool 38, and trocars 39a and 39b are used. In this abdominal endoscope system 30, first, CO 2 Gas (carbon dioxide) is supplied to the abdominal cavity 40a of the patient 40 to insufflate the abdominal cavity of the patient 40. This makes it possible to ensure a visual field or a surgical field within the abdominal cavity.

[0036] Next, each treatment tool 38 is inserted into the abdominal cavity 40a of the patient 40 via the trocar 39a, and the abdominal endoscope device 32 is inserted into the abdominal cavity 40a of the patient 40 via the trocar 39b. These trocars 39a and 39b have a metallic hollow tube and an operator-held portion, and the hollow tube is inserted into the body cavity by inserting the sharp tip of the hollow tube into the abdomen of the patient 40 while the operator is holding the operator-held tube. The treatment tool 38 and the abdominal endoscope device 32 are inserted into the trocars 39a and 39b, respectively, whose hollow tubes have been inserted into the abdominal cavity in this manner.

[0037] The following will be a representative description of the luminal endoscope system 20. The abdominal endoscope system 30 is similar to the luminal endoscope system 20, and therefore a description of the similar parts will be omitted.

[0038] 3, the light source device for a cavity 21 includes a light source 41 and a light source control unit 42. The light source 41 has, for example, a plurality of semiconductor light sources, which are turned on or off. When the light source 41 is turned on, illumination light that illuminates an observation target is emitted. In this embodiment, the light source 41 has four color LEDs, namely, a BS-LED (Blue Short-wavelength Light Emitting Diode) 41a, a BL-LED (Blue Long-wavelength Light Emitting Diode) 41b, a G-LED (Green Light Emitting Diode) 41c, and an R-LED (Red Light Emitting Diode) 41d.

[0039] The BS-LED 41a emits a first blue light BS in a wavelength band of 450±10 nm. The BL-LED 41b emits a second blue light BL in a wavelength band of 470±10 nm, which is a longer wavelength than the first blue light BS. The G-LED 41c emits green light G in a wavelength band of 500 nm to 600 nm. The R-LED 41d emits red light R in a wavelength band of 620±20 nm. The center wavelength and peak wavelength of each color of light may be the same or different.

[0040] The light source control unit 42 independently controls the turning on and off of each of the LEDs 41a to 41d, and the amount of light emitted when turned on, by independently inputting control signals to each of the LEDs 41a to 41d. The control of turning on and off the LEDs in the light source control unit 42 differs for each observation mode. In the normal mode, the light source control unit 42 simultaneously turns on the BS-LED 41a, G-LED 41c, and R-LED 41d, thereby simultaneously emitting the first blue light BS, green light G, and red light R.

[0041] In the ischemia evaluation mode, the light source control unit 42 alternately performs a first emission in which the first blue light BS, green light G, and red light R are emitted simultaneously by simultaneously turning on the BS-LED 41a, G-LED 41c, and R-LED 41d, and a second emission in which the second blue light BS, green light G, and red light R are emitted simultaneously by simultaneously turning on the BL-LED 41b, G-LED 41c, and R-LED 41d.

[0042] The light emitted from each of the LEDs 20a to 20d enters a light guide 43. The light guide 43 is built in the luminal endoscope device 22 and the universal cord. The universal cord is a cord that connects the luminal endoscope device 22 to the light source device 21 and the processor device 23. The light guide 43 propagates light to the scope tip portion 25 of the luminal endoscope device 22.

[0043] The luminal endoscope device 22 is composed of an electronic endoscope, and includes an illumination optical system 44a that irradiates light guided by a light guide 43 toward an observation target, an imaging optical system 44b that images the observation target, and an imaging control unit 55. The luminal endoscope device 22 also includes a connector unit (not shown) that detachably connects the luminal endoscope device 22 to the luminal light source device 21 and the luminal processor device 23.

[0044] The illumination optical system 44a and the imaging optical system 44b are provided at the scope tip 25 of the luminal endoscope device 22. The illumination optical system 44a has an illumination lens 45. Illumination light from the light guide 43 is irradiated onto the observation object via this illumination lens 45. The imaging optical system 44b has an objective lens 46 and an imaging sensor 48. The objective lens 46 causes return light from the observation object illuminated with the illumination light to enter the imaging sensor 48. As a result, an image of the observation object is formed on the imaging sensor 48.

[0045] The imaging sensor 48 is a color imaging sensor that captures an image of an object illuminated with illumination light and outputs an image signal. As shown in Fig. 4, a plurality of pixels 51 are two-dimensionally arranged in a matrix in the row direction (X direction) and column direction (Y direction) on the imaging surface 49 of the imaging sensor 48. Each pixel 51 is provided with any one of a B (blue) color filter 52, a G (green) color filter 53, and an R (red) color filter 54. The color filters 53 to 54 are arranged in a Bayer array, with the G color filters 53 arranged in a checkerboard pattern on every other pixel, and the B color filters 52 and the R color filters 54 arranged in a square lattice pattern on the remaining pixels.

[0046] As shown in FIG. 5, the B color filter 52 transmits light with a wavelength band of 380 nm to 560 nm. The G color filter 53 transmits light with a wavelength band of 450 nm to 630 nm. The R color filter 54 transmits light with a wavelength band of 580 nm to 760 nm. Therefore, the B pixel is sensitive to the wavelength band of 450±10 nm of the first blue light BS and the wavelength band of 470±10 nm of the second blue light BL. The G pixel is sensitive to the wavelength band of 500 nm to 600 nm of the green light G. The R pixel is sensitive to the wavelength band of 640±20 nm of the red light R.

[0047] As the imaging sensor 48 (see FIG. 3), a charge coupled device (CCD) imaging sensor or a complementary metal oxide semiconductor (CMOS) imaging sensor can be used.

[0048] Note that a complementary color image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) may be used instead of the primary color image sensor 48. When a complementary color image sensor is used, image signals of four colors, CMYG, are output, and by converting the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion, image signals of each color of RGB similar to those of the image sensor 48 can be obtained.

[0049] The imaging sensor 48 receives light from the objective lens 46 on a light receiving surface (imaging surface), photoelectrically converts the received light, and outputs an imaging signal (analog signal). The imaging control of this imaging sensor 48 is performed by an imaging control unit 55.

[0050] The imaging control unit 55 is electrically connected to the light source control unit 42, and performs imaging control in accordance with the light emission control of the light source control unit 42. As shown in Fig. 6, in the normal mode, the imaging control unit 55 controls the imaging sensor 48 to capture an image of an observation target illuminated with the first blue light BS, green light G, and red light R emitted under the control of the light source control unit 42, for each frame. As a result, a Bc image signal is output from the B pixel of the imaging sensor 48, a Gc image signal is output from the G pixel, and an Rc image signal is output from the R pixel. The imaging control unit 55 synchronizes the exposure time of the imaging sensor 48 with the illumination time of the illumination light.

[0051] In the ischemia evaluation mode, the imaging control unit 55 performs different imaging control for the first emission and the second emission. Specifically, in the first emission, as shown in FIG. 7, the imaging control unit 55 performs a first imaging for imaging one frame of the first observation target illuminated with the first blue light BS, the green light G, and the red light R that are simultaneously emitted during the first emission. As a result, during the first imaging, a B1m image signal is output from the B pixel of the imaging sensor 48, a G1m image signal is output from the G pixel, and an R1m image signal is output from the R pixel. Also, during the second emission, a second imaging is performed for imaging one frame of the first observation target illuminated with the second blue light BS, the green light G, and the red light R that are simultaneously emitted. As a result, during the second imaging, a B2m image signal is output from the B pixel of the imaging sensor 48, a G2m image signal is output from the G pixel, and an R2m image signal is output from the R pixel.

[0052] The B1m, G1m, and R1m image signals, as well as the B2m, G2m, and R2m image signals, correspond to the first spectral images of the present invention, respectively. Therefore, the first spectral images include the B1m, G1m, and R1m image signals, as well as the B2m, G2m, and R2m image signals.

[0053] In the reference value calculation mode, the light source control unit 42 and the imaging control unit 55 function in the same manner as in the ischemia evaluation mode. Therefore, in the reference value calculation mode, the light source control unit 42 alternately performs a first emission in which the first blue light BS, the green light G, and the red light R are emitted simultaneously by simultaneously turning on the BS-LED 41a, the G-LED 41c, and the R-LED 41d, and a second emission in which the second blue light BS, the green light G, and the red light R are emitted simultaneously by simultaneously turning on the BL-LED 41b, the G-LED 41c, and the R-LED 41d.

[0054] In the reference value calculation mode, the imaging control unit 55 performs different imaging control for the first emission and the second emission. Specifically, in the case of the first emission, as shown in FIG. 8, the imaging control unit 55 performs the first imaging for imaging one frame of the second observation target illuminated with the first blue light BS, the green light G, and the red light R that are simultaneously emitted during the first emission. As a result, during the first imaging, a B1n image signal is output from the B pixel of the imaging sensor 48, a G1n image signal is output from the G pixel, and an R1n image signal is output from the R pixel. Also, during the second emission, the imaging control unit 55 performs the second imaging for imaging one frame of the second observation target illuminated with the second blue light BS, the green light G, and the red light R that are simultaneously emitted. As a result, during the second imaging, a B2n image signal is output from the B pixel of the imaging sensor 48, a G2n image signal is output from the G pixel, and an R2n image signal is output from the R pixel.

[0055] The B1n, G1n, and R1n image signals, and the B2n, G2n, and R2n image signals correspond to the second spectral images of the present invention, respectively. Therefore, the second spectral images include the B1n, G1n, and R1n image signals, and the B2n, G2n, and R2n image signals.

[0056] The imaging signal (analog signal) output from the imaging sensor 48 is input to an A / D converter (not shown) through a scope cable. The A / D converter converts the imaging signal (analog signal) into an image signal (digital signal) corresponding to its voltage level. The converted image signal is input to the luminal processor 23 through the connector.

[0057] 3, the luminal processor device 23 includes a central control unit 56, an image acquisition unit 57, an image processing unit 58, and a display control unit 59. The image acquisition unit 57 includes a DSP (Digital Signal Processor) 61, a noise reduction unit 62, and a conversion unit 63. The display control unit 59 includes a display image selection unit 59a.

[0058] The DSP 61 performs various signal processing such as defect correction, offset processing, gain correction, linear matrix processing, gamma conversion, demosaic processing, and YC conversion processing on the received image signal. In the defect correction processing, signals from defective pixels of the imaging sensor 48 are corrected. In the offset processing, dark current components are removed from the defect corrected image signal to set an accurate zero level. In the gain correction processing, the signal level of each image signal is adjusted by multiplying the offset processed image signal of each color by a specific gain. In the linear matrix processing, the color reproducibility of the gain corrected image signal of each color is improved.

[0059] In the gamma conversion process, the brightness and saturation of each image signal that has been linear matrix processed are adjusted. In the demosaic process (also called isotropic process or synchronization process), a signal of the missing color of each pixel is generated to interpolate the image signal that has been gamma converted. This demosaic process ensures that all pixels have signals of each of the RGB colors. The DSP 61 performs YC conversion process on each demosaic processed image signal to convert it into a luminance signal Y and color difference signals Cb and Cr, and outputs these to the noise reduction unit 62.

[0060] The noise reduction unit 62 performs noise reduction processing on the image signal from the DSP 52. The noise reduction processing is, for example, a moving average method or a median filter method. The image signal from which noise has been reduced by the noise reduction processing is input to the conversion unit 63. The conversion unit 63 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into images of each color of BGR again.

[0061] The central control unit 56 executes three modes, namely, a normal mode, an ischemia evaluation mode, and a reference value calculation mode, by controlling the light source control unit 42 and the imaging control unit 55. The central control unit 56 is electrically connected to the image processing unit 58, and notifies the image processing unit 58 of which of the normal mode, the ischemia evaluation mode, or the reference value calculation mode has been executed.

[0062] In the normal mode, the central control unit 56 captures a normal image. In the normal image capture, the central control unit 56 controls the light source control unit 42 to emit illumination light for the normal image. In this way, when the normal image is captured, an image of an object illuminated with the first blue light BS, the green light G, and the red light R is captured, thereby obtaining a Bc image signal, a Gc image signal, and an Rc image signal.

[0063] In the ischemia evaluation mode, the central control unit 56 controls the light source control unit 42 to alternately emit the first and second light emission of illumination light, and controls the imaging control unit 55 in accordance with the first and second light emission to alternately execute the first and second imaging. In this way, when the first emission and the first imaging are performed, a B1m image signal, a G1m image signal, and an R1m image signal are obtained by imaging the first observation target illuminated with the first blue light BS, the green light G, and the red light R. In addition, when the second emission and the second imaging are performed, a B2m image signal, a G2m image signal, and an R2m image signal are obtained by imaging the first observation target illuminated with the second blue light BS, the green light G, and the red light R. The B1m image signal, the G1m image signal, and the R1m image signal, and the B2m image signal, the G2m image signal, and the R2m image signal are two sets of consecutive image signals.

[0064] In the reference value calculation mode, the central controller 56 controls the light source controller 42 to alternately emit the first and second illumination light, as in the ischemia evaluation mode, and controls the imaging controller 55 in accordance with the first and second illumination light to alternately perform the first and second imaging. In this way, when the first emission and the first imaging are performed, a B1n image signal, a G1n image signal, and an R1n image signal are obtained by imaging the second observation target illuminated with the first blue light BS, the green light G, and the red light R. In addition, when the second emission and the second imaging are performed, a B2n image signal, a G2n image signal, and an R2n image signal are obtained by imaging the second observation target illuminated with the second blue light BS, the green light G, and the red light R. The B1n image signal, the G1n image signal, and the R1n image signal, and the B2n image signal, the G2n image signal, and the R2n image signal are two sets of consecutive image signals.

[0065] 9, the image processing unit 58 includes a normal image processing unit 71 and a functional image processing unit 72, and performs predetermined image processing on the image signal from the luminal endoscope device 22. The normal image processing unit 71 generates a normal image by performing predetermined image processing on the image signal obtained in the normal mode.

[0066] The functional image processing unit 72 processes images obtained in the ischemia evaluation mode or the reference value calculation mode. The functional image processing unit 72 includes a measurement value calculation unit 73, a reference value calculation instruction receiving unit 74, a relative value calculation unit 75, and an image generation unit 76. The functional image processing unit 72 performs a series of processes based on the image signal input from the luminal endoscope device 22, calculates the relative value of the hemoglobin concentration and the relative value of the oxygen saturation, and generates a relative value image using the relative value of the hemoglobin concentration and / or the relative value of the oxygen saturation.

[0067] The measurement value calculation unit 73 has a function of calculating actual values ​​or reference values ​​of hemoglobin concentration and oxygen saturation based on a plurality of spectral images. When the spectral image is a first spectral image, the calculated hemoglobin concentration and oxygen saturation are set as first actual measured values. When the spectral image is a second spectral image, the calculated hemoglobin concentration and oxygen saturation are set as second actual measured values. Therefore, the hemoglobin concentration calculated based on the plurality of first spectral images is the first actual measured value of the hemoglobin concentration, and the oxygen saturation calculated based on the plurality of first spectral images is the first actual measured value of the oxygen saturation. Moreover, the hemoglobin concentration calculated based on the plurality of second spectral images is the second actual measured value of the hemoglobin concentration, and the oxygen saturation calculated based on the plurality of second spectral images is the second actual measured value of the oxygen saturation. The second actual measured values ​​are used to calculate the reference value.

[0068] The reference value calculation instruction receiving unit 74 receives an instruction to calculate a reference value. The relative value calculating unit 75 calculates a relative value of the hemoglobin concentration and a relative value of the oxygen saturation based on the first actual measured value of the hemoglobin concentration and the first actual measured value of the oxygen saturation, and the reference value of the hemoglobin concentration and the reference value of the oxygen saturation, calculated by the measurement value calculating unit 73. The image generating unit 76 generates a relative value image that visualizes the relative value of the hemoglobin concentration and / or the relative value of the hemoglobin oxygen saturation.

[0069] 10, the measurement value calculation unit 73 includes a signal ratio calculation unit 81, a correlation storage unit 82, an actual measurement value calculation unit 83, and a reference value calculation unit 84. The signal ratio calculation unit 81, the correlation storage unit 82, and the actual measurement value calculation unit 83 calculate first actual measurement values ​​of hemoglobin concentration and oxygen saturation based on the multiple first spectral images, and calculate second actual measurement values ​​of hemoglobin concentration and oxygen saturation based on the multiple second spectral images. The reference value calculation unit 84 calculates a reference value from the second actual measurement values.

[0070] The signal ratio calculation unit 81 calculates a signal ratio that is dependent on both the hemoglobin concentration and the oxygen saturation using a plurality of first spectral images obtained by capturing an image of a first observation target including a lesion in the ischemia evaluation mode. The signal ratio that is dependent on the hemoglobin concentration is the same as the signal ratio that is dependent on the blood volume.

[0071] The signal ratio calculation unit 81 calculates a signal ratio between pixels at the same position in the B1m image signal, G1m image signal, and R1m image signal of the first imaging, which are the first spectral image acquired in the ischemia evaluation mode, and the B2m image signal, G2m image signal, and R2m image signal of the second imaging. The signal ratio is calculated for all pixels of the image signal. Therefore, the hemoglobin concentration and the oxygen saturation are calculated for each pixel. In this embodiment, the signal ratio calculation unit 81 calculates a signal ratio B2m / (B1m+G1m) between the blue signal B1m of the first imaging and the green signal G1m of the first imaging and the green signal G2m of the second imaging, and a signal ratio R1m / G1m between the green signal G1m of the first imaging and the red signal R1m. Note that the signal ratio may be calculated only for pixels of the blood vessel portion of the image signal. In this case, the blood vessel portion is identified based on the difference between the image signal of the blood vessel portion and the image signal of the other portion.

[0072] The correlation storage unit 82 stores correlations between the signal ratios B2m / (B1m+G1m) and R1m / G1m and the hemoglobin concentration and oxygen saturation. The correlations between the signal ratios and the hemoglobin concentration are stored in a one-dimensional table defined such that the hemoglobin concentration increases as the signal ratio R1m / G1m increases, as shown in Fig. 11. The signal ratio R1m / G1m is stored in a log scale.

[0073] Meanwhile, the correlation between the signal ratio and oxygen saturation is stored in a two-dimensional table that defines the oxygen saturation contour lines in a two-dimensional space as shown in Figure 12. The position and shape of these contour lines are obtained by a physical simulation of light scattering, and are defined to change according to the blood volume. For example, if there is a change in the blood volume, the intervals between the contour lines will become wider or narrower. The signal ratios B2m / (B1m+G1m) and R1m / G1m are stored in log scale.

[0074] The above correlation is closely related to the light absorption and light scattering characteristics of oxygenated hemoglobin and reduced hemoglobin as shown in FIG. 13. In FIG. 13, a graph 91 shows an absorption coefficient 91a of oxygenated hemoglobin and an absorption coefficient 91b of reduced hemoglobin. As shown in FIG. 13, for example, at a wavelength such as 470 nm where the difference in absorption coefficient is large, it is easy to obtain information on oxygen saturation. However, a blue signal including a signal corresponding to light of 470 nm is highly dependent not only on oxygen saturation but also on blood volume. Therefore, by using the signal ratios B2m / (B1m+G1m) and R1m / G1m obtained from the blue signal B1m, the red signal R1m corresponding to light that changes mainly depending on the blood volume, and the green signal G1m that is the reference signal for the blue signal B2m and the red signal R1m, in addition to the blue signal B1m, the oxygen saturation can be accurately obtained without depending on the blood volume.

[0075] In addition, the following three points can be made based on the wavelength dependence of the absorption coefficient of hemoglobin in the tissue being observed: (1) In the vicinity of a wavelength of 470 nm (for example, in the blue wavelength region with a central wavelength of 470 nm ±10 nm), the absorption coefficient changes significantly in response to changes in oxygen saturation. (2) When averaged over the green wavelength range of 540 to 580 nm, it is less affected by oxygen saturation. (3) In the red wavelength range of 590 to 700 nm, the absorption coefficient appears to change significantly depending on the oxygen saturation level, but since the value of the absorption coefficient itself is very small, it is not affected by the oxygen saturation level.

[0076] The actual measurement value calculation unit 83 calculates both the hemoglobin concentration and the oxygen saturation level at each pixel using the correlation stored in the correlation storage unit 82 and the signal ratios B2m / (B1m+G1m) and R1m / G1m calculated by the signal ratio calculation unit 81. Regarding the hemoglobin concentration, the value corresponding to the signal ratio R1m / G1m calculated by the signal ratio calculation unit 81 in the one-dimensional table of the correlation storage unit 82 becomes the hemoglobin concentration. Meanwhile, regarding the oxygen saturation level, first, the signal ratio B2m calculated by the signal ratio calculation unit 81 in the two-dimensional space as shown in FIG. * / (B1m * +G1m * ) and signal ratio R1m * / G1m * A corresponding point P corresponding to is identified.

[0077] As shown in FIG. 14, when the corresponding point P is between the lower limit line 93 of "oxygen saturation=0% limit" and the upper limit line 94 of "oxygen saturation=100% limit", the percentage value indicated by the contour line on which the corresponding point P is located becomes the oxygen saturation. For example, in the case of FIG. 14, the contour line on which the corresponding point P is located indicates 60%, so this 60% becomes the oxygen saturation. If the corresponding point is out of the range between the lower limit line 93 and the upper limit line 94, the oxygen saturation is set to 0% when the corresponding point is located above the lower limit line 93, and the oxygen saturation is set to 100% when the corresponding point is located below the upper limit line 94. If the corresponding point is out of the range between the lower limit line 93 and the upper limit line 94, the reliability of the oxygen saturation at that pixel may be lowered so as not to be displayed.

[0078] Next, calculation of the reference values ​​of hemoglobin concentration and oxygen saturation for a plurality of second spectral images will be described. The calculation of the reference value is started by switching the mode to the reference value calculation mode when the reference value calculation instruction receiving unit 74 receives an instruction from the operator. Specifically, the operator gives an instruction by, for example, pressing the scope button which is the reference value calculation instruction unit 28b. For example, the operator presses the scope button which is the reference value calculation instruction unit 28b for an observation target for which a reference value is to be set during an examination, thereby switching to the reference value calculation mode, and a plurality of second spectral images of the observation target are obtained.

[0079] Using the acquired second spectral images, the second actual measurement values ​​of the hemoglobin concentration and the oxygen saturation at each pixel are calculated by processing them in the same manner as the first actual measurement values ​​were calculated. More specifically, the signal ratio calculation unit 81 calculates a signal ratio that is dependent on both the hemoglobin concentration and the oxygen saturation using the acquired second spectral images based on the operation of the reference value calculation instruction unit 28b. The signal ratio based on the second spectral images becomes the second actual measurement value of the oxygen saturation of the second observation object. Specifically, the second actual measurement value is calculated using the signal ratio B2n / (B1n+G1n) and the signal ratio R1n / G1n calculated by the signal ratio calculation unit 81 and the correlation and the two-dimensional space (see FIG. 14) stored in the correlation storage unit 82 for the multiple second spectral images. The second measured value of the hemoglobin concentration of the second observation object is similar to the first measured value, and since the calculation of the hemoglobin concentration is based on the signal ratio R1n / G1n, the calculation of the hemoglobin concentration is performed for each pixel of these spectroscopic images.

[0080] The reference value calculation unit 84 averages the second actual measurement values ​​of the hemoglobin concentration and oxygen saturation at each pixel calculated by the actual measurement value calculation unit 83, for example, within a range of one image or within a range of one spectroscopic image. The value calculated by averaging is set as the reference value of the hemoglobin concentration or the reference value of the oxygen saturation. When averaging, the reliability of each pixel may be weighted so as not to include values ​​in an inappropriate region for the calculation of the reference value. When the reference value is calculated, the calculated reference value is sent to the relative value calculation unit 75, and the relative value calculation unit 75 calculates the relative value using the calculated reference value.

[0081] The reference value calculation unit 84 may calculate the reference value using a plurality of sets of a plurality of second spectral images. In this case, for example, one set of spectral images is regarded as one frame, the average value of the oxygen saturation of each frame is calculated, and the calculated average values ​​are added and divided by the number of frames to calculate the reference value of the oxygen saturation of the entire plurality of frames. Regarding the first actual measurement values ​​of the hemoglobin concentration and the oxygen saturation using the first spectral image, the average values ​​of pixels at the same position may be calculated for the plurality of frames, and used as the first actual measurement values ​​of the hemoglobin concentration and the oxygen saturation.

[0082] Note that the switching to the reference value calculation mode or the calculation of the reference value may be performed by a method other than starting according to the instruction of the surgeon. For example, at the beginning of the treatment, a typical value of the hemoglobin concentration or oxygen saturation indicated by a normal mucous membrane of the digestive tract from the mouth to the anus may be set as a default reference value in advance. Also, the previous reference value may be stored, and the reference value may be used until the surgeon instructs calculation of the reference value, and the reference value may be updated when the surgeon instructs calculation of the reference value. Also, when the ischemia evaluation mode is set by the mode switching unit 28a, the mode may be automatically switched to the reference value calculation mode once, the reference value may be calculated, and the reference value may be set, and then the mode may be automatically switched to the ischemia evaluation mode.

[0083] The relative value calculation unit 75 calculates the relative values ​​of the hemoglobin concentration and the oxygen saturation based on the first spectral image from the first actual measured values ​​of the hemoglobin concentration and the oxygen saturation from the first spectral image and the reference values ​​of the hemoglobin concentration and the oxygen saturation. For example, the relative value can be a value obtained by subtracting the reference value from the first actual measured value. Specifically, the first actual measured value of the oxygen saturation from the first spectral image is calculated as StO 2 (MV), and the standard value for oxygen saturation is StO 2 (STD), the relative oxygen saturation value ΔStO 2 is the first measured value of oxygen saturation, StO 2 (MV) to oxygen saturation standard value StO 2 Therefore, the relative oxygen saturation value ΔStO 2 is, for example, the difference PD when the reference value PS of the oxygen saturation level and the first measured value PM of the oxygen saturation level are taken as shown in Fig. 15. It is expressed by the following formula.

[0084] StO 2 (MV)-StO 2 (STD) = ΔStO 2 The relative value of the hemoglobin concentration is calculated in the same manner as the relative value of the oxygen saturation. That is, the relative value can be a value obtained by subtracting a reference value from a first actual measurement value. Specifically, if the first actual measurement value of the hemoglobin concentration by the first spectral image is cHb(MV) and the reference value of the hemoglobin concentration is cHb(STD), the relative value ΔcHb of the hemoglobin concentration is a value obtained by subtracting the reference value cHb(STD) of the hemoglobin concentration from the first actual measurement value cHb(MV). Therefore, the relative value ΔcHb of the hemoglobin concentration is, for example, the difference QD when the reference value QS of the hemoglobin concentration and the actual measurement value QM of the oxygen saturation are taken as shown in FIG. 16. It is expressed by the following formula.

[0085] cHb(MV)-cHb(STD)=ΔcHb Next, the image generating unit 76 calculates the relative value ΔcHb of the hemoglobin concentration and / or the relative value ΔStO of the oxygen saturation calculated by the relative value calculating unit 75. 2Specifically, a hemoglobin concentration image is generated by visualizing the concentration relative value ΔcHb in a pseudo-color using a color table for concentration that stores pseudo-color information that changes according to concentration, and a relative oxygen saturation value ΔStO 2 The relative value image is a hemoglobin concentration image in which the relative value ΔcHb of the hemoglobin concentration is visualized, and the relative value ΔStO of the oxygen saturation is visualized. 2 The oxygen saturation image, the relative value of hemoglobin concentration ΔcHb, and the relative value of oxygen saturation ΔStO 2 The present invention includes a judgment index image obtained by imaging the above.

[0086] As shown in FIG. 17, the image generating unit 76 generates a hemoglobin concentration image that represents the relative value ΔcHb of the hemoglobin concentration calculated by the relative value calculating unit 75 in pseudo-color. The hemoglobin concentration image is composed of a video signal consisting of luminance Y and color difference signals Cb and Cr. The luminance Y is assigned to the G1m image signal, which is a green signal. This G1m image signal corresponds to reflected light in a wavelength band where absorption by hemoglobin is somewhat strong, so that the unevenness of the mucous membrane and blood vessels can be visually recognized from the image based on this signal. Therefore, by assigning the G1m image signal to the luminance, the overall brightness of the pseudo-color image can be defined. Note that instead of the G1m image signal, the G2m image signal, which is a green signal, may be assigned to the luminance.

[0087] On the other hand, as shown in Fig. 18, the color difference signal Cb and the color difference signal Cr are assigned a signal value according to the relative value ΔcHb of the hemoglobin concentration according to a color table 96. The color table 96 is a color table for a hemoglobin concentration image that stores pseudo-color information that changes according to the hemoglobin concentration. The color table 96 defines the color difference signal Cb95b so that the signal value decreases as the hemoglobin concentration increases, and defines the color difference signal Cr95a so that the signal value increases as the hemoglobin concentration increases. Therefore, the hemoglobin concentration image has an increased redness where the hemoglobin concentration is high, and as the hemoglobin concentration decreases, the saturation of the redness decreases and approaches monochrome.

[0088] In addition, as shown in FIG. 17, the image generating unit 76 calculates the relative value ΔStO 2 The oxygen saturation image is generated by expressing the relative value of oxygen saturation ΔStO in pseudocolor. Like the hemoglobin concentration image, the oxygen saturation image is composed of a video signal consisting of luminance Y and color difference signals Cb and Cr. The luminance Y is assigned to the green signal of the G1m image signal or the green signal of the G2m image signal. The color difference signals Cb and Cr are assigned to the relative value of oxygen saturation ΔStO in pseudocolor in accordance with a color table 97 as shown in FIG. 2 A signal value according to the oxygen saturation level is assigned to the color table 97. The color table 97 is a color table for an oxygen saturation image that stores pseudocolor information that changes according to the oxygen saturation level.

[0089] The color table 97 is defined such that the signal value of the color difference signal Cr95a is positive and the signal value of the color difference signal Cb95b is negative under high oxygen saturation, and conversely, the signal value of the color difference signal Cr95a is negative and the signal value of the color difference signal Cb95b is positive under low oxygen saturation. The color table 97 is defined such that the magnitude relationship between the signal values ​​of the color difference signal Cr95a and the color difference signal Cb95b is reversed under medium oxygen saturation. Therefore, as the oxygen saturation level goes from low to high, the color of the oxygen saturation image changes from blue to light blue to green to yellow to orange to red.

[0090] 20, the image generating unit 76 calculates the relative value ΔcHb of the hemoglobin concentration and the relative value ΔStO of the oxygen saturation calculated by the relative value calculating unit 75. 2 A determination index image is generated that represents the above in pseudo-color. The determination index image is composed of a video signal that is made up of a luminance Y and color difference signals Cb and Cr, similar to the hemoglobin concentration image or the oxygen saturation image.

[0091] As shown in Fig. 21, the luminance Y is assigned a luminance value 95c corresponding to the relative value ΔcHb of the hemoglobin concentration according to a luminance table 98. The luminance table 98 is a color table for a determination index image that stores luminance information that changes according to the relative value ΔcHb of the hemoglobin concentration. As shown in Fig. 22, the color difference signal Cb and the color difference signal Cr are assigned a luminance value 95c corresponding to the relative value ΔStO 2 A signal value according to the relative oxygen saturation value ΔStO 2 4 is a color table for a determination index image that stores pseudo-color information that changes depending on the

[0092] The brightness table 98 is defined so that where the relative value ΔcHb of the hemoglobin concentration is positive and large, the brightness value 95c is positive and small, whereas where the relative value ΔcHb of the hemoglobin concentration is negative and has a large absolute value, the brightness value 95c is positive and large, and where the relative value ΔcHb of the hemoglobin concentration is 0 (zero), the brightness value 95c is intermediate. Therefore, as the relative value ΔcHb of the hemoglobin concentration goes from a negative value with a large absolute value to a positive value with a large absolute value, the brightness in the image changes from bright to dark.

[0093] Color table 99 shows the relative oxygen saturation value ΔStO 2 is defined such that when the signal value of the color difference signal Cr95a is positive and large, the signal value of the color difference signal Cb95b is negative and the absolute value is large. On the other hand, when the relative value ΔStO 2is negative and has a large absolute value, conversely, the signal value of the color difference signal Cr95a is negative and has a large absolute value, and the signal value of the color difference signal Cb95b is positive and has a large signal value. Also, when the relative value ΔcHb of the hemoglobin concentration is 0 (zero), the signal value of the color difference signal Cr95a is 0 (zero) and the signal value of the color difference signal Cb95b is 0 (zero). Therefore, the relative value ΔStO of the oxygen saturation 2 As the absolute value of goes from a large negative to a large positive, the image is displayed as a gradual change from blue to red.

[0094] In this way, in the judgment index image, the relative value of hemoglobin concentration ΔcHb and the relative value of oxygen saturation ΔStO 2 Since the brightness and color difference are defined based on the above, areas with high blood volume are displayed dark and areas with low blood volume are displayed bright. Furthermore, the relative value of oxygen saturation, ΔStO 2 The high oxygen saturation level (ΔStO) is displayed in red and the low oxygen saturation level (ΔStO) is displayed in blue. 2 The area where the first measured value of oxygen saturation is 0 (zero), i.e., the area where the first measured value of oxygen saturation is the same as the reference value, is displayed in monochrome. As a result, areas of congestion (high blood volume and low oxygen) are displayed in dark blue, and areas of ischemia (low blood volume and low oxygen) are displayed in light blue. Therefore, the determination indicator image makes it possible to visually distinguish congestion and ischemia from normal cases. Note that the definitions of brightness and color difference are not limited to those described above, and for example, areas with high blood volume may be displayed bright and areas with low blood volume may be displayed dark, or may be changed by settings in the image generation unit 76.

[0095] The display control unit 59 (see FIG. 2) controls images to be displayed on the display device 12. The display image selection unit 59a displays a display image selected by the input device 14 or the like on the display device 12. For example, in the ischemia evaluation mode, at least one of a hemoglobin concentration image, an oxygen saturation image, and a determination index image is selected.

[0096] In the ischemia evaluation mode, for example, the monitor as the display device 12 displays at least one of a hemoglobin concentration image, an oxygen saturation image, and a determination index image according to the selection received by the display image selection unit 59a. The display method can be set arbitrarily, but for example, as shown in FIG. 23, the hemoglobin concentration image 101 and the oxygen saturation image 102 may be reduced so that they can be displayed side by side on one screen, and the reduced images may be displayed side by side on the same screen of the monitor at the same time. Alternatively, according to the selection received by the display image selection unit 59a, as shown in FIG. 24, either the hemoglobin concentration image 101 or the oxygen saturation image 102 may be selected, and the selected image may be displayed on the monitor while switching the selection as necessary. In addition, as shown in FIG. 25, the determination index image 103 may be displayed on the monitor.

[0097] In this way, in the ischemia evaluation mode, the relative value of hemoglobin concentration ΔcHb and the relative value of oxygen saturation ΔStO 2 Since both real-time information and the information on the blood flow can be displayed in separate images or in one image, it is possible to easily compare them using an endoscope device without using a fluorescent agent or the like. In addition, since the hemoglobin concentration image, the oxygen saturation image, and the determination index image are images of relative values, information on the hemoglobin concentration and oxygen saturation can be obtained stably. In addition, it is possible to determine the distribution or boundary of the congestion or ischemia area in the area with low oxygen saturation, and to determine the distribution or boundary between the area with low oxygen saturation and the normal area, and it is useful as a judgment material for preventing postoperative suture failure. Therefore, according to the present invention, it is possible to easily and stably provide a judgment index for ischemia or congestion that can be used as a judgment material for the resection or anastomosis area where suture failure is unlikely to occur.

[0098] Next, the operation of the present invention will be described with reference to the flowchart of FIG. 26. For example, when performing a colectomy using the luminal endoscope system 20 and the abdominal endoscope system 30, the surgeon uses the luminal endoscope device 22 set to the normal mode to find the site where the tumor has occurred in the large intestine. Then, the surgeon inserts a clip device into the tumor via the forceps channel and operates the clip device to compress a relatively thick blood vessel near the tumor with the clip. The tissue around the blood vessel compressed by the clip becomes ischemic, and the oxygen saturation of the observed tissue decreases, resulting in a hypoxic region. This hypoxic region serves as a marker for identifying the location of the tumor in the next surgery. After marking with the clip, the luminal endoscope device 22 is removed from the large intestine.

[0099] Next, in the operation, the abdominal endoscope device 32 is inserted into the abdomen of the patient 40, the ischemia evaluation mode is selected (step ST100), and the scope button of the reference value calculation instruction unit 28b is pressed for a normal region of tissue (YES in step ST110). This switches to the reference value calculation mode, and the observation target is irradiated with the first emitted illumination light. The reflected light from the observation target is captured by the imaging sensor 48, which is a color CCD consisting of B pixels, G pixels, and R pixels. This results in a first frame of image signals consisting of blue signals B1n, green signals G1n, and red signals R1n. Once the first frame of image signals is obtained, the observation target is irradiated with the second emitted illumination light. The reflected light from the observation target is captured by the imaging sensor 48, and a second frame of image signals consisting of blue signals B2n, green signals G2n, and red signals R2n are obtained (step ST120).

[0100] When the image signal of the second frame is obtained, the signal ratio calculation unit 81 calculates the signal ratio B2n / (B1n+G2n) and the signal ratio R1n / G1n for pixels at the same position between the image signal of the first frame and the image signal of the second frame. The signal ratio is calculated for all pixels. When the signal ratio is calculated, the hemoglobin concentration corresponding to the signal ratio R1n / G1n calculated by the signal ratio calculation unit 81 is calculated from the correlation stored in the correlation storage unit 82, and the oxygen saturation corresponding to the signal ratio B2n / (B1n+G2n) and the signal ratio R1n / G1n calculated by the signal ratio calculation unit 81 is calculated. The hemoglobin concentration and the oxygen saturation are calculated for all pixels.

[0101] When the hemoglobin concentration and oxygen saturation are obtained for all pixels (step ST130), the reference value calculation unit 84 calculates the average values ​​of the hemoglobin concentration and oxygen saturation for each frame (step ST140). The calculated average value of the hemoglobin concentration is set as the reference value of the hemoglobin concentration, and similarly, the calculated average value of the oxygen saturation is set as the reference value of the oxygen saturation (step ST150). After the reference value is updated, the mode is automatically switched from the reference value calculation mode to the ischemia assessment mode. The monitor displays a message indicating that the reference value has been updated, and the operator is notified that the reference value has been updated and the mode has been switched to the ischemia assessment mode.

[0102] If the scope button of the reference value calculation instruction unit 28b is not pressed (NO in step ST110), the reference value used previously is set as the reference value (step ST160). A message is displayed on the monitor to the effect that the previous reference value has been set, and the surgeon is informed that the previous reference value has been set.

[0103] Next, the location marked with the clip is observed by the luminal endoscope device 22 in the ischemia determination mode of the laparoscopic device 32. The actual measurement values ​​of hemoglobin concentration and oxygen saturation contained in the observation target by observation from the abdominal cavity side are calculated. The calculation method is the same as above, so it will be omitted (step ST170, step ST180). The relative value calculation unit 75 calculates the relative value of hemoglobin concentration ΔcHb from the difference between the previously obtained reference value of hemoglobin concentration and the actual measurement value of hemoglobin concentration of the observation target. Similarly, the relative value of oxygen saturation ΔStO is calculated from the difference between the previously obtained reference value of oxygen saturation and the actual measurement value of oxygen saturation of the observation target. 2 (step ST190).

[0104] The image generating unit 76 calculates the relative value ΔcHb of the hemoglobin concentration and the relative value ΔStO of the oxygen saturation. 2 18. Then, a hemoglobin concentration image in which the relative value ΔcHb of the hemoglobin concentration is expressed in pseudocolor is generated from the color difference signals Cb and Cr thus obtained and the luminance Y to which the green signal G1m or green signal G2m due to the first light emission is assigned (step ST200, see FIG. 18). Similarly, a color table 97 is used to generate a hemoglobin concentration image in which the relative value ΔcHb of the hemoglobin concentration is expressed in pseudocolor from the color difference signals Cb and Cr thus obtained and the luminance Y to which the green signal G1m or green signal G2m due to the first light emission is assigned (step ST200, see FIG. 18). Similarly, a color table 97 is used to generate a hemoglobin concentration image in which the relative value ΔStO of the oxygen saturation is expressed in pseudocolor from the color difference signals Cb and Cr thus obtained and the luminance Y to which the green signal G1m or green signal G2m due to the first light emission is assigned (step ST200, see FIG. 18). 2 In addition, a brightness table 98 and a color table 99 are used to generate an oxygen saturation image in which the relative value of hemoglobin concentration ΔcHb and the relative value of oxygen saturation ΔStO 2 A determination index image in which the determined values ​​are displayed in pseudo-color is generated (see Figs. 21 and 22). The generated hemoglobin concentration image, oxygen saturation image, or determination index image is displayed on the display device 12 (step ST200).

[0105] By displaying at least one image of the hemoglobin concentration image, the oxygen saturation image, or the image used for judgment on the display device 12, the surgeon can identify the position of the tumor, and confirm the oxygen saturation and ischemic state near the tumor. For example, the surgeon can confirm that the part where the oxygen saturation is reduced by the clip is the tumor. In addition, since the information on both the oxygen state and the ischemic state near the tumor can be obtained by the laparoscopic device 32, even if there is a part with low oxygen saturation, the surgeon can determine the resection or anastomosis part where suture failure is less likely to occur by selecting the boundary between the ischemic area and the normal area, avoiding the ischemic area, avoiding the area where congestion and ischemia are mixed, or selecting a uniform area. If the surgeon wants to reset the reference value again and display an image that makes it easier to judge the ischemic area, the surgeon calculates the reference value again and repeats the series of operations (NO in step ST210). After the procedure is completed, the laparoscopic device 32 is removed from the patient 40, and the procedure for completing the surgery is performed (YES in step ST210).

[0106] In the above embodiment, the relative value of oxygen saturation ΔStO 2 and the relative value ΔcHb of the hemoglobin concentration, an index value calculation unit (not shown) that calculates a congestion index value indicating the degree of congestion or an ischemia index value indicating the degree of ischemia may be provided in the endoscope system 10. By displaying the congestion index value or ischemia index value on the display device 12 together with the oxygen saturation image, it becomes easier to determine whether an area with low oxygen saturation corresponds to an area of ​​congestion or an area of ​​ischemia.

[0107] Regarding the congestion index value and the ischemia index value, for example, the relative value of oxygen saturation ΔStO 2 is negative and the relative oxygen saturation value ΔStO 2 It is preferable to increase the congestion index value and the ischemia index value as the absolute value of ΔcHb increases. The congestion index value is increased as the relative value ΔcHb of the hemoglobin concentration is positive and the absolute value of the relative value ΔcHb of the hemoglobin concentration is increased. On the other hand, the ischemia index value is increased as the relative value ΔcHb of the hemoglobin concentration is negative and the absolute value of the relative value ΔcHb of the hemoglobin concentration is increased.

[0108] In the above embodiment, the hardware structure of the processing units that execute various processes, such as the central control unit 56, the image acquisition unit 57, the image processing unit 58, and the display control unit 59, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit (Graphical Processing Unit: GPU), which is a processor having a circuit configuration designed exclusively for executing various processes.

[0109] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a GPU and a CPU). Also, multiple processing units may be configured with one processor. Examples of configuring multiple processing units with one processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as represented by computers such as clients and servers, and this processor functions as multiple processing units. Second, a form in which the functions of an entire system including multiple processing units are integrated into one processor, as represented by a system on chip (SoC). In one embodiment, a processor implemented by an IC (Integrated Circuit) chip is used. In this manner, the various processing units are configured as a hardware structure using one or more of the various processors described above.

[0110] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements. [Explanation of symbols]

[0111] 10 Endoscope System 12 Display device 14 Input Devices 20 Lumenal endoscope system 21 Luminal light source device 22 Luminal endoscope device 23 Lumen processor device 24a Luminal endoscope operation part 24b Endoscope insertion part for lumen 25 Scope tip 26 Forceps Channel 26a Forceps entrance 26b Forceps exit 27 Curved section 28a Mode switching section 28b Reference value calculation instruction section 29 Angle knob 30 Laparoscopic system 31 Light source device for abdominal cavity 32 Laparoscopic device 33 Peritoneal processor 34 Laparoscopic insertion part 35 Endoscope tip for abdominal cavity 36 Foot Switch 37 Insufflation device 38 Treatment tools 39a, 39b Trakar 40 patients 40a Intraperitoneal 41 Light source 41a BS-LED 41b BL-LED 41c G-LED 41d R-LED 42 Light source control unit 43 Light Guide 44a Illumination optical system 44b Imaging optical system 45 Lighting Lens 46 Objective Lens 48 Image Sensor 49 Imaging surface 51 pixels 52 B color filter 53 G color filter 54 R color filter 55 Imaging control section 56 Central Control Unit 57 Image acquisition section 58 Image Processing Section 59 Display control section 59a Display image selection section 61 DSP 62 Noise reduction section 63 Conversion unit 71 Normal image processing section 72 Functional Image Processing Unit 73 Measurement value calculation section 74 Reference value calculation instruction reception unit 75 Relative value calculation section 76 Image Generation Unit 81 Signal ratio calculation section 82 Correlation memory section 83 Actual value calculation section 84 Reference value calculation unit 85, 86, 91 Graphs 91a Extinction coefficient of oxygenated hemoglobin 91b Absorption coefficient of reduced hemoglobin 93 Lower limit 94 Upper limit 95a Cr 95b Cb 95c brightness value 96 Color Table 97 Color Table 98 Luminance Table 99 Color Table 101 Hemoglobin concentration image 102 Oxygen Saturation Image 103 Judgment index image X direction Row direction Y direction Column direction ST100~ST210 Step

Claims

1. a measurement value calculation unit that calculates a first actual measurement value of a concentration of hemoglobin contained in the observation target and a first actual measurement value of an oxygen saturation level of hemoglobin contained in the observation target based on the plurality of first spectral images; a reference value calculation unit that calculates the reference value of the concentration and the reference value of the oxygen saturation based on a plurality of second spectral images in a reference value calculation mode; a relative value calculation unit that calculates a relative value of the concentration and a relative value of the oxygen saturation based on the first actual measurement value of the concentration and the reference value of the concentration, as well as the first actual measurement value of the oxygen saturation and the reference value of the oxygen saturation in an ischemia determination mode; an image generating unit that generates a relative value image by visualizing the relative value of the concentration and / or the relative value of the oxygen saturation; A display unit that displays the relative value image; a mode switching unit that receives an instruction to switch to the ischemia determination mode, An endoscopic system that automatically switches to the reference value calculation mode when the mode switching unit receives an instruction to switch to the ischemia determination mode, and switches back to the ischemia determination mode after calculating the reference value.

2. 2. The endoscopic system according to claim 1, wherein the reference value calculation unit calculates the reference value of the concentration or the reference value of the oxygen saturation by averaging the second actual measured values ​​of the concentration calculated for each pixel of the plurality of second spectral images or the second actual measured values ​​of the oxygen saturation calculated for each pixel of the plurality of second spectral images, respectively.

3. The measurement value calculation unit a signal ratio calculation unit that calculates a signal ratio having a dependency on the concentration based on a plurality of the first spectral images or a plurality of the second spectral images; a correlation storage unit that stores a correlation between the concentration and the signal ratio; an actual measurement value calculation unit that calculates the first actual measurement value or the second actual measurement value of the concentration corresponding to the signal ratio based on the correlation; The endoscope system according to claim 1 or 2, comprising:

4. 4. The endoscope system according to claim 1, wherein the first spectral image is an image obtained by imaging an observation target for evaluating a congestion or ischemia state, and the second spectral image is an image obtained by imaging an observation target different from the observation target for evaluating a congestion or ischemia state.

5. An endoscopic system as described in any one of claims 1 to 4, wherein the relative value image includes a hemoglobin concentration image that visualizes the relative value of the concentration, an oxygen saturation image that visualizes the relative value of the oxygen saturation, and a judgment index image that visualizes the relative value of the concentration and the relative value of the oxygen saturation.

6. The endoscopic system of claim 5, wherein the image generating unit generates the hemoglobin concentration image by pseudo-color imaging the relative value of the concentration using the color table for concentration, which stores pseudo-color information that changes depending on the concentration, and generates the oxygen saturation image by pseudo-color imaging the relative value of the oxygen saturation using the color table for oxygen saturation, which stores pseudo-color information that changes depending on the oxygen saturation.

7. The endoscope system according to claim 5 , wherein the image generating unit generates the determination index image by assigning the relative value of the concentration to a luminance channel and the relative value of the oxygen saturation to two color difference channels.

8. a display image selection unit that accepts a selection of a display image; The endoscope system according to claim 5 , wherein the display unit displays at least one of the hemoglobin concentration image, the oxygen saturation image, and the determination index image in accordance with the selection accepted by the display image selection unit.

9. The endoscope system according to claim 8 , wherein the display unit displays both the hemoglobin concentration image and the oxygen saturation image on the same screen.

10. An endoscopic system as described in any one of claims 1 to 9, further comprising an index value calculation unit that calculates a congestion index value indicating the degree of congestion in the object of observation and / or an ischemia index value indicating the degree of ischemia in the object of observation based on the relative value of the concentration and / or the relative value of the oxygen saturation.

11. an image acquisition unit that receives a plurality of first spectral images or a plurality of second spectral images from an endoscope device that acquires the plurality of first spectral images or the plurality of second spectral images; a measurement value calculation unit that calculates a first actual measurement value of a concentration of hemoglobin contained in the observation target and a first actual measurement value of an oxygen saturation level of the hemoglobin contained in the observation target based on the plurality of first spectral images; a reference value calculation unit that calculates a reference value of the concentration and a reference value of the oxygen saturation based on a plurality of the second spectral images in a reference value calculation mode; a relative value calculation unit that calculates a relative value of the concentration and a relative value of the oxygen saturation based on the first actual measurement value of the concentration and the reference value of the concentration, as well as the first actual measurement value of the oxygen saturation and the reference value of the oxygen saturation in an ischemia determination mode; an image generating unit that generates a relative value image by visualizing the relative value of the concentration and / or the relative value of the oxygen saturation; A display unit that displays the relative value image; a mode switching unit that receives an instruction to switch to the ischemia determination mode, a processor device that automatically switches to the reference value calculation mode when the mode switching unit receives an instruction to switch to the ischemia determination mode, and switches back to the ischemia determination mode after calculating the reference value.

12. an actual measurement value calculation step in which the measurement value calculation unit calculates a first actual measurement value of a concentration of hemoglobin contained in the observation target and a first actual measurement value of an oxygen saturation of hemoglobin contained in the observation target based on the plurality of first spectral images; a reference value calculation step of calculating, in a reference value calculation mode, a reference value of the concentration and a reference value of the oxygen saturation based on a plurality of second spectral images by a reference value calculation unit; a relative value calculation step in which a relative value calculation unit calculates, in an ischemia determination mode, a relative value of the concentration and a relative value of the oxygen saturation based on the first actual measurement value of the concentration and the reference value of the concentration, and the first actual measurement value of the oxygen saturation and the reference value of the oxygen saturation; an image generating step of generating a relative value image by an image generating unit, the relative value of the concentration and / or the relative value of the oxygen saturation being visualized; a display step in which a display unit displays the relative value image; a mode switching step of a mode switching unit receiving an instruction to switch to the ischemia determination mode, An operation method for an endoscopic system in which, when the mode switching unit receives an instruction to switch to the ischemia determination mode, the mode switching unit automatically switches to the reference value calculation mode, and switches to the ischemia determination mode after calculating the reference value.

Citation Information

Patent Citations

  • Electronic endoscope system, an electronic endoscope processor, and method of acquiring blood vessel information

    JP2011194028A

  • Endoscope system, processor device in endoscope system, and image display method

    JP2013099464A

  • Endoscope system and actuation method for endoscope system

    JP2014076375A

  • Endoscope system

    JP2014094088A

  • Calculation system

    JP2017148392A