Endoscope system, operation method thereof, and operation program of endoscope system

Abstract

To provide an endoscope system capable of grasping the degree of progress of congestion by visualizing the amount of blood by special light imaging and continuously recording time-varying images from a certain reference point, and to provide an operation method of the endoscope system and an operation program of the endoscope system.SOLUTION: In a blood volume mode 55, a blood volume image 81 or at least one of a blood volume and an oxygen saturation graph 88 at each time is displayed on an extended display 18. An image acquisition unit 58 acquires a reference point image 67 and a blood amount mode image 62 at a time interval set by the user. A calculation unit 59 calculates the average values of the blood volume and the oxygen saturation on the basis of pixels included in a region of interest 66. A time-series image generation unit 60 generates a blood volume image 81 indicating the amount of change in the blood volume. A time change graph generation unit 61 generates a blood volume and an oxygen saturation graph 88 at each time when an image is acquired.SELECTED DRAWING: Figure 24

Description

Technical Field

[0001] The present invention relates to an endoscope system having an oxygen saturation imaging function, a method of operating the same, and an operation program for the endoscope system.

Background Art

[0002] In surgery, for example, when removing cancer, not only the main lesion but also blood vessels, lymph nodes, etc. that may be infiltrated or metastasized are removed. Therefore, when the cancer is dissected, a part of the blood vessel network that nourishes the organ itself is also lost, and the hemodynamics of the reconstructed organ tend to become unstable. In particular, if the blood flow at the anastomosis is poor, it may cause suture dehiscence, which is a serious complication after surgery, and it is necessary to evaluate the blood perfusion of the reconstructed organ during the operation.

[0003] Especially in the surgical field, congestion is widely recognized as an effective index for preventing suture dehiscence, and it is generally important to grasp and avoid this state. An endoscope system having a function of calculating the amount of hemoglobin as the amount of dye and emphasizing and displaying the concentration distribution is known (Patent Document 1). In addition, an endoscope system having a function of measuring the blood volume and oxygen saturation in an arbitrary region of interest is known (Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The amount of blood in the state of congestion is considered to change over time from immediately after the treatment of blood vessels. Therefore, it is required not only to confirm the state of congestion at a certain point in time (such as immediately after blood vessel treatment), but also to confirm the change in the state of blood stasis over time. However, when evaluating blood perfusion as described above, it was difficult to confirm the temporal transition of the state of congestion during the operation.

[0006] An object of the present invention is to provide an endoscope system, an operating method thereof, and an operating program for the endoscope system, which can visualize the amount of blood by special light imaging and grasp the degree of progression of congestion by continuously recording time-varying images from a certain reference point.

Means for Solving the Problems

[0007] The endoscope system of the present invention includes a processor. The processor performs an image acquisition process of acquiring images of an observation target at time intervals set in advance by a user, performs a calculation process of calculating the amount of blood and oxygen saturation for each image acquired in the image acquisition process, generates an image showing the change amount of the amount of blood in each image acquired in the calculation process based on the amount of blood at the time set by the user as a reference, generates a graph showing the temporal changes in the amount of blood and oxygen saturation in each image acquired in the calculation process, and displays at least one of the image showing the change amount of the amount of blood or the graph showing the temporal changes in the amount of blood and oxygen saturation on a display.

[0008] Preferably, in the image acquisition process, the processor performs reference point imaging in which a reference observation target is set as a region of interest.

[0009] Preferably, the processor calculates the average values of the amount of blood and oxygen saturation based on the pixels included in the region of interest, and performs control to set the calculated average values as the reference values of the amount of blood and oxygen saturation.

[0010] Preferably, the processor performs a subtraction process of subtracting the reference value of the amount of blood from the amount of blood calculated in the calculation process.

[0011] The processor preferably generates an image that overlays an image with color maps of different shades according to the amount of change, based on the blood volume calculated by subtraction processing.

[0012] The processor preferably generates an image that overlays an image with different colors for the minus-side change and the plus-side change with respect to the amount of change, based on the blood volume calculated by subtraction processing.

[0013] In the display of a graph, for each image acquired by image acquisition processing, the processor preferably performs control to generate a graph in which the calculated blood volume and oxygen saturation are plotted by performing calculation processing based on pixels within the same region as the observation target set in the region of interest.

[0014] The processor preferably performs control to thumbnail the image acquired by image acquisition processing and perform parallel display processing in time series order on the display.

[0015] The processor preferably performs control to display a scroll bar for scrolling the images that have been subjected to parallel display processing on the display.

[0016] In the graph, the processor preferably performs control to enable the user to change the display or non-display state.

[0017] The processor preferably performs control to perform pattern matching based on the blood vessel shape of the observation target in reference point imaging and perform alignment when the endoscope or the observation target moves.

[0018] When the processor sets an observation target affected by disturbance as the region of interest in reference point imaging, it preferably performs control to notify the user of operation guidance for re-setting the region of interest.

[0019] In an operating method of an endoscope system including a processor, the processor performs a step of executing an image acquisition process of acquiring an image obtained by photographing an observation target at a time interval set in advance by a user, a step of executing a calculation process of calculating a blood volume and an oxygen saturation for each image obtained by the image acquisition process, a step of generating an image showing a change amount of the blood volume of each image obtained by the calculation process based on the blood volume at a time set by the user, a step of generating a graph showing a temporal change of the blood volume and the oxygen saturation of each image obtained by the calculation process, and a step of displaying at least one of the image showing the change amount of the blood volume or the graph showing the temporal change of the blood volume and the oxygen saturation on a display.

[0020] In a program for an endoscope system including a processor, the program causes a computer to execute a function of performing an image acquisition process of acquiring an image obtained by photographing an observation target at a time interval set in advance by a user, a function of performing a calculation process of calculating a blood volume and an oxygen saturation for each image obtained by the image acquisition process, a function of generating an image showing a change amount of the blood volume of each image obtained by the calculation process based on the blood volume at a time set by the user, a function of generating a graph showing a temporal change of the blood volume and the oxygen saturation of each image obtained by the calculation process, and a function of displaying at least one of the image showing the change amount of the blood volume or the graph showing the temporal change of the blood volume and the oxygen saturation on a display.

Advantages of the Invention

[0021] According to the present invention, it is possible to visualize the blood volume by special light imaging and continuously record a time change image from a certain reference point, thereby making it possible to grasp the progress degree of congestion.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Mode for Carrying Out the Invention

[0023] As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically or electrically connected to the light source device 13 and electrically connected to the processor device 14. The extended processor device 17 is electrically connected to the light source device 13 and the processor device 14. Note that the “display” in the claims includes not only the display 15 but also the extended display 18.

[0024] The endoscope 12 has an insertion portion 12a, an operation portion 12b, a bending portion 12c, and a distal end portion 12d. The insertion portion 12a is inserted into the body of the subject. The operation portion 12b is provided at the proximal end portion of the insertion portion 12a. The bending portion 12c and the distal end portion 12d are provided on the distal end side of the insertion portion 12a. The distal end portion 12d is directed in a desired direction by the bending operation of the bending portion 12c. A forceps channel (not shown) for inserting a treatment instrument or the like is provided from the insertion portion 12a to the distal end portion 12d.

[0025] Inside the endoscope 12, an optical system for forming a subject image and an optical system for irradiating the subject with illumination light are provided. The operation portion 12b is provided with a mode switching switch 12e, a reference point image acquisition instruction switch 12f, and a zoom operation portion 12g. The mode switching switch 12e is used for switching the observation mode. The reference point image acquisition instruction switch 12f is used for instructing the acquisition of an image of a reference subject in a blood volume mode described later. The zoom operation portion 12g is used for enlarging or reducing the observation target.

[0026] The light source device 13 generates illumination light. The processor device 14 controls the system of the endoscope system 10 and further generates an image obtained by the endoscope (hereinafter referred to as an endoscope image) by performing image processing and the like on the image signal transmitted from the endoscope 12. The display 15 displays the medical image transmitted from the processor device 14. The user interface 16 has a keyboard, a mouse, a microphone, a tablet, a foot switch, a touch pen, etc., and accepts input operations such as function settings.

[0027] The endoscope system 10 has two modes: a normal mode and a blood volume mode, and these two modes can be switched by the user operating the mode switch 12e. As shown in FIG. 2, in the normal mode, a white light image with a natural color obtained by imaging the observation target using white light as the illumination light is displayed on the display 15, while nothing is displayed on the extended display 18.

[0028] As shown in FIG. 3, in the blood volume mode, the blood volume and oxygen saturation of the observation target are calculated at time intervals set in advance by the user, and the time changes of the calculated blood volume and oxygen saturation are displayed as a blood volume image on the extended display 18. Also, in the blood volume mode, a white light equivalent image NP2 with fewer short wavelength components than the white light image NP1 is displayed on the display 15.

[0029] Note that the endoscope system 10 is a flexible endoscope type for digestive organs such as the stomach and large intestine. In the blood volume mode, as shown in FIG. 4(A), an image of the oxygen saturation state inside the digestive tract, which is an image of the oxygen saturation inside the digestive tract, is displayed on the extended display 18. Also, in the case of a rigid endoscope type for the abdominal cavity such as the serosa, as shown in FIG. 4(B), an image of the oxygen saturation state on the serosa side, which is an image of the oxygen saturation on the serosa side, is displayed on the extended display 18. It is preferable to use an image obtained by adjusting the saturation with respect to the white light equivalent image for the serosa side oxygen saturation image. Regarding the adjustment of saturation, it is preferable to perform it in the blood volume mode without distinguishing between mucosa, serosa, flexible endoscope, and rigid endoscope.

[0030] In the blood volume mode, it is possible to accurately calculate the blood volume and oxygen saturation in the following cases. · When observing a predetermined target site (e.g., esophagus, stomach, large intestine) · When outside the extracorporeal environment with lighting around · When there is no residue, residual liquid, mucus, blood, or fat remaining on the mucosa and serosa · When no pigment is sprayed on the mucosa · When the endoscope 12 is more than 7 mm away from the observation site · When observing the observation site at an appropriate distance without the endoscope moving far away · An area where the illumination light is sufficiently incident · When there is little direct reflection light from the observation site · An area within 2 / 3 of the oxygen saturation image · When the movement of the endoscope is small, or when the movement of the patient such as pulsation and breathing is small · When the blood vessels deep in the gastrointestinal mucosa are not observed

[0031] The processor device 14 is electrically connected to the display 15 and the user interface 16. The processor device 14 receives the image signal from the endoscope 12 and performs various processes based on the image signal. The display 15 outputs and displays the image or information of the observation target processed by the processor device 14. The user interface 16 has a keyboard, mouse, touch pad, microphone, foot pedal, etc., and has a function of receiving input operations such as function settings.

[0032] As shown in FIG. 5, the light source device 13 includes a light source unit 20 and a light source processor 21 that controls the light source unit 20. The light source unit 20 has, for example, a plurality of semiconductor light sources, and lights or extinguishes each of them. When lighting, by controlling the light emission amount of each semiconductor light source, it emits illumination light for illuminating an observation object. In the present embodiment, the light source unit 20 has five-color LEDs, namely, a V-LED (Violet Light Emitting Diode) 20a, a BS-LED (Blue Short -wavelength Light Emitting Diode) 20b, a BL-LED (Blue Long-wavelength Light Emitting Diode) 20c, a G-LED (Green Light Emitting Diode) 20d, and an R-LED (Red Light Emitting Diode) 20e.

[0033] The V-LED 20a emits violet light V of 410 nm ± 10 nm. The BS-LED 20b emits second blue light BS of 450 nm ± 10 nm. The BL-LED 20c emits first blue light BL of 470 nm ± 10 nm. The G-LED 20d emits green light G in the green band. It is preferable that the center wavelength of the green light G is 550 nm. The R-LED 20e emits red light R in the red band. It is preferable that the center wavelength of the red light R is 620 nm. Note that the center wavelength and the peak wavelength in each of the LEDs 20a to 20e may be the same or different.

[0034] The light source processor 21 independently controls the lighting or extinguishing of each of the LEDs 20a to 20e, the light emission amount during lighting, etc. by inputting control signals to each of the LEDs 20a to 20e independently. The lighting or extinguishing control in the light source processor 21 differs depending on each mode.

[0035] The processor device 14 controls the light emission in the normal mode by the light source processor 21. As shown in FIG. 6, in the normal mode 54, the V-LED 20a, BS-LED 20b, G-LED 20d, and R-LED 20e are simultaneously lit to emit white light including purple light V with a central wavelength of 410 nm, second blue light BS with a central wavelength of 450 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm. The white light image NP1 obtained based on the white light is displayed on the display 15 (see FIG. 2).

[0036] The processor device 14 controls the light emission in the blood volume mode by the light source processor 21. As shown in FIG. 7, in the blood volume mode 55, the light emission of two frames with different emission patterns is repeated. In the first frame, the BL-LED 20c, G-LED 20d, and R-LED 20e are simultaneously lit to emit broadband first illumination light including first blue light BL with a central wavelength of 470 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm. In the second frame, the BS-LED 20b, G-LED 20d, and R-LED 20e are simultaneously lit to emit second illumination light including second blue light BS with a central wavelength of 450 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm. The white light equivalent image NP2 obtained based on the light emission of the second illumination light in the second frame is displayed on the display 15 (see FIG. 3). Further, the blood volume image OP obtained based on the light emission of the first illumination light in the first frame and the second illumination light in the second frame is displayed on the extended display 18.

[0037] The light emitted by each of the LEDs 20a to 20e is incident on the light guide 24 through an optical path coupling portion 23 composed of a mirror, a lens, etc. The light guide 24 is built into the endoscope 12 and a universal cord (a cord connecting the endoscope 12, the light source device 13, and the processor device 14). The light guide 24 propagates the light from the optical path coupling portion 23 to the tip 12d of the endoscope 12.

[0038] The distal end 12d of the endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and the illumination light propagated by the light guide 24 is irradiated onto the observation target through the illumination lens 32. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. The light from the observation target irradiated with the illumination light enters the imaging sensor 36 through the objective lens 35. Thereby, an image of the observation target is formed on the imaging sensor 36.

[0039] The imaging sensor 36 is a color imaging sensor that images the observation target being illuminated with the illumination light. Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. The spectral transmittance of the B color filter, the G color filter, and the R color filter will be described later. For example, the imaging sensor 36 is preferably a color imaging sensor with a Bayer array in which the ratio of the number of pixels of the B pixel, the G pixel, and the R pixel is 1:2:1.

[0040] The imaging sensor 36 is a color imaging sensor that images the observation target being illuminated with the illumination light. Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. The spectral transmittance of the B color filter, the G color filter, and the R color filter will be described later. For example, the imaging sensor 36 is preferably a color imaging sensor with a Bayer array in which the ratio of the number of pixels of the B pixel, the G pixel, and the R pixel is 1:2:1.

[0041] As the imaging sensor 36, a CCD (Charge Coupled Device) imaging sensor or a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor can be used. Also, instead of the primary color imaging sensor 36, a complementary color imaging sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. When using the complementary color imaging sensor, since four-color image signals of CMYG are output, by converting the four-color image signals of CMYG into three-color image signals of RGB through complementary color-primary color conversion, image signals of each RGB color similar to those of the imaging sensor 36 can be obtained.

[0042] The imaging sensor 36 is driven and controlled by the imaging processor 37. The CDS / AGC circuit 40 (Correlated Double Sampling / Automatic Gain Control) performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 36. The image signal that has passed through the CDS / AGC circuit 40 is converted into a digital image signal by the A / D converter 41 (Analog / Digital). The digital image signal after A / D conversion is input to the processor device 14.

[0043] In the normal mode 54, the imaging processor 37 controls the imaging sensor 36 to image the observation target being illuminated with purple light V, second blue light BS, green light G, and red light R for each frame. As a result, a Bc image signal is output from the B pixels of the imaging sensor 36, a Gc image signal is output from the G pixels, and an Rc image signal is output from the R pixels (see Fig. 6).

[0044] In the blood volume mode 55, when the imaging processor 37 illuminates the observation target with the first illumination light including the first blue light BL, green light G, and red light R in the first frame, as the first illumination light image, a B1 image signal is output from the B pixel of the imaging sensor 36, a G1 image signal is output from the G pixel, an R1 image signal is output from the R pixel, and a B1 image, a G1 image, and an R1 image are obtained. In the second frame, when the observation target is illuminated with the second illumination light including the second blue light BS, green light G, and red light R, as the second illumination light image, a B2 image signal is output from the B pixel of the imaging sensor 36, a G2 image signal is output from the G pixel, an R2 image signal is output from the R pixel, and control is performed so that a B2 image, a G2 image, and an R2 image are obtained (see FIG. 7).

[0045] The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. Programs related to various processes are incorporated into a program memory (not shown) in the processor device 14. By the central control unit 53 constituted by the processor executing the programs in the program memory, the functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized.

[0046] The DSP 45 performs various signal processes such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, white balance processing, YC conversion processing, and noise reduction processing on the image signal received from the endoscope 12. In the defect correction processing, the signal of the defective pixel of the imaging sensor 36 is corrected. In the offset processing, the dark current component is removed from the image signal subjected to the defect correction processing, and an accurate zero level is set. The gain correction processing adjusts the signal level of each image signal by multiplying the image signals of each color after the offset processing by a specific gain. Linear matrix processing for enhancing color reproducibility is performed on the image signals of each color after the gain correction processing.

[0047] Thereafter, the brightness and saturation of each image signal are adjusted by gamma conversion processing. The image signal after linear matrix processing is subjected to demosaicing processing (also called isotropic processing or synchronization processing), and signals of missing colors of each pixel are generated by interpolation. By the demosaicing processing, all pixels come to have signals of RGB colors. DSP45 performs YC conversion processing on each image signal after demosaicing processing, and outputs the luminance signal Y, the color difference signal Cb, and the color difference signal Cr to DSP45. DSP45 performs noise reduction processing, such as by a moving average method or a median filter method, on the image signal subjected to demosaicing processing and the like.

[0048] The image processing unit 50 performs various image processes on the image signal from DSP45. The image processes include 3×3 matrix processing, gradation conversion processing, color conversion processing such as 3D LUT (Look Up Table) processing, color enhancement processing, structure enhancement processing such as spatial frequency enhancement, and the like. In the image processing unit 50, image processing according to the mode is performed. In the case of the normal mode 54, the image processing unit 50 generates a white light image by performing the image processing for the normal mode. In the case of the blood volume mode, the image processing unit 50 generates an image equivalent to white light by performing the image processing for oxygen saturation. Also, in the case of the blood volume mode, the image processing unit 50 transmits the image signal from DSP45 to the extended processor device 17 via the image communication unit 51.

[0049] The display control unit 52 performs display control for displaying image information such as the white light image NP1 or the blood volume image BP and other information from the image processing unit 50 on the display 15. According to the display control, the white light image NP1 or the image NP2 equivalent to white light is displayed on the display 15.

[0050] The extended processor device 17 receives an image signal from the processor device 14 and performs various image processes. In the blood volume mode 55, the extended processor device 17 calculates the blood volume and oxygen saturation, and generates a blood volume image BP in which the calculated blood volume and oxygen saturation are imaged. The generated blood volume image BP is an image showing the change amount of the calculated blood volume and / or a graph showing the temporal change of the calculated blood volume and oxygen saturation. The generated blood volume image BP is displayed on the extended display 18. Further, when starting the blood volume mode 55, the extended processor device 17 sets a reference observation target as a region of interest by a user operation and performs reference point imaging to obtain reference values for calculating the blood volume and oxygen saturation. Details of the blood volume mode 55 performed by the extended processor device 17 will be described later.

[0051] As shown in FIG. 8, the extended processor device 17 includes a blood volume image processing unit 56 and a blood volume image display unit 57. The blood volume image processing unit 56 performs image processing for generating the blood volume image BP. The blood volume image display unit 57 performs control such as displaying the generated blood volume image BP on the extended display 18.

[0052] The switching from the normal mode 54 to the blood volume mode 55 may be performed according to the operation of the mode switching switch 12e by the user, or may be automatically switched according to the conditions set by the user. The blood volume image processing unit 56 switches the light emission pattern of the light source processor 21 and the transmission destination of the image signal according to the mode switching by the user's operation or setting.

[0053] The blood volume mode will be described below. In the case of the blood volume mode, as shown in FIG. 9, the blood volume image processing unit 56 includes an image acquisition unit 58, a calculation unit 59, a time-series image generation unit 60, and a time change graph generation unit 61. The image acquisition unit 58 acquires a first illumination light image by the first illumination light and a second illumination light image by the second illumination light sent from the processor device 14 in the blood volume mode 55.

[0054] The image acquisition unit 58 performs an image acquisition process of acquiring the images captured by the endoscope 12 of the observation target at time intervals set in advance by the user. The user sets the time interval for automatically acquiring the captured images at regular time intervals. Note that the setting of the time interval may be set or changed at any timing.

[0055] The image acquisition unit 58 performs reference point imaging for setting a reference observation target as the region of interest. For example, as shown in FIG. 10(A), in the blood volume mode image 62 which is either the first illumination light image or the second illumination light image acquired by the image acquisition unit 58, the observation target 63, the congestion site 64, and the region of interest target 65 are imaged. Note that the blood volume mode image 62 is an example of the endoscope image acquired by the image acquisition unit 58. Also, the region of interest target 65 is preferably indicated by a thin solid line. Also, the shape of the region of interest target 65 is not limited to a circle, and may be changed to a rectangle or the like, or the size may be changed.

[0056] As shown in FIG. 10(B), the image acquisition unit 58 performs control to acquire a reference point image 67 for setting the region including the congestion site 64 within the observation target 63 shown in the blood volume mode image 62 as the region of interest 66. For example, the user fixes the endoscope at a position where at least a part, more preferably all, of the congestion site 64 is within the region of interest target 65, and performs reference point imaging by pressing the reference point image acquisition instruction switch 12f to acquire the reference point image 67.

[0057] The calculation unit 59 performs a calculation process of calculating the blood volume and oxygen saturation for each image acquired by the image acquisition unit 58. Specifically, for each pixel included in the region of interest 66 (see FIG. 10(B)) shown in the reference point image 67 acquired by the image acquisition unit 58, the blood volume and oxygen saturation are calculated, and a calculation process is performed in which the average value within the region of interest 66 is used as the reference value for the blood volume and oxygen saturation. Note that the pixels used for the calculation may be all the pixels included in the region of interest 66, or a part of the pixels included in the region of interest 66, such as every other pixel or every two pixels. Also, the reference value is used when calculating the change amount of the blood volume of the observation target.

[0058] The above calculation process for blood volume is calculated for the pixels included in the image using the signal ratio in the normalized long-wavelength region. Specifically, as shown in FIG. 11(A), for the absorption coefficients at the respective wavelengths of oxyhemoglobin 68 and deoxyhemoglobin 69, as shown in FIG. 11(B), the reflected light 72 near 620 nm is normalized by the reflected light 71 near 550 nm, and the blood volume is calculated using the signal ratio 74 in the long-wavelength region. Further, since the signal ratio 74 in the long-wavelength region is affected by a certain amount by changes in oxygen saturation, an operation for invalidating the influence is performed. Specifically, the oxygen saturation is calculated using the signal ratio 73 in the short-wavelength region obtained by normalizing the reflected light 70 near 470 nm by the reflected light 71 near 550 nm. In FIG. 11(B), LN represents the natural logarithm.

[0059] The calculation unit 59 calculates the blood volume using the signal ratio 74 in the long-wavelength region, the signal ratio 73 in the short-wavelength region, and a look-up table (hereinafter referred to as LUT) in the first illumination light image and the second illumination light image (see FIG. 7) acquired by the image acquisition unit 58. For example, the blood volume is calculated based on the LUT 75 shown in FIG. 12. The LUT 75 is composed of the change direction 76 of oxygen saturation and the change direction 77 of blood volume perpendicular to the change 76 of oxygen saturation. The change direction 76 of oxygen saturation is represented by the contour line EL. The contour line ELL represents that the oxygen saturation is 0%, and the contour line ELH represents that the oxygen saturation is 100%. The change direction 77 of blood volume is given a certain amount of inclination in the signal ratio LN(R2 / G2) in the long-wavelength region and made orthogonal to the change direction of the change 76 of oxygen saturation, thereby invalidating the influence of oxygen saturation in the signal ratio LN(B1 / G2) and calculating so as to be affected only by the blood volume. In FIG. 12, LN represents the natural logarithm. Further, the change amount of blood volume is calculated by the following formula (1), and α indicates the inclination for the change direction 77 of blood volume to be perpendicular to the change direction 76 of oxygen saturation.

[0060] Change amount of blood volume (HBI) = X + αY (1)

[0061] The calculation unit 59 calculates the oxygen saturation of the observation target using the first illumination light image and the second illumination light image acquired by the image acquisition unit 58 and the LUT (see FIG. 7). Specifically, the calculation unit 59 calculates, for each pixel, the ratio LN(B1 / G2) between the B1 image based on the B1 image signal and the G2 image based on the G2 image signal, and the ratio LN(R2 / G2) between the R2 image and the G2 image. These ratios LN(B1 / G2) and LN(R2 / G2) are reference values calculated by the calculation unit 59 using the reference point image, and are the calculation results of operations using the pixel values within the region of interest in the reference point image. The ratio LN(B1 / G2) mainly affects the oxygen saturation, and the ratio LN(R2 / G2) mainly affects the blood volume. Therefore, by examining the balance between the ratio LN(B1 / G2) and the ratio LN(R2 / G2), it is possible to obtain the oxygen saturation of the observation target, excluding the influence on the blood volume.

[0062] Note that the ratios LN(B1 / G2) and LN(R2 / G2) hardly ever become extremely large values or, conversely, extremely small values. That is, the combination of the ratios LN(B1 / G2) and LN(R2 / G2) hardly ever exceeds the upper equivalent line representing the oxygen saturation of "100%" or falls below the lower equivalent line representing the oxygen saturation of "0%". When the oxygen saturation exceeds 100%, the calculation unit 59 sets the oxygen saturation to 100%, and when the oxygen saturation is less than 0%, the calculation unit 59 sets the oxygen saturation to 0%.

[0063] The calculation unit 59 performs a subtraction process of subtracting the reference value of the blood volume from the blood volume calculated based on the pixels included in the image. For example, as shown in FIG. 13(A), in the blood volume mode image 62, the calculation unit 59 calculates the blood volume based on the pixels 78 included in the blood volume mode image 62. Further, as shown in FIG. 13(B), the calculation unit 59 performs a subtraction process of subtracting the reference value of the blood volume calculated based on the pixels included in the region of interest 66 in the reference point image 67 from the blood volume calculated based on the pixels 78 included in the blood volume mode image 62. Therefore, the calculation unit 59 calculates the change amount of the blood volume between the reference value of the blood volume of the reference point image 67 and the blood volume of the observation target photographed by the blood volume mode image 62 through the subtraction process.

[0064] The time-series image generation unit 60 generates an image showing the change amount of the blood volume in the blood volume mode image 62, which is obtained by the subtraction process in the calculation unit 59, based on the blood volume at the time set by the user. The generation of the image is performed at a fixed time interval set by the image acquisition unit 58. Also, the time when the user sets as a reference is based on the time of acquisition of the reference point image 67.

[0065] As shown in FIG. 14, for the image showing the change amount of the blood volume, when the image acquisition unit 58 acquires the reference point image 67 and the blood volume mode image 62a, the calculation unit 59 performs the subtraction process 79, and the change amount 80a of the blood volume is calculated. Further, the time-series image generation unit 60 generates the blood volume image 81a using the blood volume mode image 62a and the calculated change amount 80a of the blood volume. Also, when the user sets a fixed time interval, the image acquisition unit 58 re-acquires the blood volume mode image 62b, the calculation unit 59 performs the subtraction process 79 using the reference point image 67 to calculate the change amount 80b of the blood volume, and the time-series image generation unit 60 generates the blood volume image 81b using the blood volume mode image 62b and the calculated change amount 80b of the blood volume. Thereafter, the series of processes are repeated.

[0066] It is preferable that the time-series image generation unit 60 performs a process of assigning color maps with different shades according to the change amount based on the blood volume calculated by the subtraction process and generating an image superimposed on the white light equivalent image NP2. For example, as shown in FIG. 15(A), in the observation target 63 photographed by the blood volume mode image 62, the time-series image generation unit 60 sets the congestion site 64 as the region of interest 66, and when the reference point photographing and the subtraction process are performed, as shown in FIG. 15(B), different color maps 80 with different shades are assigned to the blood volume after the subtraction process according to the change amount, and the blood volume image 81 may be generated by superimposing it on the blood volume mode image 62.

[0067] Further, the time-series image generation unit 60 preferably generates an image in which, based on the blood volume calculated by the subtraction process, the negative-side change and the positive-side change with respect to the change amount are represented in different colors and superimposed on the white-light equivalent image NP2. For example, in the above-described image 62 for the blood volume mode (see FIG. 15(A)), as shown in FIG. 16(A), a blood volume image 81 in which the negative-side change 82 and the positive-side change 83 are superimposed on the image 62 for the blood volume mode with different colors of blood volume superimposed may be generated. Note that the colors representing the negative-side change and the positive-side change may be specified by the user, for example, the negative-side change is blue and the positive-side change is green.

[0068] Furthermore, the time-series image generation unit 60 may generate an image in which the negative-side change and the positive-side change are represented in different colors with respect to the above-described color maps with different shades. For example, in the above-described color maps 80 with different shades (see FIG. 15(B)), as shown in FIG. 16(B), a blood volume image 81 in which the negative-side changes 84 with different shades and the positive-side changes 85 with different shades are represented in different colors on the color maps with different shades and the color maps with different shades and different colors are superimposed may be generated.

[0069] The time change graph generation unit 61 generates a graph showing the time changes in the blood volume and oxygen saturation of the image 62 for the blood volume mode calculated by the calculation unit 59. In the display of the graph, the time change graph generation unit 61 performs a calculation process based on the pixels in the same region as the observation target set in the region of interest for each image acquired by the image acquisition unit 58, plots the calculated blood volume and oxygen saturation on the graph, and controls the generation of the graph. For example, as shown in FIG. 17(A), in the pixels included in the region of interest 66 of the images 62a, 62b, 62c, and 62d for the blood volume mode acquired by the image acquisition unit 58 at regular time intervals ta, tb, tc, and td, the calculation unit 59 calculates the average value 86 of the blood volume and the average value 87 of the oxygen saturation in the region of interest 66 of each blood volume image.

[0070] The average value 86 of the blood volume and the average value 87 of the oxygen saturation of each image for the calculated blood volume are, for example, as shown in FIG. 17(B). When the time change graph generation unit 61 generates the blood volume and oxygen saturation graphs 88 at each time, the average value of each image for the blood volume obtained by the above calculation is automatically plotted for each time when the image is acquired. Since the user re-acquires the image at a fixed time interval set by the user, and the average value 86 of the blood volume and the average value 87 of the oxygen saturation in the region of interest 66 of the image are plotted, the time change of the blood volume and oxygen saturation in the region of interest 66 of the observation target can be quantitatively grasped.

[0071] The blood volume image display unit 57 displays on the display at least one of an image showing the change amount of the blood volume or the blood volume and oxygen saturation graphs 88 at each time. For example, as shown in FIG. 18(A), the blood volume image display unit 57 displays on the extended display 18 the region of interest target 65, the blood volume image 81, the blood volume and oxygen saturation graphs 88 at each time, and the region of interest indication line 92. The region of interest target 65 and the region of interest indication line 92 indicate the region used for the calculation of the blood volume and oxygen saturation graphs 88 at each time. Also, when only the graph is displayed, for example, as shown in FIG. 18(B), on the extended display 18, the region of interest target 65, the image 62 for the blood volume mode, the blood volume and oxygen saturation graphs 88 at each time, and the region of interest indication line 92 are displayed, and the blood volume image 81 with the change amount of the blood volume superimposed is made non-displayed.

[0072] When making the blood volume image 81 non-displayed, the blood volume image display unit 57 displays the image 62 for the blood volume mode instead of the blood volume image 81. Therefore, when the graph is displayed, the region of interest target 65 and the region of interest indication line 92 of the observation target are always displayed, and the blood volume and oxygen saturation graphs 88 at each time can show the user which region the calculation is based on.

[0073] In the graph, the blood volume image display unit 57 performs control that enables the user to change the display or non-display state. For example, as shown in FIG. 19, the blood volume image display unit 57 may display only the blood volume image 81 on the extended display 18. When the graph is not displayed, the region of interest target 65 is not displayed, and the visibility of the image indicating the change amount of the blood volume (for example, the above-described color map) is further improved.

[0074] Hereinafter, examples using the above configuration will be given.

[0075] [Example 1] By performing reference point imaging at a normal part, it is confirmed how much more the blood volume of the observation target is compared to the normal part. For example, as shown in FIG. 20(A), the image acquisition unit 58 sets the normal part in the observation target 63 as the region of interest 66, performs reference point imaging to acquire the reference point image 67, and further acquires the blood volume mode image 62. When the calculation unit 59 performs subtraction processing on the normal part set in the region of interest 66, if the suspicious part 89 shown in the generated blood volume image 81 is in the congestion state 90a, as shown in FIG. 20(B), regions 91a and 91b where the change amount of the blood volume is larger than that of the region of interest 66 are represented around the suspicious part 89 shown in the blood volume image 81, and are superimposed on the white light equivalent image NP2 and displayed on the extended display 18. Also, in the case of the state 90b where the difference in blood volume between the suspicious part 89 and the region of interest 66 is small, as shown in FIG. 20(C), for the suspicious part 89 shown in the blood volume image 81, the region indicating the change amount of the blood volume from the region of interest 66 is not displayed on the extended display 18. In FIGS. 20(B) and 20(C), the region of interest 66 is marked for the purpose of explanation in the figure and is not actually displayed on the extended display 18. Also, the acquisition of the blood volume mode image can be performed by the user at an arbitrary timing after the reference point imaging is performed.

[0076] The flow of a series of processes in Example 1 by the endoscope system 10 will be described with reference to the flowchart of FIG. 21. By automatically or the user operating the mode switch 12e, the blood volume mode is switched (step ST100). Using the image taken in the blood volume mode, with the reference observation target included in the region of interest target 65, the reference point image acquisition instruction switch 12f is pressed to perform reference point imaging and acquire the reference point image 67 (step ST110). Based on the pixels included in the region of interest 66 shown in the reference point image 67, a calculation process is performed to calculate the average values of the blood volume and oxygen saturation as reference values (step ST120). After the reference point imaging, an image for the blood volume mode is acquired at an arbitrary timing (step ST130). A subtraction process is performed to subtract the reference value of the blood volume obtained in the calculation process from the blood volume calculated based on the pixels included in the entire image for the blood volume mode (step ST140). The amount of change in the blood volume by the subtraction process is superimposed on the white light equivalent image NP2 to generate a blood volume image (step ST150). The generated blood volume image is displayed on the extended display 18 (step ST160).

[0077] With the above configuration, the endoscope system 10 can use the region of interest 66 as a reference in displaying the amount of change in the blood volume, so that the user can specify the observation target and relatively visually recognize the amount of change in the blood volume based on the observation target. Then, it is possible to easily determine whether the observation target is in a congested state.

[0078] [Example 2] Perform reference point imaging to set the area where congestion is confirmed as the region of interest, acquire images at time intervals set by the user, and calculate the blood volume and oxygen saturation for each image. For example, as shown in Fig. 22(A), the image acquisition unit 58 sets the congested area 64 in the observation target 63 as the region of interest 66, performs reference point imaging to acquire a reference point image 67, and further acquires an image 62 for the blood volume mode. When the calculation unit 59 performs a subtraction process on the congested area 64 set in the region of interest 66, as shown in Fig. 22(B), the time-series image generation unit 60 superimposes the positive change 83 in blood volume on the image 62 for the blood volume mode to generate a blood volume image 81. Further, the time change graph generation unit 61 plots the average value 86 of blood volume and the average value 87 of oxygen saturation to generate a graph 88 of blood volume and oxygen saturation at each time, which is displayed on the extended display 18 by the blood volume image display unit 57. When the user sets to acquire images at regular time intervals, as shown in Fig. 22(C), for each image re-acquired by the image acquisition unit 58 at regular time intervals, the time-series image generation unit 60 superimposes the positive change 83 in blood volume on the image 62 for the blood volume mode to generate a blood volume image 81. Further, the time change graph generation unit 61 plots the average value 86 of blood volume and the average value 87 of oxygen saturation to generate a graph 88 of blood volume and oxygen saturation at each time, which is displayed on the extended display 18 by the blood volume image display unit 57. Note that the display of the change amount of blood volume is not limited to the positive change 83, and the user may set it using the negative change 82, a color map with different shades, etc., to make it easier to visually recognize the change in the congestion state.

[0079] The flow of a series of processes in Example 2 by the endoscope system 10 will be described with reference to the flowchart of FIG. 23. Note that since steps ST200 to ST250 perform the same processes as steps ST100 to ST150 shown in FIG. 21, the description thereof will be omitted. In Example 2, further, a graph plotting the average values of the blood volume and oxygen saturation is generated (step ST260), and the generated blood volume image and graph are displayed on the extended display 18 (step ST270). When the user sets to re-acquire images at regular time intervals (Y in step ST280), the blood volume mode image 62 is re-acquired at the set time intervals (step ST230), and the procedure from the subtraction process (step ST240) of subtracting the reference value of the blood volume obtained by the calculation process from the blood volume calculated based on the pixels included in the entire blood volume mode image is performed again. Also, when the setting to re-acquire images is not implemented or when the setting is changed to cancel the re-acquisition of images, (N in step ST280) the blood volume mode image is not re-acquired, and the blood volume image and graph displayed on the extended display 18 are not updated.

[0080] With the above configuration, the endoscope system 10 can grasp the passage of time based on the congested site, so it is possible to confirm whether the congested state is progressing over time by limiting the site. Then, by displaying graphs showing the blood volume and oxygen saturation at each time in parallel, the oxygen saturation can be grasped simultaneously with the blood volume, and it can be determined whether the reference site is simply a site with a large blood volume or a congested state where a large amount of blood is present and oxygen is also consumed.

[0081] [Example 3] When a surgical procedure is performed to release the stagnant blood to another route by performing a vascular procedure on the site where congestion has been confirmed, it is confirmed that the stagnant blood is decreasing over time. Specifically, reference point imaging is performed based on the congested site before the vascular procedure, and an operation of observing the same region over time after the vascular procedure of the congested site is performed.

[0082] For example, as shown in FIG. 24(A), before the vascular treatment, the user uses the image acquisition unit 58 to set the congestion site 64 in the observation target 63 as the region of interest 66, and performs reference point imaging to acquire a reference point image 67. After the user acquires the reference point image 67, the user performs a vascular treatment 93 on the congestion site 64. After the implementation of the vascular treatment 93, the user sets to acquire images at regular time intervals. As shown in FIG. 24(B), when the image acquisition unit 58 acquires the blood volume mode image 62 and the calculation unit 59 performs a subtraction process on the vascular treatment site 94 set in the region of interest 66, the time-series image generation unit 60 superimposes the negative change 82 in blood volume on the blood volume mode image 62 to generate a blood volume image 81. Further, the time change graph generation unit 61 plots the average value 86 of the blood volume and the average value 87 of the oxygen saturation, and generates a blood volume and oxygen saturation graph 88 for each time, which is displayed on the extended display 18 by the blood volume image display unit 57. Further, as shown in FIG. 24(C), at regular time intervals set by the user, the image acquisition unit 58 re-acquires the blood volume mode image 62 with the vascular treatment site 94 as a reference. Also, the time-series image generation unit 60 generates a blood volume image 81 by superimposing the negative change 82 in blood volume on the blood volume mode image 62 with the congestion site 64 where the reference point imaging was performed as a reference. Further, the time change graph generation unit 61 plots the average value 86 of the blood volume and the average value 87 of the oxygen saturation based on the re-acquired blood volume mode image 62, and generates a blood volume and oxygen saturation graph 88 for each time. The above blood volume image 81 and the blood volume and oxygen saturation graphs 88 for each time are displayed on the extended display 18 by the blood volume image display unit 57. Note that the display of the change amount of the blood volume is not limited to the negative change 82, and the user may set it using the positive change 83, a color map with different shades, etc., so as to make it easier to visually recognize the time change after the vascular treatment of the congestion site 64.

[0083] A series of processes in Example 3 by the endoscope system 10 are the same as those in Example 2 (see FIG. 23).

[0084] The endoscopic system 10, with the above configuration, can grasp the time course of the congestion state after vascular treatment based on the congestion site, so it is possible to confirm the decrease over time of the blood that had been retained at the congestion site. In addition, by displaying graphs showing the blood volume and oxygen saturation at each time in parallel, the rate of change can be quantitatively grasped from the gradient of the graphs. And according to the endoscopic system 10, the blood volume can be visualized by special light imaging, and the time-varying images from a certain reference point can be continuously recorded, so it is possible to grasp the progress of congestion based on the specified site.

[0085] Note that, as shown in FIG. 25, the blood volume image display unit 57 may include a parallel display processing unit 95. The parallel display processing unit 95 controls to thumbnail the image obtained by the image acquisition unit 58 through image acquisition processing and perform parallel display processing in chronological order on the extended display.

[0086] For example, as shown in FIG. 26(A), the parallel display processing unit 95 may generate a thumbnail image 98 in which the past images 96 acquired by the image acquisition unit 58 are arranged in chronological order at the lower part of the screen of the blood volume image 81. Although the thumbnail image 98 shows five past images 96 side by side so that the figure is not complicated, actually, control is performed to set the number and form (for example, seven) that are easy for the user to visually recognize by the parallel display processing unit 95. Also, in the figure, in order to prevent complication, signs may be attached only to a part. In addition to this, the parallel display processing unit 95 may perform control to change the number of past images 96 arranged in chronological order and the position of the thumbnail image 98 according to the user's setting. For example, the user may perform a setting to change the position of the thumbnail image 98 to the upper part of the screen.

[0087] FIG. 26(B) is a diagram of a thumbnail image 98 that enlarges and displays a part of the past image 96, and is an example of the past image 96 that is actually visually recognized by the user, in which the change amount of blood volume is displayed, thumbnailed, and displayed in parallel in chronological order. The numerical value displayed at the upper part of the screen of the past image 96 indicates the past image acquisition time 97 based on the current time. Note that the past image acquisition time 97 may be controlled to be displayed in some of the past images 96 (for example, displayed in the leftmost image and the central image), or may be changed by the user. Also, the change of the past image acquisition time 97 by the user may make only the designated image invisible, or may change the display of "-2 min" to the current time or the like (for example, 15:00).

[0088] Also, as shown in FIG. 26(C), when the parallel display processing unit 95 selects the past image 96a that the user has thumbnailed in chronological order at the lower part of the screen of the blood volume image 81, the parallel display processing unit 95 may perform control to display the enlarged past image 96a on the extended display 18 by the blood volume image display unit 57. The parallel display processing unit 95 preferably performs control that allows the user to select all of the past images acquired.

[0089] By the parallel display processing unit 95 arranging the blood volume images 81 in chronological order and thumbnailing them, the user can view a plurality of images arranged in chronological order at once, and can easily grasp the time transition of the change amount of blood volume in the observation target.

[0090] Also, the parallel display processing unit 95 may perform control to display a scroll bar for scrolling the past image subjected to the above parallel display processing on the blood volume image. For example, as shown in FIG. 27(A), when the parallel display processing unit 95 displays the past image 96a that the user has thumbnailed in chronological order at the lower part of the screen of the blood volume image 81 and the past image 96a enlarged and displayed on the extended display 18 on the extended display 18 by the blood volume image display unit 57, the parallel display processing unit 95 may display a scroll bar 99 for transitioning (so-called scrolling) the past image.

[0091] As shown in FIG. 27(B), when the scroll bar 99 moves to the right, the parallel display processing unit 95 performs control to switch from the past image 96a enlarged and displayed on the extended display 18 to the past image 96b displayed in parallel to the right. As shown in FIG. 27(C), when the scroll bar 99 moves to the right end, control may be performed to enlarge and display the past image 96c displayed in parallel at the right end on the extended display 18 and switch. Note that the transition of the scroll bar 99 may be automatically controlled, or the user may move the scroll bar 99 to switch the display of the past image. Further, the past images switched by the transition of the scroll bar 99 are not limited to the five displayed on the extended display 18, and it is preferable to perform control to switch all of the acquired past images by the transition of the scroll bar 99.

[0092] By the parallel display processing unit 95 displaying a scroll bar for scrolling the past images parallelly displayed on the blood volume image, the user can fast-forward and view the past images using the scroll bar, and can qualitatively grasp the speed of change of the blood volume.

[0093] Further, as in the first modification shown in FIG. 28, the blood volume image display unit 57 may include a pattern matching processing unit 100. The pattern matching processing unit 100 performs pattern matching based on the blood vessel shape of the observation target in the reference point imaging, and performs control to perform alignment when the endoscope or the observation target moves.

[0094] For example, as shown in Fig. 29(A), when the image acquisition unit 58 performs reference point imaging at a position where the congestion site 64 in the observation target 63a is within the region of interest target 65 and acquires the reference point image 67, as shown in Fig. 29(B), the image acquisition unit 58 may recognize the blood vessel shapes 101a and 101b in the observation target 63 and perform pattern matching for fixed-point observation of the region of interest 66. The pattern matching is preferably performed using the illumination light B1 which is the emission of the first blue light so that the blood vessels are more clearly imaged (see Fig. 7). When the pattern matching is performed on the observation target 63a, as shown in Fig. 29(C), in the blood volume mode image 62, even when the position and distance between the observation target 63a and the endoscope change and the observation target 63b is imaged, the image acquisition unit 58 uses the blood vessel shapes 101a and 101b, grasps the position of the region of interest 66, and acquires the blood volume mode image 62 so that the region of interest 66 is imaged.

[0095] By performing alignment based on the blood vessel shape in the observation target by the image acquisition unit 58, even when the endoscope or the subject moves, the change amount of blood volume and the oxygen saturation can be calculated for each pixel included in the region of interest at the same position. Note that the pixels used for the calculation may be all the pixels included in the region of interest, or a part of the pixels included in the region of interest, such as every other pixel or every two pixels.

[0096] Also, as in the modification example 2 shown in Fig. 30, the blood volume image display unit 57 may include a notification unit 102. When setting an observation target affected by disturbance as the region of interest in the reference point imaging, the notification unit 102 performs control to notify the user of the operation guidance for re-setting the region of interest.

[0097] For example, as shown in Fig. 31(A), when the image acquisition unit 58 erroneously sets the fat 103 in the observation target 63 as the region of interest 66 and acquires the reference point image 67 in the reference point imaging, as shown in Fig. 31(B), operation guidance GD such as "Please avoid bleeding, residue, fat, etc." may be displayed below the reference point image 67. Also, the user may be notified by voice or the like.

[0098] Even when the user accidentally sets a disturbance such as fat as the region of interest, the blood volume image display unit 57 can prevent the user from setting a region where the change amount of blood volume and oxygen saturation cannot be accurately calculated as the region of interest by notifying the guidance.

[0099] In the above embodiment, the hardware structure of the processing unit that executes various processes such as the blood volume image processing unit 56, the blood volume image display unit 57, the image acquisition unit 58, the calculation unit 59, the time-series image generation unit 60, and the time change graph generation unit 61 are various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD) such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing various processes.

[0100] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Further, a plurality of processing units may be composed of one processor. As an example of configuring a plurality of processing units with one processor, firstly, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Secondly, as represented by a System On Chip (SoC), etc., there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. Thus, various processing units are configured using one or more of the above various processors as a hardware structure.

[0101] Furthermore, the hardware structure of these various processors is more specifically an electrical circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements. Also, the hardware structure of the storage unit is a storage device such as an HDD (hard disc drive) or an SSD (solid state drive).

[0102] [Appended Claim 1] Comprising a processor, The processor is Performing an image acquisition process of acquiring images of an observation target at time intervals preset by a user, Performing a calculation process of calculating the blood volume and oxygen saturation for each image acquired in the image acquisition process, Generating an image indicating the change amount of the blood volume of each image acquired in the calculation process based on the blood volume at the time set by the user, Generating a graph showing the temporal changes of the blood volume and the oxygen saturation of each image acquired in the calculation process, An endoscope system that displays at least one of the image indicating the change amount of the blood volume or the graph indicating the temporal changes of the blood volume and the oxygen saturation on a display. [Additional item 2] The endoscope system according to claim 1, wherein in the image acquisition process, the processor performs reference point imaging for setting a reference observation target in a region of interest. [Additional item 3] The endoscope system according to claim 1 or 2, wherein the processor calculates average values of the blood volume and the oxygen saturation based on the pixels included in the region of interest, and performs control to use the calculated average values as reference values of the blood volume and the oxygen saturation. [Additional item 4] The endoscope system according to claim 3, wherein the processor performs a subtraction process of subtracting the reference value of the blood volume from the blood volume calculated in the calculation process. [Additional item 5] The endoscope system according to claim 4, wherein the processor assigns color maps with different shades according to the change amount based on the blood volume calculated in the subtraction process, and generates an image superimposed on the image. [Additional item 6] The endoscope system according to claim 4 or 5, wherein the processor represents the change on the minus side and the change on the plus side in different colors with respect to the change amount based on the blood volume calculated in the subtraction process, and generates an image superimposed on the image. [Additional item 7] The endoscope system according to claim 3, wherein in the display of the graph, for each image acquired in the image acquisition process, the processor performs the calculation process based on the pixels in the same region as the observation target set in the region of interest, and performs control to generate the graph in which the calculated blood volume and oxygen saturation are plotted. [Additional item 8] The endoscope system according to claim 1, wherein the processor performs control to thumbnail the image acquired in the image acquisition process and perform parallel display processing in chronological order on the display. [Additional item 9] The endoscope system according to appended claim 8, wherein the processor performs control to display a scroll bar for scrolling the image subjected to the parallel display process on the display. [Appended claim 10] The endoscope system according to appended claim 1, wherein the processor performs control to enable a user to change display or non-display in the graph. [Appended claim 11] The endoscope system according to appended claim 2, wherein the processor performs pattern matching based on the blood vessel shape of the observation target in the reference point imaging, and performs control to perform alignment when the endoscope or the observation target moves. [Appended claim 12] The endoscope system according to appended claim 2, wherein when the processor sets the observation target affected by disturbance as the region of interest in the reference point imaging, the processor performs control to notify the user of operation guidance for re-setting the region of interest.

Description of reference signs

[0103] 10 Endoscope system 12 Endoscope 12a Insertion portion 12b Operation portion 12c Bending portion 12d Tip portion 12e Mode switching switch 12f Reference point image acquisition instruction switch 12g Zoom operation portion 13 Light source device 14 Processor device 15 Display 16 User interface 17 Expansion processor device 18 Expansion display 20 Light source portion 20a V-LED 20b BS-LED 20c BL―ELD 20d G-LED 20e R-LED 21 Light source processor 23 Optical path coupling section 24 Light guide 30 Illumination optical system 31 Imaging optical system 32 Illumination lens 35 Objective lens 36 Imaging sensor 37 Imaging processor 40 CDS / AGD circuit 41 A / D converter 45 DSP 50 Image processing section 51 Image communication section 52 Display control section 53 Central control section 54 Normal mode 55 Blood volume mode 56 Blood volume image processing section 57 Blood volume image display section 58 Image acquisition section 59 Calculation section 60 Time-series image generation section 61 Time change graph generation section 62, 62a, 62b, 62c Images for blood volume mode 63, 63a, 63b Observation target 64 Congested area 65 Region of interest target 66 Region of interest 67 Reference point image 68 Oxyhemoglobin 69 Deoxyhemoglobin 70 Reflected light near 470nm 71 Reflected light near 550nm 72 Reflected light near 620nm 73 Signal ratio in short wavelength region 74 Signal ratio in long wavelength region 75 LUT 76 Change direction of oxygen saturation 77 Change direction of blood volume 78 Pixel 79 Subtraction processing 80 Color maps with different shades 80a, 80b Change amount of blood volume 81, 81a, 81b Blood volume images 82 Changes on the negative side 83 Changes on the positive side 84 Changes on the negative side with different shades 85 Changes on the positive side with different shades 86 Average value of blood volume 87 Average value of oxygen saturation 88 Blood volume and oxygen saturation graphs at each time 89 Doubtful site 90a Congested state 90b State with a small difference 91a, 91b Regions with large changes in blood volume 92 Region of interest indication line 93 Vascular treatment 94 Vascular treatment site 95 Parallel display processing unit 96, 96a, 96b, 96c Past images 97 Past image acquisition time 98 Thumbnail image 99 Scroll bar 100 Pattern matching processing unit 101a, 101b Vascular shape 102 Notification unit 103 Fat NP1 White light image NP2 White light equivalent image BP Blood volume image BP1 Blood volume image of the internal digestive tract BP2 Blood volume image of the serosal side GD Operation guidance EL, ELL, ELH Contour lines ST100~ST160, ST200~ST280 Steps t0, ta, tb, tc, td, te, tf, tg, th, ti, tj Time

Claims

1. Comprising a processor, The processor: Performs an image acquisition process of acquiring endoscopic images of an observation target at time intervals set in advance by a user, Performs a calculation process of calculating the blood volume and oxygen saturation for each image acquired in the image acquisition process, Generates an image showing the change amount of the blood volume of each image acquired in the calculation process based on the blood volume at the time set by the user, Generates a graph showing the temporal changes of the blood volume and the oxygen saturation of each image acquired in the calculation process, An endoscopic system that displays at least one of the image showing the change amount of the blood volume or the graph showing the temporal changes of the blood volume and the oxygen saturation on a display.

2. The processor performs reference point imaging for setting a reference observation target as a region of interest in the image acquisition process. The endoscopic system according to claim 1.

3. The processor calculates the average values of the blood volume and oxygen saturation based on the pixels included in the region of interest, and performs control to use the calculated average values as the reference values of the blood volume and oxygen saturation. The endoscopic system according to claim 2.

4. The processor performs a subtraction process of subtracting the reference value of the blood volume from the blood volume calculated in the calculation process. The endoscopic system according to claim 3.

5. The processor assigns color maps with different shades according to the change amount based on the blood volume calculated in the subtraction process, and generates an image superimposed on the image. The endoscopic system according to claim 4.

6. The processor represents the minus-side change and the plus-side change in different colors with respect to the change amount based on the blood volume calculated in the subtraction process, and generates an image superimposed on the image. The endoscopic system according to claim 4 or 5.

7. In the display of the graph, the processor performs the calculation process based on the pixels in the same region as the observation target set in the region of interest for each image acquired in the image acquisition process, and performs control to generate the graph on which the calculated blood volume and oxygen saturation are plotted. The endoscopic system according to claim 3.

8. The processor performs control to thumbnail the images acquired in the image acquisition process and perform parallel display processing in chronological order on the display. The endoscopic system according to claim 1.

9. The endoscope system according to claim 8, wherein the processor performs control to display a scroll bar for scrolling the image subjected to the parallel display process on the display.

10. The endoscope system according to claim 1, wherein the processor performs control to enable a user to change the display or non-display of the graph.

11. The endoscope system according to claim 2, wherein the processor performs pattern matching based on the blood vessel shape of the observation target in the reference point imaging, and performs control to perform alignment when the endoscope or the observation target moves.

12. The endoscope system according to claim 2, wherein when the processor sets the observation target affected by disturbance as the region of interest in the reference point imaging, the processor performs control to notify the user of operation guidance for re-setting the region of interest.

13. Comprising a processor, The processor, Performing a step of performing an image acquisition process of acquiring an endoscope image obtained by photographing an observation target at a time interval preset by a user; Performing a step of performing a calculation process of calculating a blood volume and an oxygen saturation for each image obtained by the image acquisition process; Generating an image showing a change amount of the blood volume of each image obtained by the calculation process based on the blood volume at a time set by the user; Generating a graph image showing a time change of the blood volume and the oxygen saturation of each image obtained by the calculation process; A method of operating an endoscope system, comprising: displaying at least one of the image showing the change amount of the blood volume or the graph showing the time change of the blood volume and the oxygen saturation on a display.

14. On a computer, Having a function of performing an image acquisition process of acquiring an endoscope image obtained by photographing an observation target at a time interval preset by a user; Having a function of performing a calculation process of calculating a blood volume and an oxygen saturation for each image obtained by the image acquisition process; Having a function of generating an image showing a change amount of the blood volume of each image obtained by the calculation process based on the blood volume at a time set by the user, or having a function of generating a graph showing a time change of the blood volume and the oxygen saturation of each image obtained by the calculation process; An operation program for an endoscope system for realizing a function of displaying at least one of the image indicating the change amount of the blood volume or the graph indicating the temporal change of the blood volume and the oxygen saturation on a display.

Images