Fluorescent endoscopic apparatus

Abstract

A processor 9 constituting a fluorescent endoscopic apparatus 1 alternately turns on first and second laser diodes 321, 322, detects differences in brightness level between pixels of a first captured image obtained by capturing one of first and second fluorescent lights with an imaging element 523 at a timing when the first laser diode 321 is turned on and the corresponding pixels of a second captured image obtained by capturing the one fluorescent light with the imaging element 523 at a timing when the second laser diode 322 is turned on, and determines, on the basis of the differences, a region where the one fluorescent light has been emitted.

Description

Fluorescence endoscope device

[0001] The present invention relates to a fluorescence endoscope apparatus.

[0002] Conventionally, there has been known a fluorescence endoscope device that irradiates an observation target (a subject such as a human) with visible light such as excitation light or white light emitted from a light source device, and observes the fluorescence emitted from a fluorescent reagent contained in the observation target by the irradiation of the excitation light (see, for example, Patent Document 1).

[0003] The fluorescence endoscope device described in Patent Document 1 is capable of observing first and second fluorescence of two different wavelength bands corresponding to two types of first and second fluorescent reagents. Specifically, the fluorescence endoscope device includes a first light source that emits visible light (hereinafter referred to as white light), a second light source that emits first excitation light corresponding to the first fluorescent reagent, and a third light source that emits second excitation light corresponding to the second fluorescent reagent. The fluorescence endoscope device also includes first and second image capture elements. The first image capture element is irradiated with light from an observation target and captures images of the white light reflected by the observation target. The second image capture element captures both the first fluorescence emitted from the first fluorescent reagent contained in the observation target when irradiated with the first excitation light and the second fluorescence emitted from the second fluorescent reagent contained in the observation target when irradiated with the second excitation light.

[0004] International Publication No. 2017 / 047140

[0005] A known first fluorescent reagent is one that emits a first fluorescence having a peak wavelength near 700 nm when irradiated with a first excitation light. A known second fluorescent reagent is one that emits a second fluorescence having a peak wavelength near 800 nm when irradiated with a second excitation light. The first fluorescent reagent is also sensitive to the second excitation light, and may emit the first fluorescence even when irradiated with the second excitation light. The second fluorescent reagent is also sensitive to the first excitation light, and may emit the second fluorescence even when irradiated with the first excitation light. The use of such first and second fluorescent reagents in the fluorescence endoscope device described in Patent Document 1 presents the following problem. Specifically, the first and second fluorescence are imaged by the same first imaging element. Therefore, when attempting to observe the first and second fluorescence simultaneously, it is difficult to determine whether a high-brightness area in the image obtained by imaging with the first imaging element is due to the first fluorescence or the second fluorescence.

[0006] Here, in order to distinguish between the first and second fluorescence, a configuration using the following optical system and three image sensors is conceivable. The optical system is an optical system such as a dichroic prism that separates the reflected white light reflected by the observation object, the first fluorescence, and the second fluorescence. The three image sensors are image sensors that capture the reflected white light and the first and second fluorescence separated by the optical system. However, a configuration using the above-described optical system and three image sensors results in an increase in size of the optical system.

[0007] Therefore, there is a demand for a technique that can avoid an increase in the size of the optical system and easily distinguish between the first and second fluorescent lights.

[0008] The present invention has been made in view of the above, and an object of the present invention is to provide a fluorescence endoscope apparatus that can avoid an increase in the size of the optical system and can easily distinguish between the first and second fluorescence.

[0009] In order to solve the above-mentioned problems and achieve the object, a fluorescence endoscope device according to the present invention includes a first excitation light source that emits first excitation light for exciting a first fluorescent reagent having a fluorescence emission peak wavelength in the vicinity of 700 nm, a second excitation light source that emits second excitation light for exciting a second fluorescent reagent having a fluorescence emission peak wavelength in the vicinity of 800 nm, an image sensor that images a first fluorescence emitted from the first fluorescent reagent contained in an observation object when irradiated with the first excitation light, and a second fluorescence emitted from the second fluorescent reagent contained in the observation object when irradiated with the second excitation light, and a processor that controls operations of the first excitation light source, the second excitation light source, and the image sensor, wherein one of the first excitation light source and the second excitation light source excites one of the first fluorescent reagent and the second fluorescent reagent. the processor alternately turns on the first laser diode and the second laser diode, and calculates a difference in brightness levels between corresponding pixels of a first captured image obtained by capturing an image of one of the first fluorescence and the second fluorescence using the image sensor at a timing when the first laser diode is turned on, and a second captured image obtained by capturing an image of the one fluorescence using the image sensor at a timing when the second laser diode is turned on, and determines an area from which the one fluorescence is emitted based on the difference.

[0010] According to the fluorescence endoscope apparatus of the present invention, it is possible to avoid an increase in the size of the optical system and to easily distinguish between the first and second fluorescence.

[0011] FIG. 1 is a diagram showing the configuration of a fluorescence endoscope apparatus according to an embodiment. FIG. 2 is a diagram showing the absorption spectrum of a first fluorescent reagent. FIG. 3 is a diagram showing the absorption spectrum of a second fluorescent reagent. FIG. 4 is a diagram explaining the wavelengths of second to fourth excitation light. FIG. 5 is a block diagram showing the configuration of a camera head and a control device. FIG. 6 is a diagram showing the configuration of an imaging unit. FIG. 7 is a diagram showing the operation of a light source apparatus in a second fluorescence observation mode. FIG. 8 is a diagram showing a white image generated in the second fluorescence observation mode. FIG. 9 is a diagram showing a fluorescence image generated in the second fluorescence observation mode. FIG. 10 is a diagram showing the operation of a light source apparatus in a first fluorescence observation mode. FIG. 11 is a diagram explaining region determination processing executed by a control device. FIG. 12 is a diagram showing a fluorescence image before region determination processing is executed. FIG. 13 is a diagram showing a fluorescence image after region determination processing is executed. FIG. 14 is a diagram showing the operation of a light source apparatus in a third fluorescence observation mode. FIG. 15 is a diagram showing a fluorescence image after region determination processing is executed.

[0012] Hereinafter, a mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

[0013] [Configuration of Fluorescence Endoscope Apparatus] Fig. 1 is a diagram showing the configuration of a fluorescence endoscope apparatus 1 according to an embodiment. The fluorescence endoscope apparatus 1 is an endoscope apparatus that uses an endoscope to perform fluorescence observation of an observation target (inside a living organism). As shown in Fig. 1, this fluorescence endoscope apparatus 1 includes an insertion section 2, a light source device 3, a light guide 4, a camera head 5, a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

[0014] In this embodiment, the insertion section 2 is a rigid endoscope. That is, the insertion section 2 has an elongated shape that is entirely rigid or partially flexible and partially rigid, and is inserted into an observation object. The insertion section 2 includes an optical system (not shown) that includes one or more lenses and focuses the normal light and first and second excitation lights reflected by the observation object, as well as the first and second fluorescence emitted from the first and second fluorescent reagents contained in the observation object. Hereinafter, for convenience of explanation, the normal light and first and second excitation lights reflected by the observation object, as well as the first and second fluorescence emitted from the first and second fluorescent reagents contained in the observation object, will be referred to as a subject image.

[0015] An excitation light cut filter 22 (FIG. 1) that removes the first and second excitation lights contained in the focused subject image is disposed at the proximal end (eyepiece 21) of the insertion portion 2. The excitation light cut filter 22 is not limited to being disposed in the insertion portion 2, but may also be disposed inside the camera head 5.

[0016] One end of the light guide 4 is connected to the light source device 3. As shown in FIG. 1 , the light source device 3 includes first to third light sources 31 to 33. The first light source 31 supplies normal light (hereinafter referred to as white light) including a wavelength band of visible light to one end of the light guide 4 under the control of the control device 9. The third light source 33 corresponds to a second excitation light source according to the present invention. This third light source 33 supplies second excitation light to one end of the light guide 4 for exciting a second fluorescent reagent contained in the observation target. The first and third light sources 31 and 33 may be configured using LEDs (Light Emitting Diodes) or laser diodes.

[0017] The second light source 32 corresponds to the first excitation light source according to the present invention. This second light source 32 supplies first excitation light to one end of the light guide 4 for exciting a first fluorescent reagent contained in the observation object. The first excitation light includes third and fourth excitation light described below. As shown in FIG. 1 , this second light source 32 includes a first laser diode 321 and a second laser diode 322. The first laser diode 321 emits third excitation light for exciting the first fluorescent reagent contained in the observation object. The second laser diode 322 emits fourth excitation light for exciting the first fluorescent reagent contained in the observation object.

[0018] Here, the first fluorescent reagent has the following characteristics. FIG. 2 is a diagram showing the absorption spectrum of the first fluorescent reagent. In FIG. 2, the horizontal axis represents wavelength [nm], and the vertical axis represents absorbance of the absorption spectrum. As shown in FIG. 2, the peak wavelength of the absorption spectrum of the first fluorescent reagent is near 680 [nm]. Therefore, in order to increase the intensity of the first fluorescence emitted from the first fluorescent reagent, it is preferable to use excitation light having a wavelength near 680 [nm] as the first excitation light. Furthermore, when excited by the first excitation light, the first fluorescent reagent emits first fluorescence having a wavelength near 700 [nm].

[0019] The second fluorescent reagent has the following characteristics. FIG. 3 is a diagram showing the absorption spectrum of the second fluorescent reagent. In FIG. 3, the horizontal axis represents wavelength [nm], and the vertical axis represents absorbance of the absorption spectrum. As shown in FIG. 3, the peak wavelength of the absorption spectrum of the second fluorescent reagent is near 800 [nm]. Therefore, in order to increase the intensity of the second fluorescence emitted from the second fluorescent reagent, it is preferable to use excitation light having a wavelength near 800 [nm] as the second excitation light. Furthermore, when excited by the second excitation light, the second fluorescent reagent emits second fluorescence having a wavelength near 800 [nm].

[0020] FIG. 4 is a diagram illustrating the wavelengths of the second to fourth excitation light beams. Specifically, FIG. 4 is a diagram showing the wavelengths of the second to fourth excitation light beams relative to the absorption spectra of the first and second fluorescent reagents shown in FIGS. 2 and 3. In FIG. 4, the curve indicated by the symbol "CL1" represents the absorption spectrum of the first fluorescent reagent. The curve indicated by the symbol "CL2" represents the absorption spectrum of the second fluorescent reagent. The wavelength indicated by the symbol "P2" represents the wavelength of the second excitation light beam. The wavelength indicated by the symbol "P3" represents the wavelength of the third excitation light beam. The wavelength indicated by the symbol "P4" represents the wavelength of the fourth excitation light beam.

[0021] As can be seen from the absorption spectrum of the second fluorescent reagent shown by curve CL2 in Figure 4, the second fluorescent reagent is also sensitive to excitation light having a wavelength near 680 [nm], i.e., the first excitation light, and emits the second fluorescence even when irradiated with the first excitation light. Therefore, when observing the first and second fluorescence simultaneously, it is difficult to distinguish between the first and second fluorescence. In this embodiment, as shown in Figure 4, the third excitation light has a wavelength P3 of 675 [nm]. The fourth excitation light has a wavelength P4 of 690 [nm].

[0022] Furthermore, the second excitation light has a wavelength P2 of 800 [nm]. Therefore, as can be seen from the absorption spectrum of the first fluorescent reagent shown by curve CL1 in Figure 4, the first fluorescent reagent has no sensitivity to the second excitation light with a wavelength of 800 [nm], and does not emit the first fluorescent light even when irradiated with the second excitation light.

[0023] In this embodiment, the light source device 3 is configured as a separate entity from the control device 9, but this is not limited to this, and a configuration in which the light source device 3 and the control device 9 are provided in the same housing may also be adopted.

[0024] One end of the light guide 4 is detachably connected to the light source device 3. The other end of the light guide 4 is detachably connected to the insertion section 2. The light guide 4 propagates the white light and the first and second excitation light supplied from the light source device 3 (first to third light sources 31 to 33) from one end to the other end, and supplies them to the insertion section 2. The white light and the first and second excitation light supplied to the insertion section 2 are emitted from the tip of the insertion section 2 and irradiated onto the observation object. The white light and the first and second excitation light irradiated onto the observation object and reflected by the observation object, as well as the first and second fluorescence (subject image) emitted from the first and second fluorescent reagents contained in the observation object, are each collected by an optical system within the insertion section 2.

[0025] The camera head 5 is detachably connected to the proximal end (eyepiece 21 ( FIG. 1 )) of the insertion section 2. The camera head 5 captures an image of a subject after light is collected by the insertion section 2 and the first and second excitation lights have been removed by the excitation light cut filter 22. For ease of explanation, the image signal obtained by imaging with the camera head 5 will hereinafter be collectively referred to as a captured image. Furthermore, the subject image after the first and second excitation lights have been removed by the excitation light cut filter 22 will be referred to as an excitation light-removed subject image. The detailed configuration of the camera head 5 will be described later in the section "Configuration of the Camera Head."

[0026] One end of the first transmission cable 6 is detachably connected to the control device 9. The other end of the first transmission cable 6 is detachably connected to the camera head 5. The first transmission cable 6 transmits the captured image output from the camera head 5 to the control device 9, and also transmits the control signal, synchronization signal, clock, power, and the like sent from the control device 9 to the camera head 5.

[0027] The captured images and the like transmitted from the camera head 5 to the control device 9 via the first transmission cable 6 may be transmitted as optical signals or as electrical signals. The same applies to the transmission of control signals, synchronization signals, and clocks from the control device 9 to the camera head 5 via the first transmission cable 6.

[0028] The display device 7 is configured with a display using a liquid crystal or organic EL (Electro Luminescence) display, etc., and displays an image based on a video signal from the control device 9 under the control of the control device 9 .

[0029] One end of the second transmission cable 8 is detachably connected to the display device 7. The other end of the second transmission cable 8 is detachably connected to the control device 9. The second transmission cable 8 transmits the video signal processed by the control device 9 to the display device 7.

[0030] The control device 9 includes controllers such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and comprehensively controls the operations of the light source device 3, the camera head 5, and the display device 7. The control device 9 is not limited to a CPU or an MPU, and may include an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or the like. The detailed configuration of the control device 9 will be described later in the section "Configuration of the Control Device."

[0031] One end of the third transmission cable 10 is detachably connected to the light source device 3. The other end of the third transmission cable 10 is detachably connected to the control device 9. The third transmission cable 10 transmits a control signal from the control device 9 to the light source device 3.

[0032] [Configuration of Camera Head] Next, a description will be given of the configuration of the camera head 5. Fig. 5 is a block diagram showing the configuration of the camera head 5 and the control device 9. As shown in Fig. 5, the camera head 5 includes a lens unit 51, an imaging unit 52, and a communication unit 53.

[0033] The lens unit 51 is configured using one or more lenses, and forms an image of the subject from which excitation light has been removed on the imaging surfaces of the first and second image sensors 522 and 523, respectively.

[0034] 6 is a diagram showing the configuration of the imaging unit 52. The imaging unit 52 is a part that generates a captured image by capturing an image of a subject from which excitation light has been removed. As shown in FIGS. 5 and 6 , the imaging unit 52 includes a prism 521, first and second image sensors 522 and 523, and a signal processing unit 524.

[0035] The prism 521 has a substantially cubic shape formed by combining two triangular prism-shaped light-transmitting members 521a (FIG. 6). A dichroic filter 521b is provided at the interface between the two light-transmitting members 521a.

[0036] The dichroic filter 521b has the property of transmitting light in wavelength bands near 700 [nm] and 800 [nm] and reflecting light in other wavelength bands. Therefore, of the subject image from which excitation light has been removed that has entered the prism 521, most of the reflected white light WL ( FIG. 6 ) from which the first and second excitation lights have been removed by the excitation light cut filter 22 is reflected by the dichroic filter 521b. Then, as shown in FIG. 6 , the reflected white light WL travels toward the first image sensor 522. Meanwhile, of the subject image from which excitation light has been removed that has entered the prism 521, most of the first and second fluorescence light FL1, FL2 ( FIG. 6 ) from which the first and second excitation lights have been removed by the excitation light cut filter 22 is transmitted through the dichroic filter 521b (prism 521). The first and second fluorescence FL1 and FL2 then travel toward the second image sensor 523.

[0037] The first and second image sensors 522 and 523 receive incident light and convert it into an electrical signal (analog signal). Examples of the first and second image sensors 522 and 523 include a CMOS (Complementary Metal Oxide Semiconductor) which is a rolling shutter type image sensor in which multiple pixels are arranged two-dimensionally in horizontal line units, and a CCD (Charge Coupled Device) which is a global shutter type image sensor.

[0038] Then, the first image sensor 522 captures an image of the light reflected by the dichroic filter 521b under the control of the control device 9. For ease of explanation, the captured image generated by the first image sensor 522 will be referred to as a white image below.

[0039] Here, a color filter 522a is provided on the light receiving surface of the first image sensor 522, as shown in FIGS.

[0040] The color filter 522a is a color filter in which three filter groups are grouped according to the wavelength bands of light (red, green, blue) to be transmitted, and are arranged in a specific format (for example, a Bayer array). Specifically, the color filter 522a has an R filter group that mainly transmits light in the red wavelength band, a B filter group that mainly transmits light in the blue wavelength band, and a G filter group that mainly transmits light in the green wavelength band.

[0041] The second imaging element 523 corresponds to the imaging element according to the present invention. Under the control of the control device 9, the second imaging element 523 captures an image of light transmitted through the prism 521. For ease of explanation, the image captured by the second imaging element 523 will be referred to as a fluorescent image below. Note that the light receiving surface of the second imaging element 523 does not have a color filter 522a like the first imaging element 522. In other words, the second imaging element 523 is a so-called monochrome image sensor.

[0042] The number of pixels in the white image and the number of pixels in the fluorescent image may be different or the same.

[0043] The signal processing unit 524 outputs a captured image (digital signal) by performing signal processing on the captured image (analog signal) generated by the first and second imaging elements 522 and 523 under the control of the control device 9. For example, the signal processing unit 524 performs signal processing on the captured image (analog signal) generated by the first and second imaging elements 522 and 523, such as processing to remove reset noise, processing to multiply the analog signal by an analog gain that amplifies the analog signal (hereinafter referred to as analog gain adjustment), and A / D conversion.

[0044] The communication unit 53 functions as a transmitter that transmits the captured images sequentially output from the imaging unit 52 to the control device 9 via the first transmission cable 6 .

[0045] The communication unit 53 may transmit the white image and the fluorescent image to the control device 9 in that order, or may transmit the white image and the fluorescent image simultaneously.

[0046] [Configuration of Control Device] Next, the configuration of the control device 9 will be described with reference to Fig. 5. The control device 9 corresponds to a processor according to the present invention. As shown in Fig. 4, the control device 9 includes a communication unit 91, a processing module 92, a control unit 93, an input unit 94, an output unit 95, and a storage unit 96.

[0047] The communication unit 91 functions as a receiver that receives the captured images sequentially transmitted from the camera head 5 (communication unit 53 ) via the first transmission cable 6 .

[0048] Under the control of the control unit 93, the processing module 92 processes the captured images sequentially transmitted from the camera head 5 (communication unit 53) and received by the communication unit 91. As shown in FIG. 5 , the processing module 92 includes an image processing unit 921 and a display control unit 922.

[0049] The image processing unit 921 performs image processing on the input captured image (the captured image received by the communication unit 91). Examples of the image processing include optical black subtraction processing (clamp processing), white balance adjustment processing, demosaic processing, color correction matrix processing, gamma correction processing, YC processing for converting RGB signals into luminance color difference signals (Y, Cb / Cr signals), digital gain adjustment for multiplying by a digital gain, noise removal, and filtering for enhancing structure.

[0050] The image processing performed on the white image and the image processing performed on the fluorescent image may be different from each other, or may be the same.

[0051] Under the control of the control unit 93, the display control unit 922 generates a video signal for displaying the captured image after image processing has been performed by the image processing unit 921. Then, the display control unit 922 outputs the video signal to the display device 7 via the second transmission cable 8.

[0052] The control unit 93 is realized by a controller such as a CPU or an MPU executing various programs stored in the storage unit 96, and controls the operations of the light source device 3, the camera head 5, and the display device 7, as well as the operation of the entire control device 9. Note that the control unit 93 is not limited to a CPU or an MPU, and may include an ASIC, an FPGA, a GPU, or the like.

[0053] The input unit 94 is configured using operation devices such as a mouse, a keyboard, and a touch panel, and receives user operations from a user such as a surgeon. The input unit 94 then outputs an operation signal corresponding to the user operation to the control unit 93.

[0054] The output unit 95 is configured using a speaker, a printer, etc., and outputs various information.

[0055] The storage unit 96 stores programs executed by the control unit 93, information necessary for the processing of the control unit 93, and the like.

[0056] [Operation of Fluorescence Endoscope Apparatus] Next, the operation of the above-described fluorescence endoscope apparatus 1 will be described. In this embodiment, the fluorescence endoscope apparatus 1 is set to one of first to third fluorescence observation modes in response to, for example, a user operation on the input unit 94. The first fluorescence observation mode is a mode in which fluorescence observation is performed using the first fluorescence. The second fluorescence observation mode is a mode in which fluorescence observation is performed using the second fluorescence. The third fluorescence observation mode is a mode in which fluorescence observation is performed using both the first and second fluorescence. Below, the operation of the fluorescence endoscope apparatus 1 will be described for each of the first to third fluorescence observation modes. It is assumed that the object to be observed contains both the first and second fluorescent reagents.

[0057] [Operation of the Fluorescence Endoscope Device in the Second Fluorescence Observation Mode] Figure 7 is a diagram showing the operation of the light source device 3 in the second fluorescence observation mode. Specifically, Figure 7(a) is a time chart showing the operating state of the first light source 31. Figure 7(b) is a time chart showing the operating state of the third light source 33. Figure 7(c) is a time chart showing the operating state of the second light source 32. In the second fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522, 523 as follows.

[0058] As shown in Fig. 7A, the control unit 93 continuously lights up the first light source 31 and constantly emits white light from the first light source 31. Note that Fig. 7A shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

[0059] 7B, the control unit 93 continuously lights up the third light source 33 and constantly emits the second excitation light from the third light source 33. Note that, since FIG. 7B shows a period shorter than the cycle in which dimming control is performed, the second excitation light is emitted at a constant intensity.

[0060] Furthermore, the control unit 93 keeps the second light source 32 turned off at all times, as shown in FIG. 7(c).

[0061] Then, the control unit 93 causes the first and second image pickup elements 522 and 523 to perform an image pickup operation for each specific frame period.

[0062] 8 is a diagram showing a white image F1 generated in the second fluorescence observation mode. Specifically, the first image sensor 522 sequentially captures the reflected white light WL for each specific frame period, and sequentially generates the white image F1 (FIG. 8).

[0063] FIG. 9 is a diagram showing a fluorescence image F2 generated in the second fluorescence observation mode. Note that FIG. 9 is an image obtained by capturing the same subject as the white image F1 shown in FIG. 8 , and the region Ar1 indicated by the solid line is a region where the intensity of the second fluorescence FL2 is high. In FIG. 9 , regions other than the region Ar1 are indicated by dashed lines. The second image sensor 523 sequentially captures the second fluorescence FL2 for each specific frame period and sequentially generates fluorescence images F2 ( FIG. 9 ). Here, the wavelength P2 of the second excitation light is 800 nm. Furthermore, as shown by the curve CL1 in FIG. 4 , the first fluorescent reagent is insensitive to the second excitation light and does not emit the first fluorescence FL1 even when irradiated with the second excitation light. Therefore, the region Ar1 in the fluorescence image F2 shown in FIG. 9 is a region emitted only by the second fluorescence FL2.

[0064] Under the control of the control unit 93, the image processing unit 921 performs image processing on the white image F1 and the fluorescent image F2 received by the communication unit 91. The image processing unit 921 also superimposes the white image F1 and the fluorescent image F2, for example, using a known alpha blending process or additive blending process, to generate a superimposed image. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

[0065] [Operation of the Fluorescence Endoscope Device in the First Fluorescence Observation Mode] Figure 10 is a diagram showing the operation of the light source device 3 in the first fluorescence observation mode. Specifically, Figure 10(a) is a time chart showing the operating state of the first light source 31. Figure 10(b) is a time chart showing the operating state of the third light source 33. Figure 10(c) is a time chart showing the operating state of the second light source 32. In the first fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522, 523 as follows.

[0066] As shown in Fig. 10(a), the control unit 93 continuously lights up the first light source 31 and constantly emits white light from the first light source 31. Note that Fig. 10(a) shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

[0067] Furthermore, the control unit 93 keeps the third light source 33 turned off at all times, as shown in FIG. 10(b).

[0068] Furthermore, as shown in (c) of Fig. 10, the control unit 93 alternately turns on the first and second laser diodes 321 and 322 for each specific frame period, thereby alternately emitting the third and fourth excitation light beams. In (c) of Fig. 10, the timing at which the first laser diode 321 turns on is timing T1. The timing at which the second laser diode 322 turns on is timing T2. Note that (c) of Fig. 10 shows a period shorter than the cycle at which dimming control is performed, and therefore the third and fourth excitation light beams are each emitted at a constant intensity.

[0069] Then, the control unit 93 causes the first and second image pickup elements 522 and 523 to perform an image pickup operation for each specific frame period.

[0070] The first image sensor 522 sequentially captures the reflected white light WL for each specific frame period, and sequentially generates white images F1 (FIG. 8).

[0071] FIG. 11 is a diagram illustrating the region determination process executed by the control device 9. Specifically, FIG. 11(a) is an enlarged view of a portion of FIG. 4. FIG. 11(b) is a diagram illustrating the intensities of the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights. FIG. 11(c) is a diagram illustrating the intensities of the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights. FIG. 12 is a diagram illustrating a fluorescence image F2A before the region determination process is executed. Note that FIG. 12 is an image obtained by capturing the same subject as the white image F1 shown in FIG. 8, and the region Ar2 indicated by the solid line is a region where the intensity of at least one of the first and second fluorescence FL1 and FL2 is high. In FIG. 12, regions other than the region Ar2 are indicated by dashed lines. FIG. 13 is a diagram corresponding to FIG. 12 and illustrates a fluorescence image F2B after the region determination process has been executed.

[0072] The second image sensor 523 sequentially captures the first and second fluorescence FL1 and FL2 for each specific frame period, generating a fluorescence image F2A ( FIG. 12 ). Here, the wavelength P3 of the third excitation light is 675 nm. The wavelength P4 of the fourth excitation light is 690 nm. The second fluorescent reagent is also sensitive to the third and fourth excitation lights, as indicated by the curve CL2 in FIG. 11( a), and emits the second fluorescence FL2 even when irradiated with the third and fourth excitation lights. Therefore, it is difficult to determine whether the first or second fluorescence FL1 or FL2 is responsible for the emission of the region Ar2 in the fluorescence image F2A shown in FIG. 12 . The control device 9 then executes a region determination process to determine whether the first or second fluorescence FL1 or FL2 is responsible for the emission of the region Ar2, as described below.

[0073] Under the control of the control unit 93, the image processing unit 921 performs image processing on the white image F1 and the fluorescent image F2A received by the communication unit 91. Under the control of the control unit 93, the image processing unit 921 calculates the difference in brightness levels between corresponding pixels in the fluorescent image F2A (first captured image) generated by the second image sensor 523 at timing T1 ((c) in FIG. 10 ) when the first laser diode 321 is turned on, and the fluorescent image F2A (second captured image) generated by the second image sensor 523 at timing T2 ((c) in FIG. 10 ) when the second laser diode 322 is turned on. Here, the brightness level may be, for example, a brightness value or a pixel value. The image processing unit 921 then determines the region from which the first fluorescent light FL1 is emitted based on the magnitude of the difference. FIG. 13 illustrates an example in which the region Ar21 of the region Ar2 is determined to be the region from which the first fluorescent light FL1 is emitted.

[0074] 11(b) and 11(c), there is a difference between the difference D1 in intensity of the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights, and the difference D2 in intensity of the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights. The image processing unit 921 then uses the difference between the differences D1 and D2 to determine that the region Ar21 is a region from which the first fluorescence FL1 is emitted, based on the magnitude of the difference.

[0075] The image processing unit 921 also determines that area Ar22, other than area Ar21, is the area from which the second fluorescence FL2 is emitted. The image processing unit 921 then generates fluorescence image F2B ( FIG. 13 ) by reducing the brightness value of area Ar22 in fluorescence image F2A. For ease of explanation, the reduction in brightness value of area Ar22 is represented by a dashed line in FIG. 13 .

[0076] The image processing unit 921 then superimposes the white image F1 and the fluorescent image F2B, for example, using a known alpha blending process or additive blending process, to generate a superimposed image. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

[0077] [Operation of the Fluorescence Endoscope Apparatus in the Third Fluorescence Observation Mode] Figure 14 is a diagram showing the operation of the light source device 3 in the third fluorescence observation mode. Specifically, Figure 14(a) is a time chart showing the operating state of the first light source 31. Figure 14(b) is a time chart showing the operating state of the third light source 33. Figure 14(c) is a time chart showing the operating state of the second light source 32. In the third fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522, 523 as follows.

[0078] As shown in Fig. 14(a), the control unit 93 continuously lights up the first light source 31 and constantly emits white light from the first light source 31. Note that Fig. 14(a) shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

[0079] 14(b) and 14(c), the control unit 93 sequentially turns on the third light source 33 and the first and second laser diodes 321 and 322 for each specific frame period, thereby sequentially emitting the second to fourth excitation light beams. In FIG. 14(c), the timing at which the first laser diode 321 turns on is timing T1. The timing at which the second laser diode 322 turns on is timing T2. Note that, because FIGS. 14(b) and 14(c) show cycles that are shorter than the cycle at which dimming control is performed, the second to fourth excitation light beams are each emitted at a constant intensity.

[0080] Then, the control unit 93 causes the first and second image pickup elements 522 and 523 to perform an image pickup operation for each specific frame period.

[0081] The first image sensor 522 sequentially captures the reflected white light WL for each specific frame period, and sequentially generates white images F1 (FIG. 8).

[0082] 12 and 13, and shows a fluorescence image F2B after region determination processing has been performed. The second image sensor 523 sequentially captures the first and second fluorescence FL1 and FL2 for each specific frame period, and sequentially generates fluorescence images F2A (FIG. 12).

[0083] The image processing unit 921, under the control of the control unit 93, performs image processing on the white image F1 and the fluorescent image F2A received by the communication unit 91. Also, under the control of the control unit 93, the image processing unit 921 performs region determination processing, similar to the first fluorescence observation mode. Specifically, the image processing unit 921 calculates the difference in brightness levels between corresponding pixels in the fluorescent image F2A (first captured image) generated by the second image sensor 523 at timing T1 ((c) in FIG. 14 ) when the first laser diode 321 is turned on, and the fluorescent image F2A (second captured image) generated by the second image sensor 523 at timing T2 ((c) in FIG. 14 ) when the second laser diode 321 is turned on. Here, examples of the brightness level include brightness values ​​or pixel values. The image processing unit 921 then determines the region from which the first fluorescent light FL1 and the region from which the second fluorescent light FL2 are emitted, based on the magnitude of the difference. 15 illustrates an example in which area Ar21 of area Ar2 is determined to be the area from which the first fluorescence FL1 is emitted, and area Ar22 is determined to be the area from which the second fluorescence FL2 is emitted. The image processing unit 921 also generates a fluorescence image F2B ( FIG. 15 ) by distinguishing area Ar21 from area Ar22 in fluorescence image F2A, for example, by using different colors.

[0084] The image processing unit 921 then superimposes the white image F1 and the fluorescent image F2B, for example, using a known alpha blending process or additive blending process, to generate a superimposed image. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

[0085] The above-described embodiment provides the following advantages. In the fluorescence endoscope device 1 according to this embodiment, the control device 9 alternately lights up the first and second laser diodes 321 and 322. The control device 9 calculates the difference in brightness levels between corresponding pixels in a fluorescence image F2A generated by the second image sensor 523 at time T1 when the first laser diode 321 is turned on and a fluorescence image F2A generated by the second image sensor 523 at time T2 when the second laser diode 322 is turned on. The control device 9 then uses the difference D1 in intensity between the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights and the difference D2 in intensity between the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights to determine the region from which the first fluorescence FL1 is emitted based on the magnitude of this difference. This facilitates distinguishing between the first and second fluorescence FL1 and FL2. Furthermore, the optical system (the prism 521 including the dichroic filter 521b) that separates the reflected white light WL from the first and second fluorescence lights FL1, FL2 can also be compact. As described above, the fluorescence endoscope device 1 according to this embodiment can easily distinguish between the first and second fluorescence lights FL1, FL2 while achieving a compact optical system.

[0086] Other Embodiments Up to this point, the embodiments for carrying out the present invention have been described, but the present invention should not be limited to only the above-described embodiments. In the above-described embodiments, one of the wavelengths of the third and fourth excitation light may be set to the same wavelength as the peak wavelength in the absorption spectrum of the first fluorescent reagent. If set in this manner, the difference between the differences D1 and D2 can be made clear, and the region from which the first fluorescent light FL1 is emitted can be accurately determined.

[0087] The fluorescence image F2B may be a fluorescence image whose contrast has been enhanced by performing the region determination process described above or HDR (high dynamic range) described below. The fluorescence image F2B is a fluorescence image (third captured image) whose contrast has been enhanced by HDR, a technique for expressing a range from dark to bright areas by combining a bright image and a dark image. The bright image is the fluorescence image F2A (second captured image) generated by the second image sensor 523 at timing T2 ((c) in FIG. 10 ) when the second laser diode 321 is turned on. The dark image is the fluorescence image F2A (first captured image) generated by the second image sensor 523 at timing T1 ((c) in FIG. 10 ) when the first laser diode 321 is turned on.

[0088] In the above-described embodiment, one of the excitation light sources according to the present invention is the second light source 32 , but this is not limiting and the third light source 33 may also be used.

[0089] REFERENCE SIGNS LIST 1 Fluorescence endoscope device 2 Insertion section 3 Light source device 4 Light guide 5 Camera head 6 First transmission cable 7 Display device 8 Second transmission cable 9 Control device 10 Third transmission cable 21 Eyepiece section 22 Excitation light cut filter 31 First light source 32 Second light source 33 Third light source 51 Lens unit 52 Imaging section 53 Communication section 91 Communication section 92 Processing module 93 Control section 94 Input section 95 Output section 96 Storage section 321 First laser diode 322 Second laser diode 521 Prism 521a Light-transmitting member 521b Dichroic filter 522 First imaging element 522a Color filter 523 Second imaging element 524 Signal processing section 921 Image processing section 922 Display control section Ar1, Ar2, Ar21, Ar22 Areas CL1, CL2 Curves D1, D2 Difference F1 White image F2, F2A, F2B Fluorescence images FL1 First fluorescence FL2 Second fluorescence P2 to P4 Wavelengths T1, T2 Timing WL Reflected light of white light

Claims

1. A device comprising: a first excitation light source that emits first excitation light for exciting a first fluorescent reagent having a peak fluorescence emission wavelength in the vicinity of 700 nm; a second excitation light source that emits second excitation light for exciting a second fluorescent reagent having a peak fluorescence emission wavelength in the vicinity of 800 nm; an image capture element that captures first fluorescence emitted from the first fluorescent reagent contained in an observation object when irradiated with the first excitation light, and second fluorescence emitted from the second fluorescent reagent contained in the observation object when irradiated with the second excitation light; and a processor that controls the operations of the first excitation light source, the second excitation light source, and the image capture element, wherein one of the first excitation light source and the second excitation light source comprises: a first laser diode that emits third excitation light for exciting one of the first fluorescent reagent and the second fluorescent reagent; and a second laser diode that emits fourth excitation light having a wavelength different from the third excitation light for exciting the one fluorescent reagent, the first excitation light includes the third excitation light and the fourth excitation light; and the processor alternately turns on the first laser diode and the second laser diode, and calculates a difference in brightness levels between corresponding pixels in a first captured image obtained by causing the image sensor to capture an image of one of the first fluorescence and the second fluorescence at a timing when the first laser diode is turned on, and a second captured image obtained by causing the image sensor to capture an image of the one fluorescence at a timing when the second laser diode is turned on, and determines an area from which the one fluorescence is emitted based on the difference.

2. The fluorescence endoscope device according to claim 1, wherein the processor determines the region from which the one of the fluorescence beams is emitted based on the magnitude of the difference.

3. A fluorescence endoscope device according to claim 1, wherein the processor generates a third captured image with enhanced contrast based on the first captured image and the second captured image.

4. A fluorescence endoscope apparatus according to claim 1, wherein said one excitation light source is said first excitation light source.

5. A fluorescence endoscope device according to claim 1, wherein one of the third excitation light and the fourth excitation light has the same wavelength as the peak wavelength in the absorption spectrum of one of the fluorescent reagents.

Images