Medical control device, medical observation system, and control method

Abstract

A medical control device (9) comprises: a light source control unit (941) that controls operation of a light source device (3), and executes, in a specific order, a first control for setting a light emission energy of a first light emitted from the light source device (3) to a first light emission energy, and a second control for setting the light emission energy of the first light to a second light emission energy lower than the first light emission energy; an imaging control unit (942) that controls operation of an imaging device (5), causes a first captured image to be generated by imaging of the imaging device (5) during execution of the first control, and causes a second captured image to be generated by imaging of the imaging device (5) during execution of the second control; and an image processing unit (93) that generates a display image on the basis of the first and second captured images.

Description

Medical control device, medical observation system, and control method

[0001] The present disclosure relates to a medical control device, a medical observation system, and a control method.

[0002] Conventionally, visible light such as excitation light, which is narrow-band light emitted from a light source device, or white light, which is broadband light, is irradiated onto an observation target (a subject such as a person), and fluorescence emitted from substances contained in the observation target by the irradiation of the excitation light (hereinafter referred to as observation target fluorescence) is observed. A medical observation system is known (see, for example, Patent Document 1). According to such fluorescence observation, it is possible to grasp a tissue state that is difficult to recognize through the observation target fluorescence. Therefore, fluorescence observation can be used for various purposes and applications such as identification of a lesion part.

[0003] Japanese Patent Application Laid-Open No. 2021-132695

[0004] By the way, there is a need to observe not only one type of observation target fluorescence but also a plurality of types of observation target fluorescence in different wavelength bands. In the medical observation system described in Patent Document 1, it is possible to observe one type of observation target fluorescence, but it is not possible to observe a plurality of types of observation target fluorescence in different wavelength bands. Therefore, a technique that can favorably observe a plurality of types of observation target fluorescence in different wavelength bands is desired.

[0005] The present disclosure has been made in view of the above, and an object thereof is to provide a medical control device, a medical observation system, and a control method that can favorably observe a plurality of types of observation target fluorescence in different wavelength bands.

[0006] To solve the above-mentioned problems and achieve the objective, the medical control device according to this disclosure includes a light source control unit that controls the operation of a light source device and executes in a specific order a first control that sets the luminescence energy of a first light emitted from the light source device to a first luminescence energy and a second control that sets the luminescence energy of the first light to a second luminescence energy lower than the first luminescence energy; an imaging control unit that controls the operation of an imaging device and generates a first image by imaging with the imaging device when the first control is executed and generates a second image by imaging with the imaging device when the second control is executed; and an image processing unit that generates a display image based on the first image and the second image.

[0007] Furthermore, the medical observation system according to this disclosure comprises a light source device that emits a first light, an imaging device that images the reflected light from an object to be observed, and a medical control device that controls the operation of the light source device and the imaging device, respectively. The medical control device comprises a light source control unit that controls the operation of the light source device and executes, in a specific order, a first control that sets the emission energy of the first light to a first emission energy and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy; an imaging control unit that controls the operation of the imaging device and generates a first image by imaging with the imaging device when the first control is executed, and generates a second image by imaging with the imaging device when the second control is executed; and an image processing unit that generates a display image based on the first image and the second image.

[0008] Furthermore, the control method relating to this disclosure is a control method executed by a medical control device, wherein the medical control device controls the operation of a light source device and executes, in a specific order, a first control that sets the emission energy of a first light emitted from the light source device to a first emission energy, and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy, thereby controlling the operation of an imaging device, causing the imaging device to generate a first image when it executes the first control, and causing the imaging device to generate a second image when it executes the second control, and generating a display image based on the first image and the second image.

[0009] According to the medical control device, medical observation system, and control method described herein, multiple types of fluorescence in different wavelength bands can be observed effectively.

[0010] Figure 1 is a diagram illustrating the configuration of a medical observation system according to an embodiment. Figure 2 is a block diagram illustrating the configuration of a camera head and a control device. Figure 3 is a diagram illustrating the functions of the light source control unit and the imaging control unit. Figure 4 is a diagram illustrating the functions of the light source control unit and the imaging control unit. Figure 5 is a diagram illustrating the functions of the light source control unit and the imaging control unit. Figure 6 is a diagram illustrating the functions of the display control unit. Figure 7 is a diagram illustrating the functions of the display control unit. Figure 8 is a diagram illustrating a method for separating the signal value based on the observed fluorescence from the signal value based on unwanted light when the S / N ratio of the signal value based on the observed fluorescence is greater than 1. Figure 9 is a diagram illustrating a method for calculating the S / N ratio. Figure 10 is a diagram illustrating a modification 1 of the embodiment. Figure 11 is a diagram illustrating a modification 2 of the embodiment. Figure 12 is a diagram illustrating a modification 2 of the embodiment. Figure 13 is a diagram illustrating a modification 2 of the embodiment. Figure 14 is a diagram illustrating a modification 3 of the embodiment. Figure 15 is a diagram illustrating a modification 3 of the embodiment. Figure 16 is a diagram illustrating a modification 3 of the embodiment. Figure 17 is a diagram illustrating a modification 3 of the embodiment. Figure 18 is a diagram illustrating a modification 3 of the embodiment. Figure 19 illustrates a modification 4 of the embodiment. Figure 20 illustrates a modification 7 of the embodiment. Figure 21 illustrates a modification 8 of the embodiment. Figure 22 illustrates a modification 8 of the embodiment.

[0011] The embodiments for implementing this disclosure (hereinafter referred to as "embodiments") will be described below with reference to the drawings. However, the embodiments described below do not limit this disclosure. Furthermore, the same parts are denoted by the same reference numerals in the drawings.

[0012] [Configuration of the Medical Observation System] Figure 1 shows the configuration of the medical observation system 1 according to this embodiment. In this embodiment, the medical observation system 1 is a medical endoscope system that uses an endoscope to observe the observation target OB (inside the body). As shown in Figure 1, the medical observation system 1 comprises an insertion unit 2, a light source device 3, a light guide 4, a camera head 5, a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

[0013] In this embodiment, the insertion section 2 is made of a rigid endoscope. That is, the insertion section 2 has an elongated shape that is either entirely rigid or partially flexible with other parts rigid, and is inserted into the observation target OB. Inside the insertion section 2, there is an optical system made of one or more lenses that collects the reflected light (image of the subject) from the observation target OB.

[0014] Furthermore, an excitation light cut filter 22 (Figure 1) is provided at the base end (eyepiece 21) of the insertion section 2 to partially, substantially, or completely suppress the first and second excitation lights, which will be described later, contained in the focused reflected light (subject image). Note that the excitation light cut filter 22 is not limited to the insertion section 2, but may also be provided inside the camera head 5.

[0015] One end of the light guide 4 is connected to the light source device 3. The light source device 3 then supplies light to the one end of the light guide 4 under the control of the control device 9. As shown in Figure 1, the light source device 3 comprises a first light source 31, a second light source 32, and a third light source 33.

[0016] The first light source 31 emits first light (broadband light such as white light, or narrowband light such as red light, green light, or blue light) that includes at least a portion of the visible light wavelength band. Examples of the configuration of the first light source 31 include a configuration including a white LED (Light Emitting Diode), or a configuration including three light sources that emit red light, green light, and blue light respectively, and an optical element that combines the red light, green light, and blue light. The first light source 31 may be made of an LED or a semiconductor laser. The number of first light sources 31 may be one or more.

[0017] The second light source 32 emits a first excitation light that excites the substance contained in the object being observed OB. The second light source 32 may be composed of an LED or a semiconductor laser. There may be one or more second light sources 32. Furthermore, the second light source 32 may emit light that includes at least a portion of the visible light wavelength band, or it may emit light in the invisible light wavelength band such as infrared or ultraviolet light.

[0018] The third light source 33 emits a second excitation light to excite the substance contained in the object OB being observed. The third light source 33 may be an LED or a semiconductor laser. There may be one or more third light sources 33. Furthermore, the third light source 33 may emit light that includes at least a portion of the visible light wavelength band, or it may emit infrared or ultraviolet light. In the following, the first and second excitation lights may be collectively referred to as excitation light.

[0019] In this embodiment, a configuration is provided in which three light sources (first to third light sources 31 to 33) are provided to emit the first light and the first and second excitation lights, respectively, but the invention is not limited to this configuration. For example, a configuration may be adopted in which only one light source is provided and the wavelength is switched using a filter or the like to emit the first light and the first and second excitation lights, respectively. Alternatively, for example, a configuration may be adopted in which two light sources are provided and the wavelength is switched using a filter or the like to emit the first light and the first and second excitation lights, respectively.

[0020] Here, examples of substances contained in the observed object OB that are excited by the first and second excitation light include drugs and fluorescent dyes applied to the observed object OB, or fluorescent substances derived from the observed object OB that constitute the observed object OB itself.

[0021] Examples of the above-mentioned drugs administered to the observed OB include "5-ALA (PP-IX)", "ADS780WS", "ADS830WS", "aggregation-induced emission dots allophycocyanin (APC)", "boron-dipyrromethane (BODIPY)", "CLR 1502", "Flavins", "fluorescamine", "Fluorescein", "fluoro-gold", "green fluorescence protein", "ICG (indocyanine green)", "IRDye 78", "IR-PEG nanoparticles", "Isothiocyanate", "rose bengal", "SGM-101", and "trypan blue".

[0022] Furthermore, the fluorescent dyes mentioned above that can be applied to the observed OB include: "coumarine", "Cy3", "DyLight547", "GE3126", "metal nanoclusters", "oxacarbocyanine", "Rhodamine", "Riboflavin", "fluorescein", "AlexaFluor 488", "AlexaFluor660", "AlexaFluor680", "AlexaFluor700", "Cy5", "Cy5.5", "Dy677", "Dy682", "Dy752", "DyLight647", "HiLyte Fluor 647", "HiLyte Fluor 680", "IRDye 700DX", "methylene blue", "Porphyrins", "Porphysomes", "VivoTag-680", "VivoTag-S680", "AlexaFluor750", "AlexaFluor790", "carbocyanine", "conjugated copolymers", "CW800-CA", "Cy7", "Cy7.5", and "cyanine". Examples include "dyes", "Dy780", "HiLyte Fluor 750", "Indocarbocyanine", "IR-786", "IRDye 800CW", "IRDye 800RS", "IRDye 800BK", "Nervelight", "OTL-38 (Pafolacianine)", "Polymethine", "VivoTag-S750", "ASP5354", "Xanthene", and "LUM-015".

[0023] Furthermore, examples of fluorescent substances derived from the observed OB that constitute the observed OB itself include "collagen," "elastin," and "NADH."

[0024] In this embodiment, the light source device 3 is configured separately from the control device 9, but this is not the only option; it may also be configured to be housed in the same housing as the control device 9. Alternatively, the control device 9 may be divided into two units, and the light source device 3 may be housed in the housing of one of the two control devices 9.

[0025] One end of the light guide 4 is detachably connected to the light source device 3. The other end of the light guide 4 is detachably connected to the insertion section 2. The light guide 4 transmits light (first light and first and second excitation light) supplied from the light source device 3 from one end to the other and supplies it to the insertion section 2. The light (first light and first and second excitation light) supplied to the insertion section 2 is emitted from the tip of the insertion section 2 and irradiates the object to be observed OB. When the object to be observed OB is irradiated with the first light, the reflected light of the first light from the object to be observed OB is focused by the optical system in the insertion section 2. Similarly, when the object to be observed OB is irradiated with the first excitation light, the reflected light of the first excitation light from the object to be observed OB is focused by the optical system in the insertion section 2. The reflected light of the first excitation light includes the first excitation light reflected from the observation target OB, as well as the first fluorescence (hereinafter referred to as the first observation target fluorescence) emitted from a substance contained in the observation target OB when the first excitation light irradiates the observation target OB and excites that substance. Furthermore, when the observation target OB is irradiated with a second excitation light, the reflected light of the second excitation light from the observation target OB is focused by the optical system in the insertion section 2. The reflected light of the second excitation light includes the second excitation light reflected from the observation target OB, as well as the second fluorescence (hereinafter referred to as the second observation target fluorescence) emitted from a substance contained in the observation target OB when the second excitation light irradiates the observation target OB and excites that substance. In the following, the first and second observation target fluorescence may be collectively referred to as the observation target fluorescence.

[0026] The camera head 5 is detachably connected to the eyepiece 21 of the insertion unit 2. The insertion unit 2 and the camera head 5 correspond to the imaging device according to this disclosure. Under the control of the control device 9, the camera head 5 captures the reflected light (reflected light of the first light, reflected light of the first excitation light, and reflected light of the second excitation light) from the observation target OB that has been focused by the insertion unit 2, and generates a pixel signal. For the sake of explanation, this pixel signal may be referred to as the captured image below. The detailed configuration of the camera head 5 will be described later in "Configuration of the Camera Head".

[0027] One end of the first transmission cable 6 is detachably connected to the control device 9 via connector CN1 (Figure 1). The other end of the first transmission cable 6 is detachably connected to the camera head 5 via connector CN2 (Figure 1). The first transmission cable 6 transmits captured images and the like output from the camera head 5 to the control device 9, and also transmits control signals, synchronization signals, clock signals, and power signals output from the control device 9 to the camera head 5.

[0028] Furthermore, the captured images and other data transmitted from the camera head 5 to the control device 9 via the first transmission cable 6 may be transmitted as optical signals or as electrical signals. The same applies to the transmission of control signals, synchronization signals, and clock signals from the control device 9 to the camera head 5 via the first transmission cable 6.

[0029] The display device 7 is composed of a display using liquid crystal or organic EL (Electro Luminescence), and under the control of the control device 9, it displays an image based on a video signal from the control device 9.

[0030] One end of the second transmission cable 8 is detachably connected to the display device 7. The other end of the second transmission cable 8 is detachably connected to the control device 9. The second transmission cable 8 transmits the video signal processed by the control device 9 to the display device 7.

[0031] The control device 9 corresponds to the medical control device described herein. This control device 9 is composed of a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), etc., and comprehensively controls the operation of the light source device 3, the camera head 5, and the display device 7. The detailed configuration of the control device 9 will be described later in "Configuration of the Control Device".

[0032] One end of the third transmission cable 10 is detachably connected to the light source device 3. The other end of the third transmission cable 10 is detachably connected to the control device 9. The third transmission cable 10 transmits control signals from the control device 9 to the light source device 3.

[0033] [Camera Head Configuration] Next, the configuration of the camera head 5 will be described. Figure 2 is a block diagram showing the configuration of the camera head 5 and the control device 9. As shown in Figure 2, the camera head 5 comprises a lens unit 51, a prism 52, an imaging unit 53, and a communication unit 54.

[0034] The lens unit 51 is composed of one or more lenses. The lens unit 51 then focuses the reflected light (subject image) from the observation target OB, which has been focused in the insertion section 2, onto the imaging surfaces of the first and second image sensors 531 and 532, respectively.

[0035] The prism 52 separates the reflected light (subject image) from the observation target OB via the lens unit 51 into light of two different wavelength bands: first and second. The light of the first wavelength band is light of a wavelength band excluding at least a portion of the wavelength bands of the first and second observation target fluorescence, and includes at least a portion of the visible light wavelength band. Hereinafter, the light of the first wavelength band will be referred to as the normal subject image. The light of the second wavelength band is light of a wavelength band excluding at least a portion of the wavelength bands of the visible light, and includes at least a portion of the wavelength bands of the first and second observation target fluorescence. Hereinafter, the light of the second wavelength band that includes at least a portion of the wavelength band of the first observation target fluorescence will be referred to as the first fluorescence subject image. Furthermore, the light of the second wavelength band that includes at least a portion of the wavelength band of the second observation target fluorescence will be referred to as the second fluorescence subject image. The prism 52 then propagates the normal subject image toward the first image sensor 531. Furthermore, the prism 52 propels the first and second fluorescent subject images toward the second image sensor 532.

[0036] The imaging unit 53 captures images of the observation target OB under the control of the control device 9. As shown in Figure 2, the imaging unit 53 comprises a first image sensor 531, a second image sensor 532, and a signal processing unit 533.

[0037] The number of image sensors constituting the first image sensor 531 may be one or more. Similarly, the number of image sensors constituting the second image sensor 532 may be one or more.

[0038] The first and second image sensors 531 and 532 receive the subject image and convert it into an electrical signal (analog signal). In this embodiment, the first and second image sensors 531 and 532 are each composed of CMOS (Complementary Metal Oxide Semiconductor), which are rolling shutter type image sensors in which multiple pixels are arranged in a two-dimensional manner in units of horizontal lines. However, the first and second image sensors 531 and 532 are not limited to rolling shutter type image sensors; they may also be composed of CCD (Charge Coupled Device), which are global shutter type image sensors.

[0039] Here, the first image sensor 531, although not shown in detail in the illustration, is composed of an invalid region where the output signal is not used to generate the captured image, an optical black region (OB region), and an effective pixel region where the normal subject image formed by the lens unit 51 is converted into a pixel signal and output. Similarly, the second image sensor 532 is composed of an invalid region, an optical black region (OB region), and an effective pixel region.

[0040] The first image sensor 531 then captures a normal subject image via the prism 52 under the control of the control device 9. That is, the first image sensor 531 captures light that includes at least a portion of the visible light wavelength band. For the sake of explanation, the image generated by capturing a normal subject image will be referred to as a normal observation image below.

[0041] Furthermore, the second image sensor 532, under the control of the control device 9, captures images of the first and second fluorescent subjects via the prism 52. That is, the second image sensor 532 captures light that includes at least a portion of the wavelength band of the first and second observed fluorescence. For the sake of explanation, the image generated by capturing the first fluorescent subject will be referred to as the first fluorescence observation image. The image generated by capturing the second fluorescent subject will be referred to as the second fluorescence observation image.

[0042] Furthermore, the number of pixels in the normal observation image and the number of pixels in the first and second fluorescence observation images may be different or the same.

[0043] Under the control of the control device 9, the signal processing unit 533 performs signal processing on the captured images (analog signals) generated by the first and second imaging elements 531 and 532, and outputs captured images (digital signals). For example, the signal processing unit 533 performs processing to remove reset noise, multiplies the analog gain for amplifying the analog signal (hereinafter referred to as analog gain adjustment), and signal processing such as A / D conversion on the captured images (analog signals) generated by the first and second imaging elements 531 and 532.

[0044] The communication unit 54 functions as a transmitter that sequentially transmits the captured images output from the imaging unit 53 to the control device 9 via the first transmission cable 6. This communication unit 54 is configured by, for example, a high-speed serial interface that communicates the captured images at a transmission rate of 1 Gbps or more between the control device 9 via the first transmission cable 6.

[0045] Note that the communication unit 54 may sequentially transmit the normal observation image, the first fluorescence observation image, and the second fluorescence observation image to the control device 9, or may transmit at least two of these three images simultaneously.

[0046] [Configuration of Control Device] Next, the configuration of the control device 9 will be described while referring to FIG. 2. As shown in FIG. 2, the control device 9 includes a communication unit 91, an image memory 92, a processing module 93, a control unit 94, an input unit 95, an output unit 96, and a storage unit 97.

[0047] The communication unit 91 functions as a receiver that sequentially receives the captured images transmitted from the camera head 5 (communication unit 54) via the first transmission cable 6. This communication unit 91 is configured by, for example, a high-speed serial interface that communicates the captured images at a transmission rate of 1 Gbps or more with the communication unit 54.

[0048] The image memory 92 is configured by, for example, a DRAM (Dynamic Random Access Memory) or the like. This image memory 92 can temporarily store a plurality of frames of the captured images sequentially output from the camera head 5 (communication unit 54).

[0049] The processing module 93 corresponds to the image processing unit according to the present disclosure. Under the control of the control unit 94, this processing module 93 sequentially processes the captured images transmitted from the camera head 5 (communication unit 54) and received by the communication unit 91. As shown in FIG. 2, this processing module 93 includes a memory controller 931, a processing unit 932, and a display control unit 933.

[0050] The memory controller 931 controls the writing of the captured image to the image memory 92 and the reading of the captured image from the image memory 92. The captured image read by the memory controller 931 is input to the processing unit 932.

[0051] The processing unit 932 executes image processing on the input captured image. Examples of such image processing include optical black subtraction processing (clamp processing), white balance adjustment processing, demosaic processing, color correction matrix processing, gamma correction processing, YC processing for converting RGB signals into luminance chrominance signals (Y, Cb / Cr signals), digital gain adjustment for multiplying digital gain, noise removal, filter processing for enhancing structure, and the like.

[0052] Note that the image processing executed on the normal observation image, the image processing executed on the first fluorescence observation image, and the image processing executed on the second fluorescence observation image may all be different image processing, or at least two of the image processing may be the same image processing.

[0053] The display control unit 933 generates a display image (video signal) for display on the display device 7 based on the captured image after the image processing is executed by the processing unit 932 under the control of the control unit 94. Then, the display control unit 933 outputs the video signal to the display device 7 via the second transmission cable 8. Note that the detailed functions of the display control unit 933 will be described in "Functions of the Display Control Unit" described later.

[0054] The control unit 94 is implemented by a controller such as a CPU or MPU (Micro Processing Unit) executing various programs stored in the memory unit 97. It controls the operation of the light source device 3, the camera head 5, and the display device 7, as well as the operation of the entire control device 9. The control unit 94 is not limited to a CPU or MPU; it may also be composed of an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA. As shown in Figure 2, this control unit 94 has the functions of a light source control unit 941, an imaging control unit 942, and a mode switching unit 943.

[0055] The functions of the light source control unit 941 and the imaging control unit 942 will be explained later in the section titled "Functions of the Light Source Control Unit and the Imaging Control Unit." The functions of the mode switching unit 943 will be explained later in the section titled "Functions of the Display Control Unit."

[0056] The input unit 95 is configured using operating devices such as a mouse, keyboard, and touch panel, and accepts user operations from a user such as a surgeon. The input unit 95 then outputs an operation signal corresponding to the user operation to the control unit 94.

[0057] The output unit 96 is configured using a speaker, printer, etc., and outputs various types of information.

[0058] The memory unit 97 stores programs executed by the control unit 94, information necessary for processing by the control unit 94, and so on.

[0059] [Functions of the Light Source Control Unit and the Imaging Control Unit] Next, the functions of the light source control unit 941 and the imaging control unit 942 will be described. Figures 3 to 5 are diagrams illustrating the functions of the light source control unit 941 and the imaging control unit 942. Specifically, Figure 3 is a diagram illustrating the issues that arise from the light source control and imaging control assumed when generating a normal observation image and the first and second fluorescence observation images. Figure 4 is a diagram illustrating the light source control and imaging control of this embodiment. Figure 5 is a diagram illustrating the image generated by the light source control and imaging control of this embodiment. Here, Figures 3(a), 4(a), and 5(a) are diagrams showing the imaging control of the first image sensor 531, where the vertical axis shows the horizontal line of the first image sensor 531 (the top row shows the uppermost horizontal line (the first horizontal line), and the bottom row shows the lowermost horizontal line (the final line)), and the horizontal axis shows time. The parallelogram region is the region that contributes to the generation of a normal observation image in one field. Figures 3(b), 4(b), and 5(b) show the light source control, where the vertical axis represents the power value [W] supplied to the first light source 31 and the horizontal axis represents time (the power supply time to the first light source 31). In this embodiment, since the voltage value supplied to the first light source 31 is fixed, in Figures 3(b), 4(b), and 5(b), the vertical axis corresponds to the current value supplied to the first light source 31. Figures 3(c), 4(c), and 5(c) show the imaging control of the second image sensor 532, where the vertical axis represents the horizontal line of the second image sensor 532 (the top row represents the uppermost horizontal line (the first horizontal line), and the bottom row represents the lowermost horizontal line (the final line)) and the horizontal axis represents time. The parallelogram region is the region that contributes to the generation of the first and second fluorescence observation images in one field. Figures 3(d) and 4(d) show the light source control, with the vertical axis representing the power value [W] supplied to the second light source 32 and the horizontal axis representing time (the power supply time to the second light source 32). In this embodiment, since the voltage value supplied to the second light source 32 is fixed, in Figures 3(d) and 4(d), the vertical axis corresponds to the current value supplied to the second light source 32.Figures 3(e) and 4(e) show the light source control, with the vertical axis representing the power value [W] supplied to the third light source 33 and the horizontal axis representing time (the power supply time to the third light source 33). In this embodiment, since the voltage value supplied to the third light source 33 is fixed, in Figures 3(e) and 4(e), the vertical axis corresponds to the current value supplied to the third light source 33. In Figure 5, the symbols "WLI", "WLI1" to "WLI4" represent the captured images generated by the first image sensor 531, respectively. Also in Figure 5, the symbols "FL1", "FL2", "FL11", "FL21", "FL12", and "FL22" represent the captured images generated by the second image sensor 532, respectively. Furthermore, in Figure 5, the symbol "ArF1" represents the region in the captured image FL1 where the intensity of the first observed fluorescence is strong. Furthermore, in Figure 5, the symbol "ArF2" indicates a region in the captured image FL2 where the intensity of the second observed fluorescence is high.

[0060] First, referring to Figure 3, we will explain the light source control and imaging control assumed when generating the normal observation image and the first and second fluorescence observation images, and the problems arising from said light source control and imaging control. The imaging control unit 942 performs imaging control using a so-called rolling shutter method, in which exposure for one field period of the first and second image sensors 531 and 532 is sequentially started for each horizontal line, and readout is performed sequentially for each horizontal line after a predetermined period (so-called shutter speed) has elapsed from the start of exposure. In addition, in this imaging control, in the case of the NTSC system, one field is set to 1 / 60 [s] (Figure 3(a) and Figure 3(c)). Note that in the case of the NTSC system, 1 / 60 [s] may be used as the reference, but in the case of the PAL system, one field may be set to 1 / 50 [s], or a high-speed imaging operation (for example, 1 / 240 [s]) may be used as the reference. Furthermore, although the first and second image sensors 531 and 532 are read out at 1 / 120 [s], the readout speed of the first and second image sensors 531 and 532 is not limited to this.

[0061] As shown in Figure 3(b), the light source control unit 941 performs light source control to keep the first light source 31 continuously lit. In this light source control, the current value supplied to the first light source 31 is adjusted based on the brightness (average value of luminance, etc.) of a specific region (detection region) in the normal observation image to adjust the normal observation image to a reference brightness.

[0062] Furthermore, as shown in Figures 3(d) and 3(e), the light source control unit 941 controls the second and third light sources 32 and 33. In this embodiment, since the first and second fluorescent subject images are captured by the second image sensor 532, the light source control alternately emits the first and second excitation light in a time-division manner. Also, to prevent one of the first and second fluorescent subject images from being captured in the other frame, the light source control illuminates the first and second excitation light only during the full line exposure period TE of the second image sensor 532. In addition, the light source control adjusts the current values ​​supplied to the second and third light sources 32 and 33 in order to adjust the first and second fluorescent observation images to a reference brightness based on the brightness (average value of luminance, etc.) of a specific region (detection region) in the normal observation image.

[0063] Hereinafter, the optical path of the first light and the first and second excitation lights following the path from the light source device 3 to the light guide 4 to the insertion unit 2 to the observation target OB will be referred to as the first optical path P1 (Figure 1). Furthermore, the optical path of the return light from the observation target OB (return light of the first light, return light of the first excitation light, return light of the second excitation light) following the path from the observation target OB to the insertion unit 2 to the first and second image sensors 531 and 532 will be referred to as the second optical path P2 (Figure 1). In addition, the first and second optical paths P1 and P2 are combined and referred to as the observation optical path P0 (Figure 1). When the first light and the first and second excitation lights propagate through the first optical path P1, and when the reflected light from the observation target OB propagates through the second optical path P2, the member forming the observation optical path P0 (hereinafter referred to as the autofluorescence generating member) is irradiated with the first light, the first and second excitation lights, and the reflected light from the observation target OB, causing autofluorescence (hereinafter referred to as unwanted light) to be generated from the autofluorescence generating member. Examples of the autofluorescence generating member include components contained in multi-component glass such as lenses, organic material components contained in color filters, adhesives used to join lenses, and oil adhering to optical elements such as lenses. Such unwanted light has a wavelength band that includes the wavelength band of the observed fluorescence, which is the light to be observed in fluorescence observation, and becomes noise in the fluorescence observation. Such unwanted light is also generated when the observation target OB is irradiated with the first light or the first and second excitation lights.

[0064] Furthermore, if the fluorescence of the target object OB is weak, it is necessary to adjust the signal value based on the fluorescence of the target object captured by the second image sensor 532 in order to separate the fluorescence of the target object from the unwanted light mentioned above and perform good fluorescence observation. However, adjusting the signal value based on the fluorescence of the target object is difficult because it is affected by the following (1) to (4).

[0065] (1) Drugs Generally, the amount of light emitted from a drug that is the target of observation varies depending on the type and dosage of the drug. The type of drug administered to the target OB is selected according to the target OB (cancer, blood, lymph, etc.). Furthermore, in order to image the target of observation emitted from the drug in the medical observation system 1, a drug is selected in which the wavelength of the excitation light that excites the drug and the wavelength of the target of observation emitted from the drug can be separated. The drugs selected in this way each emit different amounts of light for the target of observation. In addition, the amount of light emitted for the target of observation can be adjusted by the dosage of the drug, but it is difficult to increase the dosage more than necessary in order to realize a minimally invasive procedure. In other words, it is difficult to adjust the amount of light emitted for the target of observation by selecting the type of drug and adjusting the dosage. As a result, it is difficult to adjust the signal value based on the target of observation captured by the second image sensor 532 by selecting the type of drug and adjusting the dosage.

[0066] (2) The amount of fluorescence light emitted from the observed OB varies depending on the location and condition of the observed OB. Specifically, in the case of an observed OB in a location or condition where the drug tends to accumulate, the amount of fluorescence light emitted from the observed OB increases. On the other hand, in the case of an observed OB in a location or condition where the drug flows easily and does not accumulate easily, the amount of fluorescence light emitted from the observed OB decreases, and the afterglow time also shortens. Furthermore, if the observed OB is a tumor, the amount of fluorescence light received by the second image sensor 532 changes depending on its extent, size, and depth. In other words, it is difficult to adjust the amount of fluorescence light emitted from the observed OB depending on the type and condition of the observed OB. As a result, it is difficult to adjust the signal value based on the fluorescence light captured by the second image sensor 532 depending on the type and condition of the observed OB.

[0067] (3) Light Source Device The amount of fluorescence observed varies depending on the amount of excitation light emitted from the light source device 3. Increasing the amount of excitation light may require adjusting the power supplied to the light source device 3. However, the amount of power that can be supplied to the light source device 3 is limited according to the upper limit of the power required to operate the entire medical observation system 1. Furthermore, the amount of excitation light must be adjusted considering factors such as heat generation in the components constituting the optical path of the excitation light (for example, heat generation between the light guide 4 and the insertion part 2), compatibility with the laser class, the amount of light energy received by the observation target OB and surrounding biological tissue (high light energy levels pose a risk of burns), or the rate of fading of the fluorescence observed from the drug. In addition, the amount of excitation light can also be adjusted by changing the number of excitation light sources mounted on the light source device 3. However, the number of light sources may affect the size of the light source device 3. The size of the light source device 3 may be limited by the size of the cart used to transport the light source device 3, etc. In other words, it is difficult to adjust the amount of fluorescence observed by adjusting the amount of excitation light. As a result, it is difficult to adjust the signal value based on the observed fluorescence image captured by the second image sensor 532 by adjusting the intensity of the excitation light.

[0068] (4) Depending on the amount of light from the target fluorescence received by the second image sensor 532, the signal value based on the target fluorescence generated from the second image sensor 532 will differ. To adjust the signal value based on the target fluorescence, it is desirable to select a second image sensor 532 that has the optimal sensitivity and configuration for imaging the target fluorescence. However, the second image sensor 532 may be required not only to output an image for fluorescence observation based on the reception of target fluorescence in a predetermined wavelength band, but also to output an image for normal light observation based on the reception of visible light such as white light. Furthermore, the second image sensor 532 may be required to output an image for fluorescence observation that corresponds to a wide wavelength band or multiple wavelength bands within the wavelength band including visible and invisible light. In addition, even when using the same drug, the amount of light from the target fluorescence may change depending on the procedure and the target OB, and the second image sensor 532 may be required to output an image for fluorescence observation that corresponds to such changes in the amount of light from the target fluorescence. In that case, the second image sensor 532 must be selected to be an element capable of observing those elements, and it may not be possible to use an image sensor with characteristics optimal for imaging fluorescence in a predetermined wavelength band. Furthermore, although the second image sensor 532 is disposed within the camera head 5, there are size and weight requirements for the camera head 5 suitable for observation, which may limit the types, including the size, of the second image sensor 532. Moreover, even when the second image sensor 532 is disposed not only within the camera head 5, but also at the tip of a rigid or flexible endoscope, there are size and weight requirements for observation, which may limit the types, including the size, of the second image sensor 532. In other words, it is difficult to adjust the signal value based on the observed fluorescence imaged by the second image sensor 532 by selecting the type of second image sensor 532.

[0069] As described above, the signal value based on the observed fluorescence image captured by the second image sensor 532 is determined within the above constraints and therefore cannot be easily adjusted.

[0070] Furthermore, in the light source control shown in Figure 3, in order to reduce unwanted light generated by the first light, it is necessary to lower the light intensity of the first light (the current value supplied to the first light source 31). However, the first light illuminates the object of observation OB and forms the background of the display image used during fluorescence observation. Therefore, if the light intensity of the first light is lowered too much, the brightness of the display image will become dim, and furthermore, if digital gain adjustment is performed to brighten the dim display image, the image quality may deteriorate.

[0071] Incidentally, the fluorescence observed can range in intensity from very strong to very weak depending on the drug or other substance. There may also be a significant difference in intensity between the first and second fluorescence observed. In other words, if there is a difference in intensity between the first and second fluorescence observed, the appropriate amount of light for the first light emitted simultaneously with the first fluorescence will be different from the appropriate amount of light for the first light emitted simultaneously with the second fluorescence.

[0072] The following describes the light source control and imaging control in this embodiment. The first excitation light corresponds to the first drug. The second excitation light corresponds to the second drug. The intensity of the first target fluorescence emitted from the first drug upon irradiation with the first excitation light is stronger than the intensity of the second target fluorescence emitted from the second drug upon irradiation with the second excitation light.

[0073] The imaging control of the first and second image sensors 531 and 532 in this embodiment is the same as the imaging control described in Figures 3(a) and 3(c), as shown in Figures 4(a) and 4(c). The light source control of the second and third light sources 32 and 33 in this embodiment is the same as the light source control described in Figures 3(d) and 3(e), as shown in Figures 4(d) and 4(e). The light source control of the first light source 31 in this embodiment is different from the light source control described in Figure 3(c), as shown in Figure 4(b).

[0074] Specifically, the light source control unit 941 performs, in a specific order, a first control that sets the amount of light emitted from the first light source 31 to a first light amount (corresponding to the first luminescence energy according to this disclosure), and a second control that sets the amount of light of the first light to a second light amount (corresponding to the second luminescence energy according to this disclosure) which is less than the first light amount. Here, the luminescence energy corresponds to the light amount (current value supplied to the first light source 31) and the PWM value of the first light source 31 (luminescence time). For this reason, the luminescence time may be considered as the first and second luminescence energies to be set in the first and second controls, in addition to the light amount. Details of the first and second light amounts will be explained later in "About the First and Second Light Amounts".

[0075] In this embodiment, the first control is performed during the same period as the emission period of the first excitation light. In this first control, as shown in Figure 4(b), the current value supplied to the first light source 31 is set to a first current value α[W] in order to set the light intensity of the first light to a first light intensity. The second control is performed during the period when the first control is not being performed. That is, the second control is performed during the period that includes the emission period of the second excitation light. In this second control, the current value supplied to the first light source 31 is set to a second current value α'[W] which is lower than the first current value α[W] and greater than 0 in order to set the light intensity of the first light to a second light intensity.

[0076] Based on the above, in this embodiment, the light source control unit 941 repeatedly and alternately executes the first control and the second control every frame. Note that when the medical control device according to this disclosure is combined with an open-field observation device used in abdominal surgery, the user may see the flashing of the first light caused by the first and second controls. Therefore, it is undesirable to make the period of the first control unnecessarily large. In this case, it is preferable that the period of the first control be 1 / 120 [s] or less.

[0077] As shown in Figure 4, with the light source control and imaging control, during the first control phase, the first image sensor 531 generates normal observation images WLI (WLI1, WLI3) as shown in Figure 5. Also during the first control phase, the second image sensor 532 generates the first fluorescence observation images FL1 (FL11, FL12).

[0078] Furthermore, during the second control phase, as shown in Figure 5, the first image sensor 531 generates the normal observation images WLI (WLI2, WLI4) through the light source control and imaging control shown in Figure 4. During the second control phase, the second image sensor 532 generates the second fluorescence observation images FL2 (FL21, FL22). In other words, the second image sensor 532 alternately generates the first fluorescence observation image FL1 and the second fluorescence observation image FL2 in a time-division manner.

[0079] The normal observation images WLI1 and WLI3 described above correspond to the first normal images relating to this disclosure. The first fluorescence observation image FL1 corresponds to the first fluorescence image relating to this disclosure. The normal observation images WLI1 and WLI3 and the first fluorescence observation image FL1 correspond to the first captured image relating to this disclosure. Furthermore, the normal observation images WLI2 and WLI4 correspond to the second normal images relating to this disclosure. Furthermore, the second fluorescence observation image FL2 corresponds to the second fluorescence image relating to this disclosure. The normal observation images WLI2 and WLI4 and the second fluorescence observation image FL2 correspond to the second captured image relating to this disclosure.

[0080] [Functions of the Display Control Unit] Next, the functions of the display control unit 933 will be explained. Figures 6 and 7 illustrate the functions of the display control unit 933. First, the functions of the mode switching unit 943 will be explained. The mode switching unit 943 switches the medical observation system 1 to either the first mode or the second mode in response to user operation to the input unit 95. Here, the first mode is a mode that uses two types of normal observation images generated during the first and second control cycles, respectively. On the other hand, the second mode is a mode that uses only one type of normal observation image generated during the first control cycle out of the two control cycles.

[0081] The display control unit 933 then generates different display images for the first mode and the second mode. The display images generated in the first mode and the display images generated in the second mode will be described in order below.

[0082] [Regarding the display images generated in the first mode] First, the display images generated in the first mode will be explained with reference to Figure 6. As shown in Figure 6, the display control unit 933 sequentially generates display images DF1 (DF11, DF12, DF13) at a specific first frame rate (for example, a period of 1 / 60 [s]). Display image DF11 is a superimposed image obtained by superimposing the normal observation image WLI2, whose brightness has been increased by analog gain adjustment by the signal processing unit 533 and digital gain adjustment by the processing unit 932, with the first and second fluorescence observation images FL11 and FL21. Display image DF12 is a superimposed image obtained by superimposing the normal observation image WLI3 with the first and second fluorescence observation images FL12 and FL21. The displayed image DF13 is a superimposed image obtained by superimposing the normal observation image WLI4, whose brightness has been increased by analog gain adjustment by the signal processing unit 533 and digital gain adjustment by the processing unit 932, with the first and second fluorescence observation images FL12 and FL22.

[0083] Note that the display image DF1 is not limited to the image described above, but may also be the image shown below. Display image DF11 is a superimposed image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL121 and FL22 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL22 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL22 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and FL21 onto an image obtained by superimposing the first and second fluorescence observation images FL11 and

[0084] [Regarding the display images generated in the second mode] Next, the display images generated in the second mode will be described with reference to Figure 7. As shown in Figure 7, the display control unit 933 sequentially generates display images DF2 (DF21, DF22, DF23) at a second frame rate lower than the first frame rate (for example, a period of 1 / 30 [s]). Display image DF21 is a superimposed image obtained by superimposing the normal observation image WLI1 and the first and second fluorescence observation images FL11 and FL21. Display image DF22 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12 and FL21. Display image DF23 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12 and FL22.

[0085] [Regarding the First and Second Light Intensities] Next, the first and second light intensities will be explained. The first light intensity (first current value α [W]) used in the first control is set to an intensity (current value) such that, in the first control, the signal value based on the first observed target fluorescence is greater than the signal value based on unwanted light captured by the second image sensor 532. In other words, the first light intensity (first current value α [W]) used in the first control is set to an intensity (current value) such that, in the first control, the signal-to-noise ratio of the signal value based on the first observed target fluorescence captured by the second image sensor 532 is greater than 1.

[0086] Furthermore, the second light intensity (second current value α[W]) used in the second control is set in the first control to an intensity (current value) such that the signal value based on the second observed target fluorescence is greater than the signal value based on unwanted light captured by the second image sensor 532. In other words, the second light intensity (second current value α[W]) used in the first control is set in the first control to an intensity (current value) such that the signal-to-noise ratio (S / N ratio) of the signal value based on the second observed target fluorescence captured by the second image sensor 532 is greater than 1. The method for calculating the S / N ratio will be explained later in "Method for Calculating the S / N Ratio".

[0087] Figure 8 illustrates a method for separating the signal value based on the observed fluorescence from the signal value based on unwanted light when the signal-to-noise ratio (S / N ratio) of the signal value based on the observed fluorescence is greater than 1. For example, when the S / N ratio of the signal value based on the observed fluorescence captured by the second image sensor 532 is greater than 1, as shown in Figure 8, the signal value based on the observed fluorescence can be further amplified by image processing by the processing unit 932, thereby separating the signal value based on the observed fluorescence from the signal value based on unwanted light.

[0088] Specifically, in the clamping process, the processing unit 932 separates the signal value based on unwanted light from the signal value based on the fluorescence of the object being observed by raising the level at which the black line is cut off (clamp value), as shown in Figures 8(a) to 8(c).

[0089] However, as shown in Figure 8(c), the signal level of the signal after increasing the clamp value becomes lower than the original state.

[0090] Therefore, as shown in Figure 8(d), the processing unit 932 increases the signal level of the signal shown in Figure 8(c) by digital gain adjustment.

[0091] [Regarding the method for calculating the signal-to-noise ratio] Figure 9 is a diagram illustrating the method for calculating the signal-to-noise ratio. Specifically, Figure 9 corresponds to Figure 1. The method for calculating the signal-to-noise ratio according to this embodiment is based on the methods described in "2.D.8. Excitation light crosstalk" and "3.G. Excitation light crosstalk" in the document titled "Performance test methods for near-infrared fluorescence imaging" in the following document A. Below, the method for calculating the signal-to-noise ratio when setting the first light intensity (first current value α [W]) will be explained. For the method for calculating the signal-to-noise ratio when setting the second light intensity (second current value α [W]), simply replace "first drug," "second light source 32," and "first excitation light" with "second drug," "third light source 33," and "second excitation light," respectively, as described below. (Reference A) "MEDICAL PHYSICS" Volume 47, Issue 8, August 2020, Pages 3389-3401

[0092] In the method for calculating the signal-to-noise ratio, as shown in Figure 9, first and second subjects (phantoms) F1 and F2 are used. The first subject F1 is a subject (phantom) that contains only scattering particles and no drug. The second subject F2 is a subject (phantom) that contains the first drug at an appropriate concentration for the biological organism to be observed.

[0093] The signal-to-noise ratio (S / N ratio) is calculated using the following formula (1): (Equation 1) S / N ratio = ((c) - (a)) / ((b) - (a)) ... (1)

[0094] Here, (a) of equation (1) is the brightness level based on the pixel signal output from the second image sensor 532 in the medical observation system 1 shown in Figure 9, with no ambient light entering the camera head 5 and with the first and second light sources 31 and 32 turned off. Note that (a) may also be the brightness level based on the pixel signal output from the second image sensor 532 in the medical observation system 1 shown in Figure 9, with a mechanical shutter placed between the camera head 5 and the eyepiece 21, and with the mechanical shutter preventing light from entering the camera head 5. Furthermore, (b) is the brightness level based on the pixel signal output from the second image sensor 532 when the first and second light sources 31 and 32 are turned on, the first light and the first excitation light are irradiated onto the first subject F1, and the reflected light from the first subject F1 is captured by the second image sensor 532. Furthermore, (c) is the brightness level based on the pixel signal output from the second image sensor 532 when the first and second light sources 31 and 32 are turned on in the medical observation system 1 shown in Figure 9, the first light and the first excitation light are irradiated onto the second subject F2, and the reflected light from the second subject F2 is captured by the second image sensor 532.

[0095] The first light quantity (first current value α[W]) according to this disclosure is preferably a light quantity (current value) such that the signal-to-noise ratio calculated by formula (1) using the pixel signal (RAW signal) before image processing is performed on the pixel signal output from the second image sensor 532 is greater than 1. Alternatively, the first light quantity (first current value α[W]) according to this disclosure is preferably a light quantity (current value) such that the signal-to-noise ratio calculated by formula (1) using the pixel signal after image processing is performed on the pixel signal output from the second image sensor 532 is 4 or more.

[0096] The embodiment described above provides the following effects. The control device 9 according to this embodiment executes the first and second controls in a specific order. The control device 9 emits a first excitation light during the first control. Furthermore, the control device 9 emits a second excitation light during the second control. The control device 9 generates a first image when the first control is executed and a second image when the second control is executed. The control device 9 then generates a display image based on the first and second image images. As a result, it is possible to observe not only one type of target fluorescence but also multiple types of target fluorescence in different wavelength bands. In particular, during the first control, the emission energy of the first light is set to the first emission energy, and during the second control, the emission energy of the first light is set to the second emission energy. As a result, it is possible to generate first and second fluorescence observation images FL1 and FL2 that reduce the influence of unwanted light caused by the first light, and the normal observation images WLI1 and WLI3 that serve as the background of the display image can be made to an appropriate brightness by the first light. Therefore, according to the control device 9 of this embodiment, multiple types of target fluorescence in different wavelength bands can be observed well.

[0097] Furthermore, the control device 9 according to this embodiment switches between a first mode and a second mode in response to user operation on the input unit 95. Therefore, it is possible to switch to a first mode that uses two types of normal observation images generated during the first and second control processes, or to a second mode that uses only one type of normal observation image generated during the first control process, thereby improving convenience.

[0098] (Other Embodiments) Up to this point, embodiments for implementing the present disclosure have been described, but the present disclosure should not be limited to the embodiments described above. In the embodiments described above, the following modifications 1 to 8 may also be adopted.

[0099] (Modification 1) Figure 10 is a diagram illustrating modification 1 of the embodiment. Specifically, Figure 10 corresponds to Figure 4. In the embodiment described above, the light source control unit 941 was set to a second light intensity (second current value α' [W]) greater than 0 during the period when the first control is not being performed (including the period when the second control is being performed), but it is not limited to this. For example, the light source control unit 941 may set the second light intensity (second current value α' [W]) to 0 during the period when the first control is not being performed (including the period when the second control is being performed), as shown in modification 1 in Figure 10(b).

[0100] Even when adopting the configuration of the modified example 1 described above, the same effects as those of the embodiment described above are achieved. Furthermore, it is possible to generate a second fluorescence observation image that minimizes the influence of unwanted light caused by the first light.

[0101] (Modification 2) Figures 11 to 13 illustrate Modification 2 of the embodiment. Specifically, Figure 11 corresponds to Figure 4. Figure 12 corresponds to Figure 5. Figure 13 corresponds to Figure 7. In the embodiment described above, light (second fluorescence subject image) including at least a part of the wavelength band of the second fluorescence to be observed is propagated by the prism 52 toward the second image sensor 532 and imaged by the second image sensor 532, but is not limited to this. In this Modification 2, the second fluorescence subject image is propagated by the prism 52 toward the first image sensor 531 and imaged by the first image sensor 531.

[0102] The light source control and imaging control in this modified example 2 are the same as those in the modified example 1 described above, as shown in Figure 11. During the first control phase, the first image sensor 531 generates the normal observation images WLI (WLI1, WLI3) as shown in Figure 12, using the light source control and imaging control shown in Figure 11. During the first control phase, the second image sensor 532 generates the first fluorescence observation images FL1 (FL11, FL12).

[0103] Furthermore, during the second control phase, as shown in Figure 12, the first image sensor 531 generates a second fluorescence observation image FL2 (FL21, FL22) through the light source control and imaging control shown in Figure 11. That is, the first image sensor 531 alternately generates the normal observation image WLI and the second fluorescence observation image FL2 in a time-division manner. Note that during the second control phase, the image captured by the second image sensor 532 is not used as a display image.

[0104] The normal observation images WLI1 and WLI3 described above correspond to the first normal images relating to this disclosure. The first fluorescence observation image FL1 corresponds to the first fluorescence image relating to this disclosure. The normal observation images WLI1 and WLI3 and the first fluorescence observation image FL1 correspond to the first captured image relating to this disclosure. Furthermore, the second fluorescence observation image FL2 corresponds to the second fluorescence image relating to this disclosure. The second fluorescence observation image FL2 corresponds to the second captured image relating to this disclosure.

[0105] In this modified example 2, a normal observation image is generated only during the first control phase, out of the first and second control phases. Therefore, the first mode, which uses two types of normal observation images generated during the first and second control phases, cannot be executed. In other words, in this modified example 2, only the second mode is executed out of the first and second modes.

[0106] The display control unit 933 then generates display images as shown below. As shown in Figure 13, the display control unit 933 sequentially generates display images DF2 (DF21, DF22, DF23) at a second frame rate (for example, a period of 1 / 30 [s]). Display image DF21 is a superimposed image obtained by superimposing the normal observation image WLI1 and the first and second fluorescence observation images FL11 and FL21. Display image DF22 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12 and FL21. Display image DF23 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12 and FL22.

[0107] Even when adopting the configuration of the modified example 2 described above, the same effects as those of the embodiment described above are achieved.

[0108] (Modification 3) Figures 14 to 17 illustrate modification 3 of the embodiment. Specifically, Figure 14 corresponds to Figure 1. Figures 15(a) to 15(d) and 15(f) correspond to Figures 4(a) to 4(e), respectively. Figure 15(e) is a diagram showing light source control, where the vertical axis shows the power value [W] supplied to the fourth light source 34, and the horizontal axis shows time (the supply time of power supplied to the fourth light source 34). In this modification 3, since the voltage value supplied to the fourth light source 34 is fixed, in Figure 15(e), the vertical axis corresponds to the current value supplied to the fourth light source 34. Figure 16 corresponds to Figure 5. Figure 17 corresponds to Figure 6. Figure 18 corresponds to Figure 7. In the embodiment described above, two types of drugs were used to acquire the first and second target fluorescence, which are the fluorescence to be observed, but the embodiment is not limited to this. As shown in Modification 3, a configuration may be adopted in which three types of drugs are used to acquire the first to third target fluorescence, which are the fluorescence to be observed.

[0109] In this modified example 3, as shown in Figure 14, the light source device 3 has a fourth light source 34 in addition to the first to third light sources 31 to 33. The fourth light source 34 emits a third excitation light that excites the substance contained in the observation target OB. The fourth light source 34 may be composed of an LED or a semiconductor laser. The number of fourth light sources 34 may be one or more. Furthermore, the fourth light source 34 may emit light that includes at least a part of the visible light wavelength band, or it may emit light in the invisible light wavelength band such as infrared light or ultraviolet light. Hereafter, the fluorescence emitted from the substance contained in the observation target OB when the third excitation light is irradiated onto the observation target OB and the substance is excited will be referred to as the third observation target fluorescence. In Figure 16, the symbol "ArF3" indicates a region in the captured image FL2 where the intensity of the third observation target fluorescence is strong.

[0110] The third excitation light corresponds to the third drug. The intensity of the observed fluorescence decreases in the following order: the intensity of the first observed fluorescence, the intensity of the third observed fluorescence emitted from the third drug upon irradiation with the third excitation light, and the intensity of the second observed fluorescence.

[0111] In this modified example 3, light containing at least a portion of the wavelength band of the second target fluorescence (second fluorescence subject image) is directed toward the first image sensor 531 by the prism 52 and imaged by the first image sensor 531. On the other hand, light containing at least a portion of the wavelength band of the first target fluorescence (first fluorescence subject image) and light containing at least a portion of the wavelength band of the third target fluorescence (third fluorescence subject image) are directed toward the second image sensor 532 by the prism 52 and imaged by the second image sensor 532.

[0112] The imaging control in this modified example 3 is the same as the imaging control in the modified example 1 described above, as shown in Figure 15. The light source control in this modified example 3 is as follows.

[0113] In this embodiment, the light source control unit 941 repeatedly executes a control pattern in which it executes the first control followed by the second control for two consecutive frames.

[0114] Specifically, as shown in Figure 15(b), the light source control unit 941 sets the light intensity of the first light to the first light intensity (first current value α[W] (corresponding to the first luminescence energy according to this disclosure)) in the first control (the entire line exposure period TE of the second image sensor 532). Furthermore, in the second control (the entire line exposure period TE of the second image sensor 532) of the first of the two consecutive frames following the first control, the light source control unit 941 sets the light intensity of the first light to the second light intensity (second current value α'[W] (corresponding to the second luminescence energy according to this disclosure)) in the second of the two consecutive frames following the first control (the entire line exposure period TE of the second image sensor 532). In addition, in the second control (the entire line exposure period TE of the second image sensor 532) of the second of the two consecutive frames following the first control, the light source control unit 941 sets the light intensity of the first light to 0. Furthermore, the light source control unit 941 sets the light intensity of the first light to 0 even outside of the period during which the first and second controls are being executed (the entire line exposure period TE of the second image sensor 532).

[0115] Furthermore, as shown in Figure 15(d), the light source control unit 941 emits a first excitation light while the first control is being executed. In addition, as shown in Figure 15(e), the light source control unit 941 emits a third excitation light while the second control is being executed in the first of the two consecutive frames following the first control. Furthermore, as shown in Figure 15(f), the light source control unit 941 emits a second excitation light while the second control is being executed in the second of the two consecutive frames following the first control.

[0116] Then, through the light source control and imaging control shown in Figure 15, during the first control phase, the first image sensor 531 generates normal observation images WLI (WLI1, WLI3) as shown in Figure 16. Also during the first control phase, the second image sensor 532 generates the first fluorescence observation images FL1 (FL11, FL12).

[0117] Furthermore, during the second control of the first of the two consecutive frames following the first control, the first image sensor 531 generates a normal observation image WLI (WLI2, WLI4) as shown in Figure 16, through the light source control and imaging control shown in Figure 15. Also, during the second control of the first frame, the second fluorescence observation image FL2 (FL21, FL23) is generated by imaging of the third fluorescence subject image by the second image sensor 532.

[0118] Furthermore, as shown in Figure 15, during the second control of the second frame of the two consecutive frames following the first control, a second fluorescence observation image FL2 (FL22, FL24) is generated by capturing a second fluorescence subject image with the first image sensor 531, as shown in Figure 16. Note that during the second control of the second frame, the image captured by the second image sensor 532 is not used as a display image.

[0119] The normal observation images WLI1 and WLI3 described above correspond to the first normal images relating to this disclosure. The first fluorescence observation image FL1 corresponds to the first fluorescence image relating to this disclosure. The normal observation images WLI1 and WLI3 and the first fluorescence observation image FL1 correspond to the first captured image relating to this disclosure. Furthermore, the second fluorescence observation images FL22 and FL24 correspond to the second fluorescence image relating to this disclosure. The normal observation images WLI2 and WLI4 and the second fluorescence observation image FL2 correspond to the second captured image relating to this disclosure.

[0120] Then, when the display control unit 933 is set to the first mode, it sequentially generates display images DF1 (DF11, DF12, DF13) as shown in Figure 17. Display image DF11 is a superimposed image obtained by superimposing the normal observation image WLI2, whose brightness has been increased by digital gain adjustment by the processing unit 932, and the first and second fluorescence observation images FL11, FL21, and FL22. Display image DF12 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12, FL21, and FL22. Display image DF13 is a superimposed image obtained by superimposing the normal observation image WLI4, whose brightness has been increased by digital gain adjustment by the processing unit 932, and the first and second fluorescence observation images FL12, FL22, and FL23.

[0121] Note that the display image DF1 is not limited to the image described above, but may also be the image shown below. Display image DF11 is a superimposed image obtained by combining the normal observation images WLI1 and WLI2 using HDR (High Dynamic Range) and superimposing the first and second fluorescence observation images FL11, FL21, and F22. Display image DF12 is a superimposed image obtained by combining the normal observation images WLI2 and WLI3 using HDR and superimposing the first and second fluorescence observation images FL12, FL21, and FL22. Display image DF13 is a superimposed image obtained by combining the normal observation images WLI3 and WLI4 using HDR and superimposing the first and second fluorescence observation images FL12, FL22, and FL23.

[0122] Furthermore, when the display control unit 933 is set to the second mode, it sequentially generates display images DF2 (DF21, DF22, DF23) as shown in Figure 18. Display image DF21 is a superimposed image obtained by superimposing the normal observation image WLI1 and the first and second fluorescence observation images FL11, FL21, and FL22. Display image DF22 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12, FL21, and FL22. Display image DF23 is a superimposed image obtained by superimposing the normal observation image WLI3 and the first and second fluorescence observation images FL12, FL22, and FL23.

[0123] Even when adopting the configuration of the modified example 3 described above, the same effects as those of the embodiment described above are achieved.

[0124] In the embodiment described above, the first and second target fluorescence were imaged by the second image sensor 532. Hereinafter, the target fluorescence imaged by the second image sensor 532 will be referred to as invisible fluorescence. In other words, the embodiment described above is configured to handle the case where there are two invisible fluorescence sources.

[0125] Furthermore, in the modified example 2 described above, the first image sensor 531 was used to image the second target fluorescence, and the second image sensor 532 was used to image the first target fluorescence. Hereafter, the target fluorescence imaged by the first image sensor 531 will be referred to as visible fluorescence. In other words, the modified example 2 described above is a configuration that corresponds to the case where there is one invisible fluorescence and one visible fluorescence.

[0126] Furthermore, in the modified example 3 described above, the first image sensor 531 captured the second target fluorescence, and the second image sensor 532 captured the first and third target fluorescence. In other words, this configuration corresponds to the case where there are two invisible fluorescence and one visible fluorescence.

[0127] The number of invisible fluorescent elements and the number of visible fluorescent elements are not limited to the embodiments and modifications 2 and 3 described above, but may be any other number, such as three invisible fluorescent elements. Even in such a configuration, the display image can be generated based on the first and second captured images according to this disclosure.

[0128] In the embodiments and modifications 1 and 2 described above, the light source control unit 941 repeatedly performed the first control and the second control alternately. In the modification 3 described above, the light source control unit 941 repeatedly performed a control pattern in which the first control was performed followed by the second control for two consecutive frames. In other words, the order in which the first control and the second control are performed differs depending on the number of types of drugs used. For this reason, the order in which the first control and the second control are performed is not limited to the order described in the embodiments and modifications 1 to 3 described above, but may be performed in any other specific order.

[0129] (Modification 4) The configuration described in the above-described embodiment and Modifications 1 to 3 may also be applied to surgeries, etc., using multiple drugs as shown in Table 1 below. In Table 1, drugs that are used are marked with "○" and drugs that are not used are marked with "×".

[0130]

[0131] Here, fluorescein emits a target fluorescence of approximately 520 nm when irradiated with excitation light in the wavelength range of approximately 470-480 nm. ALM-488 emits a target fluorescence of approximately 530 nm when irradiated with excitation light in the wavelength range of approximately 488 nm. Furthermore, LUM-015 emits a target fluorescence of approximately 675 nm when irradiated with excitation light in the wavelength range of approximately 650 nm. In addition, 5-ALA emits a target fluorescence of approximately 530-630 nm when irradiated with excitation light in the wavelength range of approximately 405 nm. Furthermore, ICG emits a target fluorescence of approximately 830 nm when irradiated with excitation light in the wavelength range of approximately 805 nm.

[0132] For example, when performing lymph node dissection during lobectomy for lung cancer, ALM-488 and ICG are used, as shown in Table 1. That is, the left recurrent laryngeal nerve is visualized using ALM-488, and the lymph nodes are visualized using ICG. By visualizing the left recurrent laryngeal nerve and lymph nodes in this way, it is possible to distinguish between the left recurrent laryngeal nerve and the lymph nodes, thereby reducing recurrent laryngeal nerve paralysis during lobectomy. In this case, since two drugs, ALM-488 and ICG, are used, one of the two drugs is excited by one of the first and second excitation lights, and the other drug is excited by the other excitation light. The physician performing the lobectomy can then determine the location of the left recurrent laryngeal nerve and lymph nodes from the image displayed on the display device 7.

[0133] Furthermore, for example, when performing lateral lymph node dissection during rectal resection for colorectal cancer, ALM-488 and ICG are used as shown in Table 1. That is, ALM-488 is used to visualize neurovascular bundles, and ICG is used to visualize lymph nodes. By visualizing these neurovascular bundles and lymph nodes, it is possible to distinguish between them and reduce urinary dysfunction, defecation dysfunction, male sexual dysfunction, or motor dysfunction caused by nerve damage during rectal resection. In this case, since two drugs, ALM-488 and ICG, are used, one of the two drugs is excited by one of the first and second excitation lights, and the other drug is excited by the other excitation light. The physician performing the rectal resection can then understand the location of the neurovascular bundles and lymph nodes from the image displayed on the display device 7.

[0134] Furthermore, for example, in malignant tumor resection surgery for pediatric brain tumors, ALM-488 and ICG are used when performing malignant tumor resection, as shown in Table 1. That is, nerves are visualized by using ALM-488, and blood vessels (blood flow) are visualized by using ICG. By visualizing nerves and blood vessels in this way, it is possible to distinguish malignant tumors from nerves and blood vessels, prevent cancer recurrence after malignant tumor resection surgery, and further prevent damage to nerves and blood vessels during malignant tumor resection surgery. In this case, since two drugs, ALM-488 and ICG, are used, one of the two drugs is excited by one of the first and second excitation lights, and the other drug is excited by the other excitation light. The physician performing the malignant tumor resection surgery can then grasp the location of nerves and blood vessels from the image displayed on the display device 7.

[0135] Furthermore, for example, in mastectomy for breast cancer, when removing malignant tumors or dissecting axillary lymph nodes, ALM-488, LUM-015, and ICG are used as shown in Table 1. Specifically, ALM-488 is used to visualize nerves such as the intercostal brachial nerve and thoracodorsal nerve, LUM-015 is used to visualize malignant tumors, and ICG is used to visualize lymph nodes. By visualizing nerves, malignant tumors, and lymph nodes in this way, it is possible to distinguish between malignant tumors, nerves, and lymph nodes, and reduce nerve damage during mastectomy. In this case, since three drugs, ALM-488, LUM-015, and ICG, are used, one of the three drugs is excited by one of the first to third excitation lights, one of the other two drugs is excited by one of the other two excitation lights, and the remaining drug is excited by the remaining excitation light. Furthermore, the surgeon performing the mastectomy can determine the locations of nerves, malignant tumors, and lymph nodes from the image displayed on the display device 7.

[0136] Furthermore, for example, when performing brain tumor treatment using photodynamic diagnosis (PDD), ALM-488 and 5-ALA (5-aminolevulinic acid) are used, as shown in Table 1. Specifically, the area near the tumor is visualized by fluorescein, the fluorescent dye of ALM-488, and the tumor cells are visualized by protoporphyrin (PpIX), which is biosynthesized from 5-ALA. By visualizing tumor cells, including the area near the tumor, it is possible to prevent damage during brain tumor treatment and safely remove the tumor. In this case, since two drugs, ALM-488 and 5-ALA, are used, one of the two drugs is excited by one of the first and second excitation lights, and the other drug is excited by the other excitation light. The physician performing the brain tumor treatment can then determine the location of the tumor cells and the area near the tumor from the image displayed on the display device 7.

[0137] Furthermore, for example, when performing transurethral cystectomy in the surgical treatment of superficial urinary tract cell carcinoma using photodynamic diagnosis, fluorescein and 5-ALA are used as shown in Table 1. That is, the tumor is visualized by PpIX, which is biosynthesized from 5-ALA. However, PpIX has the drawback of visualizing not only the tumor but also highly metabolic inflammatory tissue present in the bladder. Therefore, false positives of highly metabolic inflammatory tissue caused by PpIX are reduced by fluorescein. In this case, since two drugs, fluorescein and 5-ALA, are used, one of the two drugs is excited by one of the first and second excitation lights, and the other drug is excited by the other excitation light. Then, the physician performing the transurethral cystectomy can determine the location of the tumor and the false positives of highly metabolic inflammatory tissue caused by PpIX from the image displayed on the display device 7.

[0138] Even when adopting the configuration of the modified example 4 described above, the same effects as those of the embodiment and modified examples 1 to 3 described above are achieved.

[0139] (Modification 5) In the embodiments and modifications 1 and 3 described above, the mode switching unit 943 switched the medical observation system 1 to a first mode or a second mode in response to user operation to the input unit 95, but is not limited thereto. In this modification 5, the mode switching unit 943 detects the movement of the observation target OB in the captured image based on at least one of the first captured image and the second captured image according to the disclosure, and switches to a first mode or a second mode based on the detection result. A known method such as the block matching method can be used for detecting the movement. If the amount of movement is large as a result of the detection, the mode switching unit 943 switches to a first mode that uses two types of normal observation images generated during the first and second control, respectively. On the other hand, if the amount of movement is small as a result of the detection, the mode switching unit 943 switches to a second mode that uses only one type of normal observation image generated during the first control of the first control.

[0140] Even when adopting the configuration of the modified example 5 described above, the same effects as those of the embodiment and modified examples 1 and 3 described above are achieved.

[0141] (Modification 6) In the embodiments and modifications 1 and 3 described above, the mode switching unit 943 switched the medical observation system 1 to a first mode or a second mode in response to user operation on the input unit 95, but is not limited to this. In this modification 6, the mode switching unit 943 switches to a first mode or a second mode based on the type of imaging device (insertion unit 2 and camera head 5) according to this disclosure.

[0142] For example, consider a case where there is a highly sensitive imaging device (insertion part 2 and camera head 5) and a less sensitive imaging device. Here, "high sensitivity" means that the brightness is high and there is little image degradation. For example, when comparing a large-diameter insertion part 2 and a small-diameter insertion part 2, if the lengths are the same, the large-diameter insertion part 2 will have higher sensitivity than the small-diameter insertion part 2. Also, when comparing image sensors 531 and 532 with 4K pixels and image sensors 531 and 532 with HD pixels, if the image sensor sizes are the same, the HD image sensors 531 and 532 will have higher sensitivity than the 4K image sensors 531 and 532.

[0143] The mode switching unit 943 then switches to either the first mode or the second mode, for example, when using a highly sensitive imaging device (insertion unit 2 and camera head 5). On the other hand, the mode switching unit 943 switches to the second mode when using a less sensitive imaging device.

[0144] Furthermore, for example, the mode switching unit 943 switches to either the first mode or the second mode when using an imaging device (insertion unit 2 and camera head 5) capable of observing two invisible fluoresces. On the other hand, the mode switching unit 943 switches to the second mode when using an imaging device capable of observing one invisible fluorescence and one visible fluorescence (in the case of modified example 2).

[0145] Even when adopting the configuration of the modified example 6 described above, the same effects as those of the embodiment and modified examples 1 and 3 described above are achieved.

[0146] (Modification 7) The medical observation system according to Modification 7 is a medical observation system that uses a so-called videoscope (flexible endoscope) having an imaging unit on the tip side of the insertion part. For the sake of explanation, the medical observation system 1 according to Modification 7 will be referred to as medical observation system 1B below.

[0147] Figure 19 shows a modified example of the embodiment, part 7. As shown in Figure 19, the medical observation system 1B includes an endoscope 300B that captures an internal image of the observation site by inserting an insertion part 2B into the body and outputs the captured image, a light source device 3 that emits a first light and excitation light from the tip of the endoscope 300B, a control device 9 that processes the captured image output from the endoscope 300B, and a display device 7 that is connected to the control device 9 via a second transmission cable 8 and displays an image based on the video signal processed by the control device 9.

[0148] As shown in Figure 19, the endoscope 300B comprises an insertion section 2B having a flexible, elongated shape, an operating section 301 connected to the base end of the insertion section 2B and receiving various operations, and a universal cord 302 extending from the operating section 301 in a direction different from the direction in which the insertion section 2B extends, and containing various cables connected to the light source device 3 and the control device 9. As shown in Figure 19, the insertion section 2B comprises a tip section 24, a flexible bending section 25 connected to the base end of the tip section 24 and composed of a plurality of bending pieces, and a flexible, elongated flexible tube section 26 connected to the base end of the bending section 25.

[0149] The tip portion 24 incorporates a configuration substantially similar to that of the camera head 5 described in Embodiment 2 above, although this is not shown in detail. The captured image captured by the tip portion 24 (image sensor) is output to the control device 9 via the operation unit 301 and the universal code 302.

[0150] Even when adopting the configuration of the modified example 7 described above, the same effects as those of the embodiment described above are achieved.

[0151] (Modification 8) The medical observation system according to Modification 8 is a medical observation system that uses a surgical microscope to magnify and image a predetermined field of view of the inside (in vivo) or surface (biological surface) of the subject being observed. For the sake of explanation, the medical observation system 1 according to Modification 3 will be referred to as medical observation system 1C below.

[0152] Figure 20 shows a modified example of the embodiment, part 8. As shown in Figure 20, the medical observation system 1C includes a surgical microscope 12 that captures images for observing a subject and outputs the captured images, a control device 9 that processes the captured images output from the surgical microscope 12, and a display device 7 connected to the control device 9 via a second transmission cable 8 and which displays images based on the video signals processed by the control device 9.

[0153] As shown in Figure 20, the surgical microscope 12 comprises a microscope unit 121 that magnifies and images minute parts of a subject and outputs the captured image, a support unit 122 connected to the base end of the microscope unit 121 and including an arm that rotatably supports the microscope unit 121, and a base unit 123 that rotatably holds the base end of the support unit 122 and is movable on the floor. The control device 9 is installed on the base unit 123 as shown in Figure 20. Although not specifically shown in the illustration, a light source device 3 that emits first light and excitation light from the surgical microscope 12 to the object being observed is also installed on the base unit 123. Note that the base unit 123 may be fixed to the ceiling or wall to support the support unit 122, rather than being movably installed on the floor.

[0154] Although not shown in detail in the illustration, the microscope unit 121 incorporates a configuration substantially similar to that of the camera head 5 described in Embodiment 2 above. The image captured by the microscope unit 121 (image sensor) is output to the control device 9 via the first transmission cable 6, which is wired along the support unit 122.

[0155] Even when adopting the configuration of the modified example 8 described above, the same effects as those of the embodiment described above are achieved.

[0156] (Modification 9) Figures 21 and 22 illustrate modification 9 of the embodiment. Specifically, Figure 21 is a side view of the ring light 15. Figure 22 is a front view of the ring light 15 (left side in Figure 21). In this modification 9, in addition to the insertion part 2 described in the above embodiment, the ring light 15 shown in Figures 21 and 22 is detachably connected to the camera head 5. That is, depending on the user's usage, the camera head 5 may be connected to either the insertion part 2 or the ring light 15, as shown in Figure 21.

[0157] Unlike the insertion unit 2, the ring light 15 is not inserted into the observation target OB, but rather supplies first light and excitation light to the surgical unit and captures second light (subject image), which is the reflected light of the first light and excitation light from the surgical unit. As shown in Figures 21 and 22, the ring light 15 comprises an illumination unit 151 and a subject image capture unit 152 for capturing the subject image.

[0158] As shown in Figures 21 and 22, the illumination unit 151 comprises a housing 1511 and a plurality of illumination lenses 1512. The housing 1511 has an annular shape centered on the optical axis Ax. The other end of the light guide 4 is detachably connected to the housing 1511.

[0159] As shown in Figure 22, the multiple illumination lenses 1512 are arranged at predetermined intervals along the circumferential direction centered on the optical axis Ax on the front end face of the housing 1511. The multiple illumination lenses 1512 each irradiate the surgical area with the first light supplied from the light source device 3 and introduced into the housing 1511 via the light guide 4.

[0160] The subject image acquisition section 152 extends along the optical axis Ax. Within the subject image acquisition section 152, there is an optical system composed of one or more lenses that focuses the second light (subject image) illuminated from multiple illumination lenses 1512 and transmitted through the surgical section. Furthermore, a connection section 1521 is provided at the base end (right side in Figure 21) of the subject image acquisition section 152. This connection section 1521 is designed (shaped) to be compatible with the eyepiece section 21 in the insertion section 2 and is detachably connected to the camera head 5.

[0161] Even when adopting the configuration of the modified example 9 described above, the same effects as those of the embodiment described above are achieved.

[0162] The following configurations also fall within the technical scope of this disclosure: (1) A medical control device comprising: a light source control unit that controls the operation of a light source device and performs, in a specific order, a first control that sets the emission energy of a first light emitted from the light source device to a first emission energy, and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy; an imaging control unit that controls the operation of an imaging device and generates a first image by imaging of the imaging device when the first control is executed, and generates a second image by imaging of the imaging device when the second control is executed; and an image processing unit that generates a display image based on the first image and the second image. (2) The medical control device according to (1), wherein the first light includes at least a portion of the visible light wavelength band. (3) The medical control device according to (1) or (2), wherein the light source control unit, in the first control, emits a first excitation light from the light source device to excite a substance contained in the object to be observed, and in the second control, emits a second excitation light from the light source device to excite a substance contained in the object to be observed. (4) The medical control device according to (3), wherein the first excitation light is excitation light corresponding to a first drug, the second excitation light is excitation light corresponding to a second drug, and the fluorescence intensity emitted from the first drug upon irradiation with the first excitation light is stronger than the fluorescence intensity emitted from the second drug upon irradiation with the second excitation light.(5) The medical control device according to (3) or (4), wherein the imaging device comprises a first image sensor for imaging the reflected light of the first light from the object to be observed, and a second image sensor for imaging the reflected light of the first excitation light and the reflected light of the second excitation light from the object to be observed, and the first image includes a first normal image generated when the first image sensor images the reflected light of the first light during the execution of the first control, and a first fluorescence image generated when the second image sensor images the reflected light of the first excitation light during the execution of the first control, and the second image includes a second normal image generated when the first image sensor images the reflected light of the first light during the execution of the second control, and a second fluorescence image generated when the second image sensor images the reflected light of the second excitation light during the execution of the second control. (6) The medical control device according to (5), further comprising a mode switching unit for switching to a first mode or a second mode, wherein the image processing unit generates the display image based on the first normal image and the second normal image and the first fluorescence image and the second fluorescence image when it is in the first mode, and generates the display image based on either one of the first normal image and the second normal image and the first fluorescence image and the second fluorescence image when it is in the second mode. (7) The medical control device according to (6), wherein the mode switching unit detects the movement of the object being observed in the captured image based on at least one of the first captured image and the second captured image, and switches to the first mode or the second mode based on the detection result. (8) The medical control device according to (6), wherein the mode switching unit switches to the first mode or the second mode based on the type of the imaging device. (9) The medical control device according to any one of (6) to (8) above, wherein the light source control unit sets the second light emission energy to 0 when the second mode is in operation.(10) The medical control device according to any one of (5) to (9), wherein the first luminescence energy is a luminescence energy such that the signal value based on the return light of the first excitation light is greater than the signal value based on unwanted light generated due to at least the first light imaged by the second image sensor, and the second luminescence energy is a luminescence energy such that the signal value based on the return light of the second excitation light is greater than the signal value based on the unwanted light imaged by the second image sensor. (11) The medical control device according to (10), wherein the unwanted light includes autofluorescence generated from an autofluorescence generating member that forms the optical path of the first light. (12) The medical control device according to (3), wherein the imaging device comprises a first image sensor that images the return light of the first light and the return light of the second excitation light from the object to be observed, and a second image sensor that images the return light of the first excitation light from the object to be observed, wherein the first image includes a first normal image generated by the first image sensor imaging the return light of the first light when the first control is performed and a first fluorescence image generated by the second image sensor imaging the return light of the first excitation light when the first control is performed, and the second image includes a second fluorescence image generated by the first image sensor imaging the return light of the second excitation light when the second control is performed. (13) The medical control device according to (12), wherein the first emission energy is an emission energy in which the signal value based on the return light of the first excitation light is greater than the signal value based on unwanted light generated by at least the first light imaged by the second image sensor, and the second emission energy is 0.(14) A medical observation system comprising: a light source device that emits a first light; an imaging device that images the reflected light from an object to be observed; and a medical control device that controls the operation of the light source device and the imaging device, respectively, wherein the medical control device comprises: a light source control unit that controls the operation of the light source device and performs in a specific order a first control that sets the emission energy of the first light to a first emission energy and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy; an imaging control unit that controls the operation of the imaging device and generates a first image by imaging of the imaging device when the first control is executed and generates a second image by imaging of the imaging device when the second control is executed; and an image processing unit that generates a display image based on the first image and the second image. (15) The medical observation system according to (14), wherein the first light is broadband light or narrowband light. (16) The medical observation system according to (14) or (15), wherein the light source device includes a first light source that emits the first light, and the first light source is comprised of one or more. (17) The medical observation system according to any one of (14) to (16), wherein the light source device includes a first light source that emits the first light, and the first light source is comprised of an LED or a semiconductor laser. (18) The medical observation system according to any one of (14) to (17), wherein the light source device emits a first excitation light for exciting a substance contained in the object to be observed and a second excitation light for exciting a substance contained in the object to be observed, and the light source control unit, in the first control, causes the light source device to emit the first excitation light for exciting a substance contained in the object to be observed, and in the second control, causes the light source device to emit the second excitation light for exciting a substance contained in the object to be observed. (19) The medical observation system according to (18), wherein the first excitation light and the second excitation light are each narrowband light. (20) The medical observation system according to (18) or (19), wherein the light source device includes a second light source that emits the first excitation light and a third light source that emits the second excitation light, and the second light source and the third light source each consist of one or more.(21) The medical observation system according to any one of (18) to (20), wherein the light source device includes a second light source that emits the first excitation light and a third light source that emits the second excitation light, and the second light source and the third light source are each composed of an LED or a semiconductor laser. (22) The medical observation system according to any one of (14) to (21), wherein the imaging device includes a first image sensor that images light including at least a portion of the visible light wavelength band and a second image sensor that images light including at least a portion of the non-visible light wavelength band. (23) A control method performed by a medical control device, wherein the medical control device controls the operation of a light source device, and performs in a specific order a first control to set the luminescence energy of a first light emitted from the light source device to a first luminescence energy, and a second control to set the luminescence energy of the first light to a second luminescence energy lower than the first luminescence energy, and controls the operation of an imaging device, causing the imaging device to generate a first image when it is performing the first control, and causing the imaging device to generate a second image when it is performing the second control, and a display image is generated based on the first image and the second image.

[0163] 1, 1B, 1C Medical observation system 2, 2B Insertion section 3 Light source device 4 Light guide 5 Camera head 6 First transmission cable 7 Display device 8 Second transmission cable 9 Control device 10 Third transmission cable 12 Surgical microscope 15 Ring light 21 Eyepiece section 22 Excitation light cut filter 24 Tip section 25 Bending section 26 Flexible tube section 31 First light source 32 Second light source 33 Third light source 34 Fourth light source 51 Lens unit 52 Prism 53 Imaging section 54 Communication section 91 Communication section 92 Image memory 93 Processing module 94 Control section 95 Input section 96 Output section 97 Storage section 121 Microscope section 122 Support section 123 Base section 151 Illumination section 152 Subject image capture section 300B Endoscope 301 Control Unit 302 Universal Code 531 First Image Sensor 532 Second Image Sensor 533 Signal Processing Unit 931 Memory Controller 932 Processing Unit 933 Display Control Unit 941 Light Source Control Unit 942 Imaging Control Unit 943 Mode Switching Unit 1511 Housing 1512 Illumination Lens 1521 Connection Unit ArF1 to ArF3 Region Ax Optical Axis CN1, CN2 Connectors DF1, DF2, DF11 to DF13, DF21 to DF23 Display Image F1 First Subject F2 Second Subject FL1, FL11, FL12 First Fluorescence Observation Image FL2, FL21 to FL24 Second Fluorescence Observation Image OB Observation Target P0 Observation Optical Path P1 First Optical Path P2 Second optical path TE Full line exposure period WLI, WLI1-WLI4 Normal observation image

Claims

A light source control unit that controls the operation of a light source device and performs, in a specific order, a first control that sets the emission energy of a first light emitted from the light source device to a first emission energy, and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy. An imaging control unit controls the operation of the imaging device, generates a first image by imaging with the imaging device when the first control is executed, and generates a second image by imaging with the imaging device when the second control is executed, A medical control device comprising an image processing unit that generates a display image based on the first captured image and the second captured image. 　 The first light is, A medical control device according to claim 1, comprising at least a portion of the visible light wavelength band. 　 The light source control unit, In the first control described above, the light source device emits a first excitation light that excites the substance contained in the object to be observed. The medical control device according to claim 1, wherein in the second control, a second excitation light is emitted from the light source device to excite a substance contained in the object to be observed. 　 The first excitation light is, This is the excitation light corresponding to the first drug, The second excitation light described above is This is excitation light corresponding to the second drug, The intensity of fluorescence emitted from the first drug upon irradiation with the first excitation light is, The medical control device according to claim 3, wherein the fluorescence intensity emitted from the second drug by irradiation with the second excitation light is stronger than the fluorescence intensity emitted from the second drug by irradiation with the second excitation light. 　 The imaging device is A first image sensor that captures the reflected light of the first light from the object being observed, The system includes a second image sensor that captures the reflected light of the first excitation light and the reflected light of the second excitation light from the object being observed, respectively. The first captured image is A first normal image generated by the first image sensor capturing the reflected light of the first light during the execution of the first control, The first fluorescence image is generated when the second image sensor captures the return light of the first excitation light during the execution of the first control, The second image described above is A second normal image is generated when the first image sensor captures the reflected light of the first light during the execution of the second control, The medical control device according to claim 3, further comprising: a second fluorescence image generated by the second image sensor capturing the return light of the second excitation light during the execution of the second control; 　 The system further includes a mode switching unit for switching between a first mode and a second mode. The aforementioned image processing unit, In the first mode, the display image is generated based on the first normal image and the second normal image, and the first fluorescence image and the second fluorescence image. In the case of the second mode, the medical control device according to claim 5 generates the display image based on either the first normal image or the second normal image, and the first fluorescence image or the second fluorescence image. 　 The mode switching unit is The medical control device according to claim 6, which detects the movement of the object being observed within the captured image based on at least one of the first captured image and the second captured image, and switches to the first mode or the second mode based on the detection result. 　 The mode switching unit is The medical control device according to claim 6, which switches to the first mode or the second mode based on the type of imaging device. 　 The light source control unit, The medical control device according to claim 6, wherein the second light emission energy is set to 0 when the second mode is in operation. 　 The first luminescence energy is, The emission energy is such that the signal value based on the return light of the first excitation light is greater than the signal value based on unwanted light generated at least from the first light captured by the second image sensor, The second luminescence energy described above is The medical control device according to claim 5, wherein the luminescence energy is such that the signal value based on the return light of the second excitation light is greater than the signal value based on the unwanted light captured by the second image sensor. 　 The aforementioned unwanted light is The medical control device according to claim 10, comprising autofluorescence generated from an autofluorescence generating member that forms the optical path of the first light. 　 The imaging device is A first image sensor that captures the reflected light of the first light and the reflected light of the second excitation light from the object being observed, The system comprises a second image sensor that captures the reflected light of the first excitation light from the object being observed, The first captured image is A first normal image generated by the first image sensor capturing the reflected light of the first light during the execution of the first control, The first fluorescence image is generated when the second image sensor captures the return light of the first excitation light during the execution of the first control, The second image described above is The medical control device according to claim 3, comprising a second fluorescence image generated by the first image sensor capturing the return light of the second excitation light during the execution of the second control. 　 The first luminescence energy is, The emission energy is such that the signal value based on the return light of the first excitation light is greater than the signal value based on unwanted light generated at least from the first light captured by the second image sensor, The second luminescence energy described above is A medical control device according to claim 12, wherein the value is 0. 　 A light source device that emits a first light, An imaging device that captures the reflected light from the object being observed, The system includes a medical control device that controls the operation of the light source device and the imaging device, respectively. The aforementioned medical control device is A light source control unit that controls the operation of the light source device and performs, in a specific order, a first control that sets the emission energy of the first light to a first emission energy, and a second control that sets the emission energy of the first light to a second emission energy lower than the first emission energy, An imaging control unit controls the operation of the imaging device, generates a first image by imaging with the imaging device when the first control is executed, and generates a second image by imaging with the imaging device when the second control is executed, A medical observation system comprising an image processing unit that generates a display image based on the first captured image and the second captured image. 　 The first light is, The medical observation system according to claim 14, wherein the light is broadband or narrowband. 　 The aforementioned light source device is Includes a first light source that emits the first light, The first light source is, A medical observation system according to claim 14, comprising one or more components. 　 The aforementioned light source device is Includes a first light source that emits the first light, The first light source is, A medical observation system according to claim 14, comprising an LED or a semiconductor laser. 　 The aforementioned light source device is A first excitation light that excites the substance contained in the object being observed and a second excitation light that excites the substance contained in the object being observed are emitted, The light source control unit, In the first control described above, the light source device emits a first excitation light that excites the substance contained in the object to be observed. The medical observation system according to claim 14, wherein in the second control, a second excitation light is emitted from the light source device to excite a substance contained in the object to be observed. 　 The first excitation light and the second excitation light are The medical observation system according to claim 18, wherein each is narrowband light. 　 The imaging device is A first image sensor that images light including at least a portion of the visible light wavelength band, A medical observation system according to claim 14, further comprising a second image sensor that images light including at least a portion of the non-visible light wavelength band. 　 A control method performed by a medical control device, The aforementioned medical control device is The operation of the light source device is controlled, and a first control is performed in a specific order to set the emission energy of the first light emitted from the light source device to a first emission energy, and a second control is performed to set the emission energy of the first light to a second emission energy lower than the first emission energy. The operation of the imaging device is controlled, and when the first control is executed, a first image is generated by imaging with the imaging device, and when the second control is executed, a second image is generated by imaging with the imaging device. A control method for generating a display image based on the first captured image and the second captured image.

Images