Medical observation system, control device and control method

Abstract

Provided are a medical observation system, a control device, and a control method capable of preventing flickering of observation locations in an observed body and of achieving a compact device size. This medical observation system 1 is provided with: an image capturing element 212; a second control unit 94 that causes a light source device 3 to simultaneously emit second visible light and excitation light; and an image processing unit 91 that generates, to be included in image data, a fluorescent image based on first pixel values output by pixels on which first filters are arranged and a background image based on second pixel values output by pixels on which second filters are arranged.

Description

[Technical field]

[0001] The present disclosure relates to a medical observation system, a control device, and a control method for generating image data by imaging an object to be observed. [Background technology]

[0002] Conventionally, in surgical microscopes, a technology has been known in which visible light observation and infrared light observation are performed by providing a cut filter on the incident side of one of the two imaging elements, which blocks excitation light for exciting a fluorescent substance and transmits fluorescence from the fluorescent substance (see, for example, Patent Document 1).

[0003] In addition, a medical observation system is provided with a Bayer array imaging element and an imaging element sensitive to infrared light, and by irradiating white light and infrared light, a normal image using white light and an infrared fluorescence image using infrared light are obtained (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-49168 A [Patent Document 2] JP 2016-087062 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned Patent Document 1, two imaging elements are used for visible light observation and infrared light observation, which causes a problem that it is difficult to reduce the size of the device.

[0006] Furthermore, in the above-mentioned Patent Document 2, white light and infrared light are alternately irradiated, which causes a problem that the observation location of the observation object flickers.

[0007] The present disclosure has been made in view of the above, and has an object to provide a medical observation system, a control device, and a control method that can prevent flickering of an observation location on an observation object and enable the device to be miniaturized. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems and achieve the object, the medical observation system according to the present disclosure includes a light source device capable of irradiating a subject with first visible light and second visible light in different wavelength bands and excitation light that excites a fluorescent substance to emit fluorescence; an image sensor having a pixel unit having a plurality of pixels, as well as a first filter that is capable of transmitting the first visible light and the fluorescence and a second filter that is capable of transmitting the second visible light and the fluorescence, on a light-receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and reflected light of at least one of the first visible light and the second visible light reflected from the subject; a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; and an image processing unit that generates a fluorescence image based on a first pixel value output by a pixel in which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel in which the second filter is arranged.

[0009] Furthermore, the medical observation system according to the present disclosure includes a light source device capable of irradiating a subject with first visible light and second visible light of different wavelength bands and excitation light that excites a fluorescent substance to emit fluorescence; a dichroic prism that separates the fluorescence and light reflected from the subject of at least one of the first visible light and the second visible light into a plurality of wavelength bands; a plurality of image sensors that receive each of the light of the plurality of wavelength bands separated by the dichroic prism and generate a plurality of image data; a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; and an image processing unit that generates a background image and a fluorescence image based on the plurality of image data.

[0010] Furthermore, a control device according to the present disclosure is a control device that controls a light source device capable of irradiating a subject with first visible light and second visible light of different wavelength bands and excitation light that excites a fluorescent substance to emit fluorescence, and a medical imaging device that has a pixel unit having a plurality of pixels, and a first filter that is transparent to the first visible light and the fluorescence and a second filter that is transparent to the second visible light and the fluorescence on the light receiving surface of each of the plurality of pixels, and generates image data by capturing at least one of the fluorescence and reflected light of at least one of the first visible light and the second visible light reflected from the subject, and the control device includes a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light, and an image processing unit that generates a fluorescence image based on first pixel values ​​output by pixels in which the first filter is arranged, which are included in the image data, and a background image based on second pixel values ​​output by pixels in which the second filter is arranged.

[0011] Furthermore, a control method according to the present disclosure is a control method executed by a control device that controls a light source device capable of irradiating a subject with first visible light and second visible light in different wavelength bands and excitation light that excites a fluorescent substance to emit fluorescence, and a medical imaging device having a pixel unit having a plurality of pixels, and a first filter capable of transmitting the first visible light and the fluorescence and a second filter capable of transmitting the second visible light and the fluorescence on the light receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and reflected light of at least one of the first visible light and the second visible light reflected from the subject, wherein the control method causes the light source device to simultaneously irradiate the second visible light and the excitation light, and generates a fluorescence image based on first pixel values ​​output by pixels in which the first filter is disposed, which are included in the image data, and a background image based on second pixel values ​​output by pixels in which the second filter is disposed. Effect of the Invention

[0012] According to the present disclosure, it is possible to prevent flickering at an observation location and to reduce the size of the device. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a medical observation system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the medical observation system according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating a schematic configuration of a pixel unit according to the first embodiment. In FIG. [Figure 4] FIG. 4 is a diagram illustrating a schematic configuration of a color filter according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating the sensitivity of the image sensor after light passes through a color filter in each wavelength band according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an outline of the processing executed by the medical observation system according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing a schematic overview of the processing in the fluorescence observation mode executed by the medical observation system according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating a schematic diagram of a generation method used by the first generation unit according to the first embodiment to generate a fluorescent image. [Figure 9] FIG. 9 is a schematic diagram showing another example of a generation method used by the first generation unit according to the first embodiment to generate a fluorescence image. [Figure 10] FIG. 10 is a diagram illustrating a generation method when the second generation unit according to the first embodiment generates a background image. [Figure 11] FIG. 11 is a diagram illustrating a schematic diagram of another generation method used by the second generation unit according to the first embodiment to generate a background image. [Figure 12] FIG. 12 is a conceptual diagram showing a configuration of an imaging unit according to a modification of the first embodiment. In FIG. [Figure 13] FIG. 13 is a diagram showing a schematic configuration of a medical observation system according to the second embodiment. [Figure 14]FIG. 14 is a block diagram showing the functional configuration of the main parts of the medical observation system according to the second embodiment. As shown in FIG. [Figure 15] FIG. 15 is a diagram showing a schematic configuration of a medical observation system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, a mode for carrying out the present disclosure (hereinafter, referred to as "embodiment") will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the following embodiment. Also, each figure referred to in the following description merely shows a schematic representation of the shape, size, and positional relationship to the extent that the contents of the present disclosure can be understood. In other words, the present disclosure is not limited to only the shape, size, and positional relationship exemplified in each figure.

[0015] (Embodiment 1) [Overview of medical observation system] Fig. 1 is a diagram showing the overall configuration of a medical observation system according to embodiment 1. The medical observation system 1 shown in Fig. 1 comprises a medical observation device 2 having a function as a microscope for magnifying and observing a minute part of an observation subject, a light source device 3 for supplying illumination light to the observation device 2 via a light guide 4 made of optical fiber or the like, a display device 8 for displaying an image based on image data captured by the observation device 2, and a control device 9 for overall control of the operation of the medical observation system 1.

[0016] [Outline of the observation device] First, we will explain the schematic configuration of the observation device 2. The observation device 2 includes a microscope unit 5 for observing a minute portion of an observation object, a support unit 6 connected to the base end of the microscope unit 5 and rotatably supporting the microscope unit 5, and a base unit 7 rotatably holding the base end of the support unit 6 and movable on a floor surface.

[0017] The microscope section 5 has a cylindrical appearance, and includes an optical system having a zoom function and a focus function, an image sensor (not shown) that receives an object image formed by the optical system and performs photoelectric conversion to generate image data, and a light emitting section (not shown) that irradiates the object to be observed with illumination light. In addition, various switches constituting an input section 24 that receives input of operation instructions for the observation device 2 are provided on the side of the microscope section 5. A cover glass (not shown) that protects the internal optical system and the like is provided on the opening surface at the lower end of the microscope section 5. A user such as an operator moves the microscope section 5, changes the angle of the microscope section 5, changes the mode of the observation device 2, and performs zoom or focus operations while holding the microscope section 5 and operating the various switches. The shape of the microscope section 5 is not limited to a cylindrical shape, and may be, for example, a polygonal tube shape.

[0018] Under the control of the control device 9, the light source device 3 supplies at least one illumination light of white light and infrared light including light in the red wavelength band, light in the green wavelength band, and light in the blue wavelength band to the observation device 2 via the light guide 4. The light source device 3 is configured using a discharge lamp such as a xenon lamp or a metal halide lamp, a solid-state light-emitting element such as an LED (Light Emitting Diode) or an LD (Laser Diode), or a light-emitting member such as a halogen lamp.

[0019] The display device 8 displays the display image generated by the control device 9 and various information related to the medical observation system. The display device 8 is configured using liquid crystal, organic EL (Electro Luminescence), etc. The display device 8 displays a 2D image or a 3D image.

[0020] The control device 9 comprehensively controls each part of the medical observation system 1. The control device 9 is realized by using a processor having hardware such as a general-purpose processor such as a central processing unit (CPU) or an application specific integrated circuit (ASIC) or a graphics processing unit (GPU) that executes specific functions. The control device 9 may also be configured by using a field programmable gate array (FPGA: not shown), which is a type of programmable integrated circuit. When configured by an FPGA, a memory for storing configuration data may be provided, and the FPGA, which is a programmable integrated circuit, may be configured by the configuration data read from the memory. The detailed configuration of the control device 9 will be described later.

[0021] [Functional configuration of medical observation system] Next, a description will be given of the functional configuration of the medical observation system 1. Fig. 2 is a block diagram showing the functional configuration of the medical observation system 1.

[0022] [Functional configuration of the observation device] First, the functional configuration of the observation device 2 will be described. The observation device 2 includes a microscope unit 5 , a detection unit 23 , an input unit 24 , and a first control unit 25 .

[0023] The microscope unit 5 includes an imaging unit 21 that generates image data by enlarging an image of an object to be observed, and a light emission unit 22 that irradiates illumination light supplied from the light source device 3 toward the object to be observed.

[0024] The imaging section 21 includes an optical system 211, an imaging element 212, and a cut filter 213. The imaging section 21 functions as the medical imaging device according to the first embodiment.

[0025] The optical system 211 has zoom and focus functions, and forms an image of a subject on the light receiving surface of the image sensor 212 via a cut filter 213. The optical system 211 is realized using one or more lenses, a motor for moving the lenses along the optical path L1, or the like.

[0026] The image sensor 212 receives the subject image formed by the optical system 211 via a cut filter 213 and performs photoelectric conversion to generate image data (RAW data). The image sensor 212 is realized using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image sensor 212 has a pixel unit 212a and a color filter 212b.

[0027] 3 is a diagram showing a schematic configuration of the pixel section 212a. As shown in FIG. 3, the pixel section 212a includes a plurality of pixels P such as photodiodes that accumulate electric charges according to the amount of light. n,m (n=an integer equal to or greater than 1, m=an integer equal to or greater than 1) are arranged in a two-dimensional matrix. The pixel section 212a is configured to generate a plurality of pixels P n,m Pixel P in the readout area that is arbitrarily set as the readout target n,m The image signal is read out as image data from the pixel unit 212a and output to the control device 9. Specifically, the image data generated by the pixel unit 212a is transmitted to the control device 9 via a transmission cable. Note that the image data generated by the pixel unit 212a may be transmitted to the control device 9 as an optical signal by performing E / O conversion on the image data.

[0028] Fig. 4 is a diagram showing a schematic configuration of the color filter 212b. The color filter 212b shown in Fig. 4 is configured in a Bayer array with 2 x 2 as one unit. The color filter 212b is configured using a filter R that transmits light in the red wavelength band, two filters G (filter Gr, filter Gb) that transmit light in the green wavelength band, and a filter B that transmits light in the blue wavelength band.

[0029] In the following, a pixel P having a filter R disposed on its light receiving surface is referred to as a pixel P. n,m The pixel P is an R pixel, and the filters Gr and Gb are arranged on the light receiving surface. n,m+1 Gr pixel, pixel P n+1,m The pixel P is a Gb pixel (hereinafter, Gr and Gb are collectively referred to as G pixel), and filter B is arranged on the light receiving surface. n+1,m+1 is described as a B pixel. Furthermore, in the first embodiment, the filter R functions as a first filter that transmits light in the red wavelength band (first visible light) and fluorescence, the filter G functions as a second filter that transmits light in the green wavelength band (second visible light) and fluorescence, and the filter B functions as a third filter that transmits light in the blue wavelength band (third visible light) and fluorescence. That is, the pixel value of the R pixel includes red reflected light and fluorescent components, the pixel value of the G pixel (Gr pixel, Gb pixel) includes green reflected light and fluorescent light, and the pixel value of the B pixel includes blue reflected light and fluorescent light.

[0030] The cut filter 213 is disposed on an optical path L1 between the optical system 211 and the image sensor 212. The cut filter 213 blocks light of a wavelength component (e.g., 740±10 nm) of the excitation light contained in the subject image formed by the optical system 211, and transmits light of wavelength components other than the excitation light.

[0031] Here, the spectral characteristics of each pixel will be described. FIG. 5 is a diagram showing the sensitivity of the image sensor after transmission through a color filter in each wavelength band. In FIG. 5, the horizontal axis indicates the wavelength (nm) and the vertical axis indicates the spectral characteristics. Also, in FIG. 5, the curve L B represents the spectral characteristics of the B pixel, and curve L G represents the spectral characteristics of the G pixel, and curve L R indicates the spectral characteristics of the R pixel, and the line L IR The curve L indicates the wavelength band of the fluorescence emitted by the fluorescent material when it is irradiated with excitation light. CUT indicates the transmission characteristic of the cut filter 213.

[0032] Curve L in Figure 5 B and the line LIR As shown in FIG. 5, the B pixel is sensitive to light in the blue wavelength band (435 nm to 480 nm) (hereinafter simply referred to as "blue light"), and is also sensitive to fluorescence in the wavelength band (830±10 nm) emitted by irradiating a fluorescent material with excitation light. G and the line L IR As shown in FIG. 5, the G pixels (Gr and Gb pixels) are sensitive to light in the green wavelength band (500 nm to 560 nm) (hereinafter simply referred to as "green light"), and are also sensitive to fluorescence in the wavelength band (830±10 nm) emitted by irradiating a fluorescent material with excitation light. R and the line L IR As shown in the figure, the R pixel is sensitive to light in the red wavelength band (610 nm to 750 nm) (hereinafter simply referred to as "red light"), and is also sensitive to fluorescence in the wavelength band (830±10 nm) emitted when a fluorescent substance is irradiated with excitation light.

[0033] The light output unit 22 has an illumination optical system configured using one or more lenses. The light output unit 22 irradiates illumination light of at least one of white light, light of a red wavelength band, light of a green wavelength band, light of a blue wavelength band, and infrared light supplied from the light source device 3 via a light guide 4 in the same direction as the imaging direction of the imaging unit 21. The light output unit 22 may omit optical transmission such as a light guide by providing an LED (Light Emitting Diode) or a laser light source in the microscope unit 5.

[0034] The detection unit 23 sequentially detects status information of the observation device 2. The status information of the observation device 2 includes information related to the position, focus, and zoom of the imaging unit 21. The detection unit 23 has various sensors for detecting this information.

[0035] The input unit 24 accepts input of operation instructions for the imaging unit 21. The input unit 24 has a focus switch and a zoom switch that accept inputs instructing focus and zoom operations in the imaging unit 21, an electric field of view movement mode switch that accepts an input instructing an electric field of view movement mode, and a mode changeover switch that accepts an input instructing switching of the observation mode of the medical observation system 1. The various switches, buttons, etc. that constitute the input unit 24 are provided on the side of the microscope unit 5 as shown in FIG.

[0036] The first control unit 25 controls the operation of the imaging unit 21 in response to an operation instruction received by the input unit 24 or an operation instruction input from a control device 9 described later. The first control unit 25 also comprehensively controls the observation device 2 in cooperation with a second control unit 94 of the control device 9 described later. The first control unit 25 is configured using a memory and a processor such as a CPU, FPGA, or ASIC.

[0037] [Configuration of the light source device] Next, the configuration of the light source device 3 will be described. The light source device 3 includes a first light source section 31, a second light source section 32, a third light source section 33, and a fourth light source section .

[0038] The first light source unit 31 supplies red light to the light emitting unit 22 of the observation device 2 via the light guide 4 under the control of the control device 9. The first light source unit 31 is realized using a red LED or the like.

[0039] The second light source unit 32 supplies green light to the light emitting unit 22 of the observation device 2 via the light guide 4 under the control of the control device 9. The second light source unit 32 is realized using a green LED or the like.

[0040] Under the control of the control device 9, the third light source unit 33 supplies blue light to the light emitting unit 22 of the observation device 2 via the light guide 4. The third light source unit 33 is realized using a blue LED or the like.

[0041] The fourth light source unit 34 supplies infrared light for exciting fluorescent substances to the light emitting unit 22 of the observation device 2 via the light guide 4. The fourth light source unit 34 supplies infrared light (wavelength band 740±10 nm) that functions as excitation light for exciting fluorescent substances under the control of the control device 9. The second light source unit 32 is configured using a semiconductor laser element capable of emitting infrared light (700 to 1000 nm) used in ICG (Indocyanine Green) observation, a filter that transmits only a predetermined wavelength band (wavelength band 740±10 nm), and the like. In the following, the light will be described as infrared light, but is not limited to this. For example, the light may be light (wavelength band 415±10 nm) used in PDD (Photo Dynamic Diagnosis) observation, in which a photosensitive substance such as a hematoporphyrin derivative is pre-accumulated in tumor tissue and the fluorescence is observed, and light (wavelength band 390-470 nm+wavelength band 540-560 nm) used in AFI (Auto Fluorescence Imaging) observation, in which autoluminescence from fluorescent substances such as collagen is observed.

[0042] [Configuration of the control device] Next, the functional configuration of the control device 9 will be described. The control device 9 includes an image processing unit 91, an input unit 92, a recording unit 93, and a second control unit 94.

[0043] The image processing unit 91 performs various image processing on the image data transmitted from the observation device 2 to generate a display image (video data) for display on the display device 8. Here, the image processing includes various image processing such as color correction, color enhancement, and edge enhancement. The image processing unit 91 is realized using a memory and a processor such as a GPU (Graphics Processing Unit), ASIC, or FPGA. The image processing unit 91 has at least a subtraction unit 911, a first generation unit 912, a second generation unit 913, and a synthesis unit 914.

[0044] The subtraction unit 911 subtracts the first pixel value output by the pixel (R pixel) in which the first filter (filter R) is arranged from the second pixel value output by the pixel (B pixel or G pixel) in which the second and third filters (filter G or filter B) included in the image data input from the imaging unit 21 are arranged, and outputs the result of this subtraction to the second generation unit 913. Specifically, the subtraction unit 911 may subtract from the first pixel value a multiplication result obtained by multiplying the first pixel value by a value obtained by dividing the spectral sensitivity of the fluorescence wavelength of the second filter by the spectral sensitivity of the fluorescence wavelength of the first filter, and output the result of this subtraction to the second generation unit 913. That is, the subtraction unit 911 may change the second pixel value based on the first pixel value, and output the changed result (changed second pixel value) to the second generation unit 913. The calculation method by the subtraction unit 911 will be described later.

[0045] The first generating unit 912, under the control of the second control unit 94, generates a fluorescent image based on a first pixel value output by a pixel (R pixel) in which a first filter (filter R) included in the image data input from the imaging unit 21 is arranged, and outputs the fluorescent image to the synthesis unit 914. Specifically, the first generating unit 912 generates a fluorescent image by complementing pixel values ​​of B pixels and G pixels based on the pixel value of the R pixel included in the image data, and outputs the generated fluorescent image to the synthesis unit 914. Furthermore, the first generating unit 912 performs a coloring process on the fluorescent image. Specifically, the first generating unit 912 performs coloring of the fluorescent image by a color tone conversion process or the like based on the luminance value of the fluorescent image, and outputs the colored fluorescent image to the synthesis unit 914. For example, the first generating unit 912 performs a coloring process so that the fluorescent region is green based on the luminance value of the fluorescent image. The first generating section 912 may set the color to be used in the fluorescent image based on a designation signal that designates the color of the fluorescent region of the fluorescent image input from the input section 92 via the second control section 94.

[0046] The second generating unit 913, under the control of the second control unit 94, generates a background image based on second and third pixel values ​​output by pixels (G pixels, B pixels) in which the second and third filters (G filter, B filter) included in the image data input from the imaging unit 21 are arranged, and outputs the background image to the synthesis unit 914. Specifically, the second generating unit 913 generates a background image based on the input result input from the subtraction unit 911. Furthermore, the second generating unit 913 may perform a black and white conversion process on the background image and output it to the synthesis unit 914. For example, the second generating unit 913 performs a saturation reduction process to reduce the saturation of the background image, and outputs the black and white background image to the synthesis unit 914.

[0047] The synthesis unit 914 generates a synthetic image by synthesizing the fluorescent image input from the first generation unit 912 and the background image input from the second generation unit 913, and outputs this synthetic image to the display device 8. Specifically, the synthesis unit 914 may generate the synthetic image by synthesizing the fluorescent image and the background image at a predetermined ratio (for example, 1:1).

[0048] The input unit 92 is realized using a user interface such as a keyboard, a mouse, a touch panel, a foot switch, etc., and accepts input of various information.

[0049] The recording unit 93 is configured using a semiconductor memory such as a flash memory or a dynamic random access memory (DRAM), and has a program recording unit 931 that temporarily records various programs executed by the medical observation system 1 and data being processed.

[0050] The second control unit 94 comprehensively controls each unit of the medical observation system 1. The second control unit 94 is realized by using a general-purpose processor such as a CPU having an internal memory (not shown) in which a program is recorded, or a dedicated processor such as various arithmetic circuits that execute specific functions such as an ASIC or an FPGA. It may also be configured by using an FPGA, which is a type of programmable integrated circuit. In addition, when configured by an FPGA, a memory for storing configuration data may be provided, and the FPGA, which is a programmable integrated circuit, may be configured by the configuration data read from the memory. The second control unit 94 causes the light source device 3 to irradiate the subject with first visible light and second visible light of mutually different wavelength bands, and excitation light that excites a fluorescent material to emit fluorescence. The second control unit 94 also causes the light source device 3 to irradiate the subject with the second visible light and the excitation light at the same time. Specifically, when the medical observation system 1 is in the first observation mode, the second control unit 94 causes the light source device 3 to irradiate the first visible light and the second visible light, while when the medical observation system 1 is in the second observation mode, the second control unit 94 causes the light source device 3 to irradiate the second visible light and the excitation light. Here, the first observation mode is a white light observation mode in which white light is irradiated onto the subject. Also, the second observation mode is a fluorescence observation mode in which fluorescence is observed. More specifically, the second control unit 94 causes the light source device 3 to simultaneously irradiate visible light including at least one of green light and blue light, and excitation light that excites the fluorescent material to emit fluorescence. Furthermore, the second control unit 94 causes the microscope unit 5 to capture the reflected light of the visible light reflected by the subject and the fluorescence to generate image data.

[0051] [Medical Observation System Processing] Next, a description will be given of the processing executed by the medical observation system 1. Fig. 7 is a flowchart showing an outline of the processing executed by the medical observation system 1. Note that, in the following, for the sake of simplicity, only the white light observation mode and the fluorescence observation mode will be described among the multiple observation modes that can be executed by the medical observation system 1.

[0052] 7, the second control unit 94 judges whether or not the medical observation system 1 is set to a white light observation mode in which white light is irradiated onto the subject (step S101). If the second control unit 94 judges that the medical observation system 1 is set to the white light observation mode in which white light is irradiated onto the subject (step S101: Yes), the medical observation system 1 proceeds to step S102, which will be described later. On the other hand, if the second control unit 94 judges that the medical observation system 1 is set to a fluorescence observation mode in which fluorescence is irradiated onto the subject (step S101: No), the medical observation system 1 proceeds to step S107, which will be described later.

[0053] In step S102, the second control unit 94 causes the first light source unit 31, the second light source unit 32, and the third light source unit 33 to emit light to irradiate white light. At this time, the fourth light source unit 34 is turned off.

[0054] Next, the second control unit 94 controls the first control unit 25 to cause the imaging unit 21 to receive reflected light from the subject and capture an image (step S103).

[0055] Thereafter, the image processing unit 91 performs various image processes on the image data input from the imaging unit 21 to generate a white light observation image (step S104).

[0056] Next, the display device 8 displays the white light observation image input from the image processing unit 91 (step S105), thereby allowing a user such as a doctor to observe the observation subject.

[0057] Thereafter, the second control unit 94 judges whether or not an instruction signal to end observation of the observation object has been input from the input unit 92 (step S106). If the second control unit 94 judges that an instruction signal to end observation of the observation object has been input from the input unit 92 (step S106: Yes), the medical observation system 1 ends this process. On the other hand, if it is judged that an instruction signal to end observation of the observation object has not been input from the input unit 92 (step S106: No), the medical observation system 1 returns to the above-mentioned step S101.

[0058] In step S107, the second control unit 94 judges whether or not the medical observation system 1 is set to a fluorescence observation mode in which at least the excitation light is irradiated onto the subject (step S101). If the second control unit 94 judges that the medical observation system 1 is set to a fluorescence observation mode in which at least the excitation light is irradiated onto the subject (step S107: Yes), the medical observation system 1 proceeds to step S108, which will be described later. On the other hand, if the second control unit 94 judges that the medical observation system 1 is not set to a fluorescence observation mode in which at least the excitation light is irradiated onto the subject (step S107: No), the medical observation system 1 proceeds to step S106.

[0059] In step S108, the second control unit 94 causes the fourth light source unit 34 to emit light so as to irradiate the excitation light toward the subject administered with the fluorescent substance, and also causes the second light source unit 32 and the third light source unit 33 to emit light so as to irradiate green light and blue light toward the subject (step S108). Specifically, as shown in FIG. 7, the second control unit 94 causes the fourth light source unit 34 in the light source device 3 to emit light so as to irradiate the excitation light IR1 toward the subject O1 administered with the fluorescent substance. In this case, the second control unit 94 causes the second light source unit 32 and the third light source unit 33 to emit light simultaneously with the fourth light source unit 34, thereby irradiating the excitation light IR 1、 green light W G and Blue Light W B is directed toward the object O1.

[0060] Next, the second control unit 94 causes the imaging unit 21 to receive the fluorescent light IR2 emitted by the object O1 and capture an image, and also causes the imaging unit 21 to capture the green light W G and Blue Light W B In this case, as shown in FIG. 7, the cut filter 213 blocks the excitation light IR1 reflected by the object O1 and blocks the fluorescence IR2 and green light W from the object O1. G and Blue Light W B Each of the filters (filter R, filter G, and filter B) for each pixel (R pixel, G pixel, and B pixel) in the image sensor 212 has sensitivity in the infrared light region. Because the light source device 3 is not irradiating red light and the excitation light IR1 is blocked by the cut filter 213, only the fluorescence IR2 is incident on the R pixel of the image sensor 212. Furthermore, the fluorescence IR2 and the green light W reflected by the subject O1 are incident on the G pixel of the image sensor 212. G Furthermore, the fluorescent light IR2 and the blue light W reflected by the object O1 are incident on the B pixel of the image sensor 212. B are incident on the G pixel and B pixel. At this time, the pixel values ​​output by each of the G pixel and B pixel are dominated by the green reflected light and blue reflected light because the intensity of the fluorescence IR2 incident on each of the G pixel and B pixel is weaker than the intensity of the green reflected light and blue reflected light. Furthermore, the pixel value of the R pixel is dominated by the fluorescence IR2 because the light source device 3 is not irradiating red light. In other words, the image processing unit 91 can use the output value of the R pixel as the output value based on the fluorescence IR2, and can use the output values ​​of the G pixel and the B pixel as output values ​​based on visible light (green reflected light and blue reflected light).

[0061] Thereafter, the subtraction unit 911 performs a subtraction process to subtract the first pixel value output by the pixel (R pixel) in which the first filter (filter R) is arranged from the second pixel value output by the pixel (G pixel or B pixel) in which the second and third filters (filter G or filter B) included in the image data input from the imaging unit 21 are arranged (step S110).

[0062] Here, the subtraction process performed by the subtraction unit 911 will be described in detail. If the value of the fluorescent component contained in the pixel value output from the Gr pixel (hereinafter simply referred to as "IRgr"), the value of the fluorescent component contained in the pixel value output from the Gb pixel (hereinafter simply referred to as "IRgb"), and the value of the fluorescent component contained in the pixel value output from the B pixel (hereinafter simply referred to as "IRb"), the pixel value excluding the fluorescent component of each pixel can be estimated from the value of the fluorescent component contained in the pixel value output from the R pixel (hereinafter simply referred to as "IRr"). IR In the above, if the spectral sensitivities of the R, G (Gr, Gb) and B pixels are r, g and b [%], the fluorescent components input to the R, G (Gr, Gb) and B pixels are considered to be constant. Therefore, the following formula is established. IRgr≒IRgb≒(g / r)*IRr (1) IRb ≈ (b / r) * IRr (2) Furthermore, since the light source device 3 is not emitting red light, the pixel value of the R pixel = IRr. Therefore, if the pixel value of the R pixel is R, the pixel value of the Gr pixel is Gr, the pixel value of the Gb pixel is Gb, and the pixel value of the B pixel is B, the subtraction unit 911 calculates values ​​obtained by subtracting the values ​​of the fluorescent component from the pixel value of the G pixel (Gr = Gb) and the pixel value of the B pixel (B) by performing the following equations (3) to (5). Pixel value of Gr pixel = Gr-(g / r)*IRgr ≒ Gr-(g / r)*R (3) Pixel value of Gb pixel = Gb-(g / r)*IRgb ≒ Gb-(g / r)*R (4) Pixel value of B pixel = B-(b / r)*IRb ≒ B-(b / r)*R (5)

[0063] Next, the image processing unit 91 generates a fluorescent image and a visible light image under the control of the second control unit 94 (step S111). Specifically, the first generating unit 912 generates a fluorescent image using the pixel values ​​of each R pixel included in the image data input from the image sensor 212. In this case, as shown in FIG. 7, the first generating unit 912 generates a fluorescent image P1 in which pixel values ​​corresponding to the positions of the B pixels and G pixels are complemented by performing image processing using the pixel values ​​of each R pixel. Note that the fluorescent image P1 in FIG. 7 is hatched for convenience in order to represent fluorescence. Furthermore, the second generating unit 913 generates a background image P2 using the pixel values ​​of each G pixel and each B pixel included in the image data input from the image sensor 212 and input from the subtraction unit 911. In this case, as shown in FIG. 7, the second generating unit 913 generates a background image P2 in which pixel values ​​corresponding to the positions of the R pixels are complemented by performing demosaic processing or the like using the pixel values ​​of each G pixel and each B pixel. It should be noted that the background image P2 in FIG. 7 is hatched for the sake of convenience.

[0064] Here, a description will be given of a method for generating the fluorescent image P1 by the first generating section 912. Fig. 8 is a diagram showing a schematic diagram of the method for generating the fluorescent image P1 by the first generating section 912.

[0065] As shown in FIG. 8, the first generation unit 912 does not use the pixel values ​​of the B pixel (A3) and the G pixel (A1, A2) in the unit Z1 in the Bayer array included in the image data input from the image sensor 212, but generates the pixel values ​​of the R pixel (pixel P n,m ) as the pixel values ​​of the B pixel (A3) and the G pixel (A1, A2), thereby complementing the pixel values ​​corresponding to the positions of the B pixel and the G pixel. n,m+2 ) as the pixel values ​​of the B pixel and the G pixel (e.g., A4), thereby complementing the pixel values ​​corresponding to the positions of the B pixel and the G pixel. n+2,m) as the pixel values ​​of a B pixel and a G pixel (e.g., A5), thereby complementing the pixel values ​​corresponding to the respective positions of the B pixel and G pixel. In this way, the first generator 912 uses the pixel value of the R pixel in each unit to generate a fluorescence image P1 in which the pixel values ​​corresponding to the respective positions of the B pixel and G pixel are complemented.

[0066] FIG. 9 is a schematic diagram showing another example of the generation method used by the first generation section 912 to generate the fluorescent image P1.

[0067] 9, the first generation unit 912 generates a fluorescence image P1 by complementing pixel values ​​corresponding to the respective positions of the B and G pixels using the average value of the pixel values ​​of adjacent R pixels. Specifically, the first generation unit 912 complements the pixel value of the G pixel (A1) with the pixel value of the R pixel (pixel P n,m ) and the pixel value of R pixel (pixel P n,m+2 ) and the average value of the pixel value ((pixel P n,m (pixel value of pixel P n,m+2 Similarly, the first generation unit 912 sets the pixel value of the G pixel (A2) to the pixel value of the R pixel (pixel P n,m ) and the pixel value of R pixel (pixel P n+2,m ) and the average value of the pixel value ((pixel P n+2,m (pixel value of pixel P n,m The first generation unit 912 also calculates the pixel value of the B pixel (A3) by multiplying the pixel value of the R pixel (pixel P n,m ) and the pixel value of R pixel (pixel P n+2,m+2 ) and the average value of the pixel value ((pixel P n2,m (pixel value of pixel P n+2,m+2 In this way, the first generation section 912 generates the fluorescence image P1 by interpolating pixel values ​​corresponding to the respective positions of the B and G pixels using the average value of the pixel values ​​of the adjacent R pixels.

[0068] FIG. 10 is a diagram showing a schematic diagram of a generation method used by the second generation section 913 to generate the background image P2.

[0069] As shown in FIG. 10, the second generation unit 913 generates the R pixel (pixel P n,m ) is regarded as black (pixel value 0) and a demosaic process is performed to generate a background image P2.

[0070] FIG. 11 is a diagram showing a schematic diagram of another generation method used by the second generation section 913 to generate the background image P2.

[0071] As shown in FIG. 11, the second generation unit 913 generates the R pixel (pixel P n,m ) from the pixel values ​​of the surrounding pixels, and performs demosaic processing to generate a background image P2. For example, the second generation unit 913 n,m ) pixel value of the eight surrounding pixels (pixel P n-1,m-1 , pixel P n-1,m1 , pixel P n-1,m+1 , pixel P n,m-1 , pixel P n,m+1 , pixel P n+1,m+1 , pixel P n+1,m , pixel P n+1,m+1 ) or the average pixel value of the four surrounding G pixels (pixel P n-1,m1 , pixel P n,m-1 , pixel P n,m+1 , pixel P n+1,m ) is the average pixel value.

[0072] Thereafter, the second generating unit 913 performs a black-and-white conversion process on the background image P2 and outputs the black-and-white conversion result to the combining unit 914 (step S112). Specifically, the second generating unit 913 may generate a grayscale image by performing a grayscale conversion process on the background image P2 and output the grayscale image to the combining unit 914.

[0073] Next, the first generating unit 912 performs a coloring process on the fluorescent image P1 and outputs the result to the combining unit 914 (step S113). Specifically, the first generating unit 912 performs coloring on the fluorescent image P1 by a color tone conversion process or the like based on the luminance value of the fluorescent image P1, and outputs the resultant fluorescent image P1 to the combining unit 914. For example, the first generating unit 912 colors the fluorescent image P1 in green.

[0074] Thereafter, the synthesis unit 914 generates a synthetic image P3 by synthesizing the fluorescent image P1 generated by the first generation unit 912 and the background image P2 generated by the second generation unit 913 (step S114). Specifically, as shown in Fig. 8, the synthesis unit 914 generates a synthetic image P3 by synthesizing the fluorescent image P1 and the background image P2, and outputs this synthetic image P3 to the display device 8. In this case, the synthesis unit 914 generates the synthetic image P3 by synthesizing the fluorescent image P1 and the background image P2 at a predetermined ratio (for example, 1:1).

[0075] After that, the display device 8 displays the composite image P3 input from the composition unit 914 (step S115). As a result, as shown in Fig. 8, a user such as a doctor can grasp the position of the fluorescent region Q1 by observing the composite image P3 displayed by the display device 8. After step S112, the medical observation system 1 proceeds to step S106.

[0076] According to the above-described first embodiment, the second control unit 94 causes the light source device 3 to irradiate the second visible light and the excitation light, and the image processing unit 91 generates a background image based on the second pixel values ​​output by the pixels (G pixels, B pixels) in which the second filters (filter G, filter B) included in the image data are arranged, and a fluorescence image based on the first pixel values ​​output by the pixels (R pixels) in which the first filter (filter R) included in the image data is arranged. In this method, it is sufficient to irradiate the visible light and the infrared excitation light from the light source device 3 simultaneously, and there is no need to irradiate them alternately, so that flickering of the observation location of the observation object can be prevented and the device can be made more compact.

[0077] Furthermore, in embodiment 1, a fluorescent image P1 and a normal white light observation image can be generated using one image sensor 212 with a normal Bayer array, so that the observation location of the object can be observed by appropriately switching between the white light observation mode and the fluorescent observation mode without using a special image sensor.

[0078] Furthermore, according to the first embodiment, the image processing unit 91 performs black-and-white processing on the background image, and then combines the black-and-white processed background image with the fluorescent image, thereby making it possible to emphasize the fluorescent area on the combined image.

[0079] Furthermore, according to the first embodiment, the image processing unit 91 performs monochromatic processing on the fluorescent image, and then combines the monochromatically processed fluorescent image with the background image, making it possible to emphasize the fluorescent area on the combined image.

[0080] Furthermore, according to the first embodiment, when the medical observation system 1 is set to the white light observation mode, the second control unit 94 causes the first light source unit 31, the second light source unit 32, and the third light source unit 33 to emit light and irradiate white light, whereas when the medical observation system 1 is set to the fluorescence observation mode, the second control unit 94 causes the second light source unit 32, the third light source unit 33, and the fourth light source unit 34 to emit light and irradiate visible light and excitation light simultaneously. Therefore, it is possible to observe the observation location of the observation object by appropriately switching between the white light observation mode and the fluorescence observation mode.

[0081] Furthermore, according to the first embodiment, the image processing unit 91 subtracts from the second and third pixel values ​​(the pixel values ​​of the G pixel and the B pixel) the multiplication result obtained by dividing the spectral sensitivity of the fluorescence wavelength of the second and third filters (filter G, filter B) by the spectral sensitivity of the fluorescence wavelength of the first filter (filter R) and multiplying the result by the first pixel value (pixel value of the R pixel), and generates a background image based on the subtraction result, so that a background image excluding the fluorescent component can be generated.

[0082] In the first embodiment, in order to generate a background image, the image processing unit 91 subtracts from the second and third pixel values ​​(the pixel values ​​of the G pixel and the B pixel) the result of multiplying the first pixel value (the pixel value of the R pixel) by a value obtained by dividing the spectral sensitivity of the fluorescence wavelength of the second and third filters (filter G and filter B) by the spectral sensitivity of the fluorescence wavelength of the first filter (filter R). However, it is also possible to simply use the first pixel value as is.

[0083] In the first embodiment, when the medical observation system 1 is set to the fluorescence observation mode, the second control unit 94 causes each of the second light source unit 32 and the third light source unit 33 to emit light, but visible light may be irradiated by causing at least one of the first light source unit 31, the second light source unit 32, and the third light source unit 33 to emit light. In this case, the image processing unit 91 may generate a background image using pixel values ​​from pixels that are most sensitive to light emitted and irradiated as visible light, and generate a fluorescence image using pixel values ​​of the other pixels. Specifically, when the second light source unit 32 emits visible light and irradiates the observation subject with green light, the image processing unit 91 may generate a background image using pixel values ​​of G pixels included in the image data input from the imaging unit 21, and generate a fluorescence image using pixel values ​​of R pixels and B pixels.

[0084] Furthermore, in the first embodiment, the image processing unit 91 performs black-and-white conversion processing on the background image, but it may also perform black-and-white conversion processing on the fluorescent image, or on each of the background image and the fluorescent image. This makes it possible to emphasize the fluorescent area on the composite image. Of course, the image processing unit 91 may omit the black-and-white conversion processing on each of the background image and the fluorescent image. This makes it possible to simplify the processing.

[0085] Furthermore, in the first embodiment, the image processing unit 91 performs monochromatic processing on the fluorescent image, but it may perform coloring processing on the background image, or it may perform coloring processing on each of the background image and the fluorescent image. In this case, the image processing unit 91 performs coloring processing on the background image and the fluorescent image so that they are different colors from each other. This makes it possible to emphasize the fluorescent area on the composite image. Of course, the image processing unit 91 may omit coloring processing on each of the background image and the fluorescent image. This makes it possible to simplify the processing.

[0086] In addition, in the first embodiment, the first light source unit 31, the second light source unit 32, and the fourth light source unit 34 emit light in the fluorescence observation mode, but for example, a light source capable of emitting white light and a cut filter that blocks the blue wavelength band and transmits light in the other wavelength bands may be provided, and the cut filter may be disposed on the optical path of the white light emitted by the white light in the fluorescence observation mode. Of course, the transmission characteristics of the cut filter may be such that it blocks the green wavelength band and transmits light in the other wavelength bands.

[0087] Moreover, in the first embodiment, a cut filter that blocks light in one of the green and blue wavelength bands and transmits light in the other wavelength band is removably provided in the optical path between the optical system 211 and the cut filter 213, and can be applied even when the cut filter is inserted in the optical path between the optical system 211 and the cut filter 213 in the fluorescence observation mode.

[0088] Furthermore, in the first embodiment, when image processing unit 91 generates a composite image using a fluorescent image and a background image, the composite image may be generated by discarding parts of the fluorescent image whose luminance values ​​are lower than a predetermined value and combining the fluorescent image with the background image, thereby making it possible to further emphasize the fluorescent region.

[0089] (Modification of the first embodiment) Next, a description will be given of a modified example of the first embodiment. In the first embodiment described above, imaging is performed using one imaging element 212 (single plate) configured in a Bayer array, but in the modified example of the first embodiment, multiple imaging elements are used.

[0090] Fig. 12 is a conceptual diagram showing a configuration of an imaging section according to a modification of Embodiment 1. Imaging section 21 shown in Fig. 12 includes imaging elements 212R, 212G, and 212B, and a dichroic prism 214, instead of imaging element 212 described above.

[0091] The image sensor 212R receives the red light dispersed by the pixel portion 212a described above and the dichroic prism 214 described below, and generates image data by performing photoelectric conversion.

[0092] The imaging element 212G receives green light dispersed by the pixel portion 212a described above and a dichroic prism 214 described below, and generates image data by performing photoelectric conversion.

[0093] The imaging element 212B receives blue light dispersed by the pixel portion 212a described above and a dichroic prism 214 (described later) and performs photoelectric conversion to generate image data.

[0094] Dichroic prism 214 outputs red light and fluorescence out of the light incident via cut filter 213 to imaging element 212R, green light and fluorescence to imaging element 212G, and blue light and fluorescence to imaging element 212B.

[0095] According to the modification of the first embodiment explained above, the same effects as those of the first embodiment described above can be achieved.

[0096] (Embodiment 2) Next, a description will be given of embodiment 2. In the above-mentioned embodiment 1, a surgical microscope was described as the medical observation system, but in embodiment 2, an endoscope system having a rigid endoscope will be described as the medical observation system. Note that the same components as those in the medical observation system 1 according to the above-mentioned embodiment 1 are given the same reference numerals, and detailed description will be omitted.

[0097] [Configuration of medical observation system] Fig. 13 is a diagram showing a schematic configuration of a medical observation system according to embodiment 2. A medical observation system 1B shown in Fig. 13 is a system used in the medical field to observe biological tissue in a subject such as a living organism. Note that in embodiment 2, a rigid endoscope system using a rigid scope (insertion section 102) shown in Fig. 13 will be described as the medical observation system 1B.

[0098] The medical observation system 1B shown in FIG. 13 includes an insertion section 102, a light source device 3, a light guide 104, an endoscopic camera head 105 (an endoscopic imaging device), a first transmission cable 106, a display device 8, a second transmission cable 108, a control device 9, and a third transmission cable 1010.

[0099] The insertion section 102 is hard or at least partially soft and has an elongated shape. The insertion section 102 is inserted into a subject such as a patient via a trocar. The insertion section 102 is provided with an optical system such as a lens that forms an observation image.

[0100] One end of the light guide 104 is detachably connected to the light source device 3, and the other end is detachably connected to the insertion portion 102. The light guide 104 guides the illumination light supplied from the light source device 3 from one end to the other, and supplies it to the insertion portion 102.

[0101] The eyepiece 121 of the insertion portion 102 is detachably connected to the endoscopic camera head 105. Under the control of the control device 9, the endoscopic camera head 105 receives an observation image formed by the insertion portion 102 and performs photoelectric conversion to generate image data (RAW data), and outputs the image data to the control device 9 via a first transmission cable 106.

[0102] One end of the first transmission cable 106 is detachably connected to the control device 9 via a video connector 161, and the other end is connected to the endoscopic camera head 105 via a camera head connector 162. The first transmission cable 106 transmits image data output from the endoscopic camera head 105 to the control device 9, and also transmits setting data, power, and the like output from the control device 9 to the endoscopic camera head 105.

[0103] One end of the second transmission cable 108 is detachably connected to the display device 8, and the other end is detachably connected to the control device 9. The second transmission cable 108 transmits image data that has been subjected to image processing in the control device 9 to the display device 8.

[0104] One end of the third transmission cable 1010 is detachably connected to the light source device 3, and the other end is detachably connected to the control device 9. The third transmission cable 1010 transmits control data from the control device 9 to the light source device 3.

[0105] [Functional configuration of the main parts of the medical observation system] Next, a description will be given of the functional configuration of the main parts of the above-mentioned medical observation system 1B. Fig. 14 is a block diagram showing the functional configuration of the main parts of the medical observation system 1B.

[0106] [Configuration of the endoscope camera head] First, a description will be given of the configuration of the endoscopic camera head 105. The endoscopic camera head 105 includes an image sensor 212, a cut filter 213, a lens unit 501, a camera head memory 502, and a camera head control unit 503.

[0107] The lens unit 501 forms an object image, which is collected by the optical system of the insertion portion 102, on the light receiving surface of the image sensor 212. The lens unit 501 is capable of changing the focal position. The lens unit 501 is configured using a plurality of lenses.

[0108] Camera head memory 502 records various information related to endoscopic camera head 105 (for example, pixel information of image sensor 212, characteristics of cut filter 213). Camera head memory 502 also records various setting data and control parameters transmitted from control device 9 via first transmission cable 106. Camera head memory 502 is configured using a non-volatile memory or a volatile memory.

[0109] The camera head control unit 503 controls the operation of each unit constituting the endoscopic camera head 105 based on the setting data received from the control device 9 via the first transmission cable 106. The camera head control unit 503 is realized using a TG (Timing Generator), a processor which is a processing device having hardware such as a CPU, and a memory which is a temporary storage area used by the processor.

[0110] The medical observation system 1B configured in this way performs the same processing as the medical observation system 1 described above (see FIG. 6).

[0111] According to the second embodiment described above, the same effects as those of the first embodiment can be achieved, and the endoscopic camera head 105 can be made smaller.

[0112] (Embodiment 3) Next, a third embodiment will be described. In the third embodiment, a case where the medical observation system is applied to a flexible endoscope system using a flexible endoscope will be described. Note that the same components as those in the medical observation system 1 according to the first embodiment will be assigned the same reference numerals and detailed description will be omitted.

[0113] [Overview of medical observation system] Fig. 15 is a diagram showing a schematic configuration of a medical observation system according to embodiment 3. The medical observation system 1C shown in Fig. 15 is inserted into a subject to capture images of the inside of the subject's body, thereby generating image data, and displays an image based on the image data.

[0114] 15, the medical observation system 1C includes an endoscope 201 that captures an in-vivo image of an observation site by inserting an insertion portion 202 into a subject to generate image data, a light source device 3, a display device 8, and a control device 9. The endoscope 201 includes an imaging unit 21 provided at a tip portion 203 of the insertion portion 202.

[0115] The medical observation system 1C configured in this way performs the same processing as the medical observation system 1 described above (see FIG. 6).

[0116] According to the above-described third embodiment, even if the medical observation system 1C includes a flexible endoscope 201, it is possible to obtain the same effects as those of the first embodiment.

[0117] (Other embodiments) Various inventions can be formed by appropriately combining a plurality of components disclosed in the medical observation systems according to the above-mentioned embodiments 1 to 3 of the present disclosure. For example, some components may be deleted from all the components described in the medical observation systems according to the above-mentioned embodiments 1 to 3 of the present disclosure. Furthermore, the components described in the medical observation systems according to the above-mentioned embodiments of the present disclosure may be appropriately combined.

[0118] In addition, in the medical observation systems according to the first to third embodiments of the present disclosure, the above-mentioned "unit" can be read as "means" or "circuit," etc. For example, the control unit can be read as control means or control circuit.

[0119] In addition, the programs executed by the medical observation systems according to the first to third embodiments of the present disclosure are provided as file data in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a digital versatile disk (DVD), a USB medium, or a flash memory.

[0120] Furthermore, the programs executed by the medical observation systems according to the first to third embodiments of the present disclosure may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0121] In the description of the flowcharts in this specification, the order of processing between timings is clearly indicated using expressions such as "first," "then," and "continue," but the order of processing required to implement the present disclosure is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed as long as there is no contradiction. For example, the generation of a fluorescent image, the colorization process, and the generation of a background image and the black and white conversion process can be processed in parallel.

[0122] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the present disclosure section.

[0123] The present technology can also be configured as follows. (Appendix 1) a light source device capable of irradiating a subject with first and second visible light beams having mutually different wavelength bands and excitation light for exciting a fluorescent material to emit fluorescence; an image sensor having a pixel section having a plurality of pixels, and a color filter in which a first filter capable of transmitting the first visible light and the fluorescent light and a second filter capable of transmitting the second visible light and the fluorescent light are provided on a light receiving surface of each of the plurality of pixels, and which generates image data by capturing an image of at least one of the fluorescent light and at least one of the first visible light and the second visible light reflected from the subject; a control unit that causes the light source device to irradiate the second visible light and the excitation light; an image processing unit that generates a fluorescent image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Equipped with Medical observation system. (Appendix 2) A medical observation system according to (Appendix 1), a cut filter provided on an incident surface side of the image sensor to block the excitation light while transmitting the reflected light and the fluorescent light; Medical observation system. (Appendix 3) A medical observation system according to (Appendix 1) or (Appendix 2), The image processing unit includes: modifying second pixel values ​​based on the first pixel values, and generating the background image based on the modified second pixel values; Medical observation system. (Appendix 4) A medical observation system according to (Appendix 3), The image processing unit includes: subtracting the first pixel value from the second pixel value, and generating the background image based on a result of the subtraction; Medical observation system. (Appendix 5) A medical observation system according to any one of (Appendix 1) to (Appendix 4), The image processing unit includes: subtracting from the second pixel value a multiplication result obtained by multiplying a value obtained by dividing the spectral sensitivity of the fluorescence wavelength of the second filter by the spectral sensitivity of the fluorescence wavelength of the first filter by the first pixel value; generating the background image based on the subtraction result; Medical observation system. (Appendix 6) A medical observation system according to any one of (Appendix 1) to (Appendix 5), The control unit is In the first observation mode, the light source device is caused to irradiate the first visible light and the second visible light, In the case of a second observation mode, the light source device is caused to irradiate the second visible light and the excitation light. Medical observation system. (Appendix 7) A medical observation system according to any one of (Appendix 1) to (Appendix 6), The light source device is A third visible light having a wavelength band different from each of the first visible light and the second visible light can be irradiated, The imaging element includes: a third filter capable of transmitting the third visible light and the fluorescent light; generating the background image based on a third pixel value output by a pixel on which the third filter is arranged and which is included in the image data, and the second pixel value; Medical observation system. (Appendix 8) A medical observation system according to (Appendix 7), The first visible light is It is light in the red wavelength range, The second visible light is one of light in a green wavelength band and light in a blue wavelength band; The third visible light is the other of the light in the green wavelength band and the light in the blue wavelength band; The first filter comprises: a red filter capable of transmitting the light in the red wavelength band and the fluorescent light, The second filter comprises: one of a green filter and a blue filter capable of transmitting the one of the light in the green wavelength band and the light in the blue wavelength band, and the fluorescent light; The third filter comprises: the other of the green filter and the blue filter capable of transmitting the other of the light in the green wavelength band and the light in the blue wavelength band, and the fluorescent light; Medical observation system. (Appendix 9) A medical observation system according to (Appendix 8), The light source device includes: A first light source unit capable of emitting light in a red wavelength band; A second light source unit capable of emitting light in the green wavelength band; A third light source unit capable of emitting light in the blue wavelength band; A fourth light source unit capable of emitting the excitation light; Equipped with The control unit is In the case of a white light observation mode in which observation is performed using white light, the first light source unit, the second light source unit, and the third light source unit are caused to emit light, In the case of a fluorescence observation mode in which the fluorescence is observed, the second light source unit, the third light source unit, and the fourth light source unit are caused to emit light. Medical observation system. (Appendix 10) A medical observation system according to any one of (Appendix 1) to (Appendix 9), The image processing unit includes: generating a composite image by combining the background image and the fluorescent image; Medical observation system. (Appendix 11) A medical observation system according to any one of (Appendix 1) to (Appendix 9), The image processing unit includes: performing a black-and-white process on at least one of the background image and the fluorescent image; Medical observation system. (Appendix 12) A medical observation system according to any one of (Appendix 1) to (Appendix 9), The image processing unit includes: performing a coloring process on at least one of the background image and the fluorescent image; Medical observation system. (Appendix 13) A medical observation system according to any one of (Appendix 1) to (Appendix 12), The fluorescent material is Indocyanine green, The excitation light is The central wavelength is 740 nm. Medical observation system. (Appendix 14) a light source device capable of irradiating a subject with first and second visible light beams having mutually different wavelength bands and excitation light for exciting a fluorescent material to emit fluorescence; a dichroic prism that separates the fluorescence and at least one of the first visible light and the second visible light reflected from the subject into a plurality of wavelength bands; and a plurality of image capturing elements that receive each of the light beams into the plurality of wavelength bands separated by the dichroic prism and generate a plurality of image data. a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; an image processing unit that generates a background image and a fluorescent image based on the plurality of image data; Equipped with Medical observation system. (Appendix 15) A medical observation system according to any one of (Appendix 1) to (Appendix 14), a support portion that rotatably supports a medical imaging device including the imaging element; a base portion that rotatably holds a base end portion of the support portion and is movable on a floor surface; Further comprising: Medical observation system. (Appendix 16) A medical observation system according to any one of (Appendix 1) to (Appendix 14), an insertion section that can be inserted into a subject and has an optical system that collects the reflected light and the fluorescent light to form a subject image on a light receiving surface of the image sensor; Medical observation system. (Appendix 17) A medical observation system according to (Appendix 16), The insertion section is detachable from the medical imaging device including the imaging element. Medical observation system. (Appendix 18) A control device for controlling a light source device capable of irradiating a subject with first visible light and second visible light in mutually different wavelength bands and excitation light for exciting a fluorescent substance to emit fluorescence, and a medical imaging device having a pixel unit having a plurality of pixels and a color filter in which a first filter capable of transmitting the first visible light and the fluorescence and a second filter capable of transmitting the second visible light and the fluorescence are provided on a light receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and light reflected by the subject of at least the first visible light and the second visible light, a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; an image processing unit that generates a fluorescent image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Equipped with Control device. (Appendix 19) A control method executed by a control device that controls a light source device capable of irradiating a subject with first visible light and second visible light in mutually different wavelength bands and excitation light for exciting a fluorescent substance to emit fluorescence, and a medical imaging device having a pixel unit having a plurality of pixels and a color filter in which a first filter capable of transmitting the first visible light and the fluorescence and a second filter capable of transmitting the second visible light and the fluorescence are provided on a light receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and light reflected by the subject of at least one of the first visible light and the second visible light, causing the light source device to simultaneously irradiate the second visible light and the excitation light; generating a fluorescence image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Control methods. [Explanation of symbols]

[0124] 1,1B,1C Medical Observation System 2. Observation equipment 3 Light source device 4 Light Guide 5. Microscope Section 6 Support part 7 Base 8 Display device 9 Control device 21 Imaging unit 22 Light emitting part 23 Detection unit 24 Input section 25 First control section 31 First light source unit 32 Second light source section 33 Third light source 34 Fourth Light Source 91 Image Processing Section 92 Input section 93 Recording Department 94 Second control section 102 Insertion part 104 Light Guide 105 Endoscope camera head 106 First Transmission Cable 108 Second Transmission Cable 121 Eyepiece 161 Video Connector 162 Camera head connector 201 Endoscope 202 Insertion part 203 Tip 211 Optical system 212, 212B, 212G, 212R Image sensor 212a Pixel section 212b Color Filter 213 Cut Filter 214 Dichroic Prism 501 Lens unit 502 Camera Head Memory 503 Camera head control unit 911 Subtraction section 912 First Generation Section 913 Second Generation 914 Synthesis Section 931 Program Recording Section 1010 Third Transmission Cable P1 Fluorescence image P2 Background Image P3 composite image Q1 Fluorescence region

Claims

1. a light source device capable of irradiating a subject with first visible light and second visible light having mutually different wavelength bands and excitation light for exciting a fluorescent material to emit fluorescence; an image sensor having a pixel unit including a plurality of pixels, a first filter capable of transmitting the first visible light and the fluorescent light and a second filter capable of transmitting the second visible light and the fluorescent light, on a light receiving surface of each of the plurality of pixels, and configured to generate image data by capturing an image of at least one of the fluorescent light and at least one of the first visible light and the second visible light reflected from the subject; a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; an image processing unit that generates a fluorescent image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Equipped with The image processing unit includes: modifying a second pixel value based on the first pixel value, subtracting the first pixel value from the modified second pixel value, and generating the background image based on a result of the subtraction; A black and white process is performed on the background image; A coloring process is performed on the fluorescent image. generating a composite image by combining the black-and-white processed background image and the color processed fluorescent image; Medical observation system.

2. 2. The medical observation system according to claim 1, a cut filter provided on an incident surface side of the image sensor to block the excitation light while transmitting the reflected light and the fluorescent light; Medical observation system.

3. 2. The medical observation system according to claim 1, The control unit is In the case of a first observation mode, the light source device is caused to irradiate the first visible light and the second visible light, In the case of a second observation mode, the light source device is caused to irradiate the second visible light and the excitation light. Medical observation system.

4. 2. The medical observation system according to claim 1, The light source device is A third visible light having a wavelength band different from each of the first visible light and the second visible light can be irradiated, The imaging element includes: a third filter capable of transmitting the third visible light and the fluorescent light; generating the background image based on a third pixel value output by a pixel on which the third filter is arranged and which is included in the image data, and the second pixel value; Medical observation system.

5. 5. The medical observation system according to claim 4, The first visible light is It is light in the red wavelength range, The second visible light is one of light in a green wavelength band and light in a blue wavelength band; The third visible light is the other of the light in the green wavelength band and the light in the blue wavelength band; The first filter comprises: a red filter capable of transmitting the light in the red wavelength band and the fluorescent light, The second filter comprises: one of a green filter and a blue filter capable of transmitting the one of the light in the green wavelength band and the light in the blue wavelength band, and the fluorescent light; The third filter includes: the other of the green filter and the blue filter capable of transmitting the other of the light in the green wavelength band and the light in the blue wavelength band, and the fluorescent light; Medical observation system.

6. 6. The medical observation system according to claim 5, The light source device is A first light source unit capable of emitting light in a red wavelength band; A second light source unit capable of emitting light in the green wavelength band; a third light source unit capable of emitting light in the blue wavelength band; A fourth light source unit capable of emitting the excitation light; Equipped with The control unit is In the case of a white light observation mode in which observation is performed using white light, the first light source unit, the second light source unit, and the third light source unit are caused to emit light, In the case of a fluorescence observation mode in which the fluorescence is observed, the second light source unit, the third light source unit, and the fourth light source unit are caused to emit light. Medical observation system.

7. 2. The medical observation system according to claim 1, The fluorescent material is Indocyanine green, The excitation light is The central wavelength is 740 nm. Medical observation system.

8. 2. The medical observation system according to claim 1, a support portion that rotatably supports a medical imaging device including the imaging element; a base portion that rotatably holds a base end portion of the support portion and is movable on a floor surface; Further comprising: Medical observation system.

9. 2. The medical observation system according to claim 1, an insertion section that can be inserted into a subject and has an optical system that collects the reflected light and the fluorescent light to form a subject image on a light receiving surface of the image sensor; Medical observation system.

10. 10. The medical observation system according to claim 9, The insertion section is detachable from the medical imaging device including the imaging element. Medical observation system.

11. A control device for controlling a light source device capable of irradiating a subject with first visible light and second visible light in mutually different wavelength bands and excitation light for exciting a fluorescent substance to emit fluorescence, and a medical imaging device having a pixel unit having a plurality of pixels, and a first filter capable of transmitting the first visible light and the fluorescence and a second filter capable of transmitting the second visible light and the fluorescence on a light receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and light reflected by the subject of at least one of the first visible light and the second visible light, a control unit that causes the light source device to simultaneously irradiate the second visible light and the excitation light; an image processing unit that generates a fluorescent image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Equipped with The image processing unit includes: modifying a second pixel value based on the first pixel value, subtracting the first pixel value from the modified second pixel value, and generating the background image based on a result of the subtraction; A black and white process is performed on the background image; A coloring process is performed on the fluorescent image. generating a composite image by combining the black-and-white processed background image and the color processed fluorescent image; Control device.

12. A control method executed by a control device that controls a light source device capable of irradiating a subject with first visible light and second visible light in mutually different wavelength bands and excitation light for exciting a fluorescent substance to emit fluorescence, and a medical imaging device having a pixel unit having a plurality of pixels, and a first filter capable of transmitting the first visible light and the fluorescence and a second filter capable of transmitting the second visible light and the fluorescence on a light receiving surface of each of the plurality of pixels, and generating image data by capturing at least one of the fluorescence and light reflected by the subject of at least one of the first visible light and the second visible light, a control step of causing the light source device to simultaneously irradiate the second visible light and the excitation light; an image processing step of generating a fluorescent image based on a first pixel value output by a pixel on which the first filter is arranged, which is included in the image data, and a background image based on a second pixel value output by a pixel on which the second filter is arranged; Including, The image processing step includes: modifying a second pixel value based on the first pixel value, subtracting the first pixel value from the modified second pixel value, and generating the background image based on a result of the subtraction; A black and white process is performed on the background image; A coloring process is performed on the fluorescent image. generating a composite image by combining the black-and-white processed background image and the color processed fluorescent image; Control methods.

Images