Medical image processing device and medical observation system

Abstract

A medical image processing device 9 includes an image processing unit 92 for processing pixel signals acquired from pixels of an imaging element 513. The pixel signal includes: a first pixel signal having an unnecessary light component containing autofluorescence generated by light from a member forming an optical path for observation when light emitted from the light source device propagates through the optical path for observation, and an observation target fluorescence component containing fluorescence generated by the member forming the optical path for observation when light emitted from the light source device propagates through the optical path for observation; the observation object fluorescence component is emitted from a substance contained in the observation object excited by light; and a second pixel signal having at least an unnecessary light component. The image processing unit 92 includes a signal correction unit 923 that generates a corrected pixel signal on the basis of the first and second pixel signals and a correction coefficient set on the basis of spectral characteristics of fluorescence to be observed.

Description

Technical Field

[0001] This invention relates to medical image processing devices and medical observation systems. Background Technology

[0002] Previously, a medical observation system (for example, see Patent Document 1) was known to irradiate an object of observation (such as a human subject) with a first light emitting visible light (such as excitation light as narrow band light or white light as broadband light) emitted from a light source device, and to observe the fluorescence emitted by a substance contained in the object of observation (hereinafter referred to as object fluorescence) through the irradiation of the excitation light.

[0003] Such fluorescence observation allows us to understand tissue conditions that are difficult to discern under white light by observing the fluorescence of the object. Therefore, fluorescence observation can be used for various purposes and applications, such as lesion identification.

[0004] Reference List

[0005] Patent documents

[0006] Patent Document 1: JP 2021-132695 A Summary of the Invention

[0007] Technical issues

[0008] On the other hand, the medical observation system has an observation optical path, which includes: a first optical path for propagating first light from a light source device to the object being observed; and a second optical path for propagating second light, which serves as return light (containing fluorescence of the object being observed) from the first light irradiated by the first light. When the first and second light propagate through the observation optical path, applying the first and second light to the components constituting the observation optical path can induce spontaneous fluorescence generated from the components. This spontaneous fluorescence may contain wavelengths of the fluorescence of the object being observed (which is the object of fluorescence observation) and may become noise during fluorescence observation.

[0009] Therefore, a technique that can effectively perform fluorescence observation is needed.

[0010] The present invention is made in view of the above circumstances, and its object is to provide a medical image processing device and a medical observation system capable of performing fluorescence observation well.

[0011] Solution to the problem

[0012] To address the aforementioned problems and achieve the objective, the medical image processing apparatus according to this disclosure includes an image processing unit configured to process pixel signals acquired from pixels of an image sensor. The pixel signals include: a first pixel signal containing a component of non-essential light and a component of fluorescence of the observed object, the non-essential light including spontaneous fluorescence generated from components constituting the observation optical path by light emitted from a light source device as it propagates through the observation optical path, the component of fluorescence of the observed object being emitted by a light-excited substance contained in the observed object; and a second pixel signal containing at least the component of non-essential light. The image processing unit also includes a signal correction unit configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal, and the second pixel signal, the correction coefficient being set based on the spectral characteristics of the fluorescence of the observed object.

[0013] A medical observation system according to this disclosure includes: a light source device including a first light source configured to emit first light comprising narrowband light that excites a substance contained in an object under observation; an imaging device including an image sensor configured to capture second light, the second light being a return light from the first light from the object under observation; and a medical image processing device including an image processing unit configured to process pixel signals corresponding to pixels of the image sensor, wherein the pixel signals include: a first pixel signal comprising a component of non-essential light and a component of fluorescence of the object under observation, the non-essential light comprising spontaneous fluorescence generated from a component of the observation optical path constituting the first light as the first light emitted from the light source device propagates through the observation optical path, the component of fluorescence of the object under observation being emitted from a substance contained in the object under observation excited by the first light; and a second pixel signal comprising at least a component of non-essential light, and the image processing unit including a signal correction unit configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal, and the second pixel signal, the correction coefficient being set based on the spectral characteristics of the fluorescence of the object under observation.

[0014] Advantages of the present invention

[0015] The medical image processing apparatus and medical observation system according to the present invention can perform fluorescence observation well. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the configuration of a medical observation system according to an embodiment.

[0017] Figure 2 This is a block diagram showing the configuration of the camera and control device.

[0018] Figure 3 This is a diagram showing the configuration of the color filters.

[0019] Figure 4 This is a diagram showing the configuration of the color filters.

[0020] Figure 5 This is a diagram illustrating a first variation of the implementation method.

[0021] Figure 6 This is a diagram illustrating a first variation of the implementation method.

[0022] Figure 7 This is a diagram illustrating a first variation of the implementation method.

[0023] Figure 8 This is a diagram illustrating a first variation of the implementation method.

[0024] Figure 9 This is a diagram illustrating a second variation of the embodiment.

[0025] Figure 10 This is a diagram illustrating a second variation of the embodiment.

[0026] Figure 11 This is a diagram illustrating a third variation of the embodiment.

[0027] Figure 12 This is a diagram illustrating a fourth variation of the embodiment.

[0028] Figure 13 This is a diagram illustrating a fourth variation of the embodiment.

[0029] Figure 14 This is a diagram illustrating a fourth variation of the embodiment.

[0030] Figure 15 This is a diagram illustrating a fifth variation of the embodiment.

[0031] Figure 16 This is a diagram illustrating a fifth variation of the embodiment.

[0032] Figure 17 This is a diagram illustrating a sixth variation of the embodiment.

[0033] Figure 18 This is a diagram illustrating a sixth variation of the embodiment.

[0034] Figure 19 This is a diagram illustrating the seventh variation of the embodiment.

[0035] Figure 20 This is a diagram illustrating the eighth variation of the embodiment.

[0036] Figure 21 This is a diagram illustrating the ninth variation of the embodiment.

[0037] Figure 22 This is a diagram illustrating the ninth variation of the embodiment. Detailed Implementation

[0038] In the following description, methods for implementing this disclosure (hereinafter referred to as embodiments) will be described with reference to the accompanying drawings. Note that this disclosure is not limited to the embodiments described below. In the drawings, the same parts are labeled with the same reference numerals.

[0039] [Configuration of the Medical Observation System]

[0040] Figure 1 This is a diagram showing the configuration of the medical observation system 1 according to this embodiment.

[0041] In this embodiment, the medical observation system 1 is a medical endoscope system that uses an endoscope to observe an object OB inside a living organism. For example... Figure 1 As shown, the medical observation system 1 includes an insertion unit 2, a light source device 3, a light guide 4, a camera 5, a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

[0042] In this embodiment, the insertion unit 2 is configured as a rigid endoscope. That is, the insertion unit 2 has an elongated shape that is entirely rigid or partially rigid with a portion of flexibility, so as to be inserted into the object of observation OB. The insertion unit 2 includes an optical system having one or more lenses and is configured to converge the second light (the image of the subject) from the object of observation OB, as described below.

[0043] The light source device 3 is connected to one end of the light guide 4. The light source device 3 includes a first light source 31 ( Figure 1 The first light source, under the control of the control device 9, provides first light to one end of the light guide 4. In this embodiment, excitation light (narrowband light) is used as the first light, which excites the material contained in the observed object OB. The first light source 31 can be configured as a light-emitting diode (LED) or a semiconductor laser. The number of first light sources 31 emitting the first light can be one or more. Furthermore, the excitation light can be visible light or invisible light. In addition to the first light source 31, the light source device 3 may also include a second light source emitting broadband light such as white light.

[0044] Examples of substances contained in the object of observation OB that are excited by excitation light include reagents or fluorescent dyes applied to the object of observation OB, or fluorescent substances derived from the object of observation OB, i.e., substances that constitute the object of observation OB.

[0045] Examples of the reagents applied to the observed object OB include “5-ALA (PP-IX)”, “ADS780WS”, “ADS830WS”, “Aggregation-induced emission phycoerythrin (APC)”, “BODIPY”, “CLR 1502”, “Flavins”, “fluorescamine”, “fluorescein”, “fluoro-gold”, “green fluorescent protein”, “indocyanine green (ICG)”, “IRDye 78”, “IR-PEG nanoparticles”, “isothiocyanate”, “rose Bengal”, “SGM-101” and “trypan blue”.

[0046] Additionally, examples of the aforementioned fluorescent dyes applied to the observed object OB include: coumarin, Cy3, DyLight547, GE3126, metal nanoclusters, oxacarbocyanine, Rhodamine, Riboflavin, Fluorescein, AlexaFluor660, AlexaFluor680, AlexaFluor700, Cy5, Cy5.5, Dy677, Dy682, Dy752, DyLight647, HiLyte Fluor 647, HiLyte Fluor 680, IRDye700DX, and methylene blue. "blue", "Porphyrins", "Porphysomes", "VivoTag-680", "VivoTagS680", "AlexaFluor750", "AlexaFluor790", "carbocyanine", "conjugated copolymer", "CW800-CA", "Cy7", "Cy7.5", "cyaninedyes", "Dy780", "HiLyte Fluor 750", "indole carbocyanine", "IR-786", "IRDye 800CW", "IRDye800RS", "IRDye 800BK", "Nervelight", "OTL-38 (Pafratinine)", "Polymethine", "VivoTag-S750", "Xanthene", and "LUM-015".

[0047] In addition, examples of fluorescent substances derived from the observed object OB that constitute the observed object OB itself include "collagen", "elastin" and "NADH".

[0048] In this embodiment, the light source device 3 and the control device 9 are separate. However, the configuration is not limited to this; the light source device 3 can also be installed inside the same housing as the control device 9.

[0049] One end of the light guide 4 is detachably connected to the light source device 3. The other end of the light guide 4 is detachably connected to the insertion unit 2. The light guide 4 transmits first light provided from the light source device 3 (first light source 31) from one end to the other end and provides the light to the insertion unit 2. The first light provided to the insertion unit 2 is emitted from the distal end of the insertion unit 2 and illuminates the object OB. After illuminating the object OB, the first light returns from the object OB, and this returned light, defined as the second light (subject image), is converged by the optical system within the insertion unit 2. The second light includes: the first light reflected by the object OB; and fluorescence emitted by a substance contained in the object OB due to the first light (excitation light) illuminating the object OB (hereinafter referred to as object fluorescence).

[0050] Camera 5 corresponds to the imaging apparatus according to this disclosure. Camera 5 is detachably connected to the eyepiece 21 of the insertion unit 2. Under the control of the control device 9, camera 5 captures the second light converged by the insertion unit 2 and generates a pixel signal. Hereinafter, for ease of description, the pixel signal is referred to as the captured image.

[0051] The detailed configuration of Camera 5 will be described in the section "Camera Configuration" below.

[0052] One end of the first transmission cable 6 is connected to connector CN1 ( Figure 1 The first transmission cable 6 is detachably connected to the control device 9. The other end of the first transmission cable 6 is connected to connector CN2. Figure 1 The first transmission cable 6 is detachably connected to the camera 5. The captured image output from the camera 5 is transmitted to the control device 9, and the control signals, synchronization signals, clock, power, etc. output from the control device 9 are transmitted separately to the camera 5.

[0053] Captured images transmitted from camera 5 to control device 9 via the first transmission cable 6 can be transmitted as optical signals or as electrical signals. The same applies to transmitting control signals, synchronization signals, and clock signals from control device 9 to camera 5 via the first transmission cable 6.

[0054] The display device 7 uses a liquid crystal or organic electroluminescent (EL) display, and displays images based on video signals from the control device 9 under the control of the control device 9. Note that the display includes wearable displays.

[0055] One end of the second transmission cable 8 is detachably connected to the display device 7. The other end of the second transmission cable 8 is detachably connected to the control device 9. The second transmission cable 8 transmits the video signal processed by the control device 9 to the display device 7.

[0056] The control device 9 corresponds to the medical image processing device according to the present disclosure. The control device 9 is implemented by a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), etc., and comprehensively controls the operation of the light source device 3, the camera 5, and the display device 7.

[0057] The detailed configuration of control device 9 will be described in the section "Configuration of Control Device" below.

[0058] One end of the third transmission cable 10 is detachably connected to the light source device 3. The other end of the third transmission cable 10 is detachably connected to the control device 9. The third transmission cable 10 transmits control signals from the control device 9 to the light source device 3.

[0059] [Camera Configuration]

[0060] Next, the configuration of camera 5 will be described.

[0061] Figure 2 This is a block diagram showing the configuration of camera 5 and control device 9.

[0062] like Figure 2 As shown, the camera 5 includes an imaging unit 51 and a communication unit 52.

[0063] Imaging unit 51 captures the observed object OB under the control of control device 9. For example... Figure 2 As shown, the imaging unit 51 includes an excitation light cutoff color filter 511, a lens unit 512, an image sensor 513, a color filter 514, and a signal processing unit 515.

[0064] The excitation cutoff color filter 511 is a color filter that partially, substantially, or completely suppresses the excitation light contained in the second light (subject image) converged by the insertion unit 2.

[0065] The excitation light cutoff color filter 511 can be disposed in the insertion unit 2 (e.g., eyepiece 21), and is not limited to the camera 5.

[0066] The lens unit 512 converges the second light (the image of the subject) that has been converged by the insertion unit 2 and passed through the excitation light cutoff filter 511 onto the light receiving surface of the image sensor 513.

[0067] In this embodiment, only one image sensor 513 is provided. The image sensor 513 is configured as a charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS), etc., which receives second light (subject image) that passes through the excitation light cutoff filter 511 and is focused by the lens unit 512, and converts the received light into an electrical signal. The image sensor 513 captures the second light (subject image) under the control of the control device 9. Through this process, a pixel signal (captured image) is generated.

[0068] Color filter 514 corresponds to the spectral dispersion component according to this disclosure. Color filter 514 is disposed on the light-receiving surface of image sensor 513.

[0069] The detailed configuration of color filter 514 will be described in the section “Configuration of Color Filter” below.

[0070] Under the control of the control device 9, the signal processing unit 515 performs signal processing on the captured image (analog signal) generated by the image sensor 513 and outputs the captured image (digital signal).

[0071] For example, the signal processing unit 515 performs signal processing on the captured image (analog signal) generated on the image sensor 513, such as processing to remove reset noise, processing to multiply the analog signal by analog gain to amplify the analog signal, A / D conversion, etc.

[0072] The communication unit 52 functions as a transmitter, sequentially transmitting the captured images output from the imaging unit 51 to the control device 9 via the first transmission cable 6. For example, the communication unit 52 includes a high-speed serial interface that communicates with the control device 9 via the first transmission cable 6 at a transmission rate of 1 Gbps or higher for the captured images. Incidentally, for example, if the image sensor 513 is an image sensor with HD resolution or lower resolution, the communication unit 52 can communicate at a transmission rate below 1 Gbps.

[0073] [Color Filter Configuration]

[0074] Next, the configuration of color filter 514 will be described.

[0075] Figure 3 and Figure 4 This is a diagram showing the configuration of color filter 514. Specifically, Figure 3 This is a diagram showing the color filter 514. Figure 4This is a diagram showing the spectral characteristics of color filter 514. The spectral characteristics of color filters, such as color filter 514, are also called transmission spectral characteristics. Figure 4 In the figure, the horizontal axis represents the wavelength [nm], and the vertical axis represents the relative response [%].

[0076] Color filter 514 is a color filter arranged in a specific form (e.g., Bayer array) by grouping three groups according to the wavelength of the light to be transmitted (red, green and blue).

[0077] Specifically, such as Figure 3 As shown, the color filter 514 includes: an R color filter group 514r that primarily transmits light in the red band; a B color filter group 514b that primarily transmits light in the blue band; a first G color filter group (arranged in the same column as the R color filter group 514r) that primarily transmits light in the green band; and a second G color filter group (arranged in the same column as the B color filter group 514b) that primarily transmits light in the green band. Figure 3 In this context, the first G filter group and the second G filter group are collectively referred to as G filter group 514g. Figure 3 In the diagram, the R color filter group 514r is labeled with the letter "R", the color filter group 514g with the letter "G", and the B color filter group 514b with the letter "B". Figure 4 In the diagram, the spectral characteristics of the R color filter group 514r are represented by curve CR, the spectral characteristics of the G color filter group 514g are represented by curve CG, and the spectral characteristics of the B color filter group 514b are represented by curve CB. As can be seen from curves CR, CG, and CB, although the transmission peak wavelengths of color filter groups 514r, 514g, and 514b are different from each other, their spectral characteristics overlap in some bands.

[0078] exist Figure 4 In the attached diagram, the label "SPF1" indicates the fluorescence spectrum of the observed object. Figure 4 In the attached figure, the reference numeral "SPU" indicates the wavelength range capable of producing the following non-essential light.

[0079] The first light emitted from the light source device 3 passes through the first optical path P1 ( Figure 1 The object OB is illuminated by the first light. The first optical path P1 is the path from the light source device 3 through the light guide 4 and the insertion unit 2 to the object OB. When the object OB is illuminated by the first light, the substance contained in the object OB is excited, and the object fluorescence is emitted from the substance. The object fluorescence is the object light in fluorescence observation. The second light, as the return light of the first light containing the object light, travels through the second optical path P2 (…). Figure 1The light propagates to the image sensor 513. The second optical path P2 is the optical path along the path from the observed object OB through the insertion unit 2 to the image sensor 513. The excitation light contained in the second light is partially, substantially, or completely suppressed by the excitation light cutoff filter 511 as it propagates through the second optical path P2.

[0080] Here, when the first light propagates in the first optical path P1 and when the second light propagates in the second optical path P2, the observation optical path P0 constitutes the first light P1 and the second light P2. Figure 1 A component of an optical system (hereinafter referred to as an unnecessary light generating member) is irradiated by a first light and a second light, thereby potentially generating spontaneous fluorescence (hereinafter referred to as unnecessary light) from the unnecessary light generating member. Examples of unnecessary light generating members include: components contained in multi-component glass such as optical fibers and lenses; material components contained in color filters; adhesives used to bond lenses to other lenses or to bond other optical components to each other; and oils adhering to optical components such as lenses. Figure 4 The wavelength range of the SPU that can generate unnecessary light has a band that includes the fluorescence of the object being observed, which is the object light in fluorescence observation and may be noise during fluorescence observation.

[0081] When the fluorescence of the observed object OB is particularly weak, it is necessary to adjust the signal value based on the fluorescence of the observed object captured by the image sensor 513 to separate the fluorescence of the observed object from the aforementioned unnecessary light in order to perform good fluorescence observation. However, due to the following (1) to (4), it is difficult to adjust the signal value based on the fluorescence of the observed object.

[0082] (1) Reagents

[0083] Typically, the amount of fluorescence emitted from a reagent varies depending on the type and dosage of the reagent.

[0084] The type of reagent to be applied to the observable OB (e.g., cancer, blood, and lymph) is selected based on the observed object (OB). Furthermore, in order to capture the fluorescence emitted by the reagent from the medical observation system 1, the selected reagent is one capable of distinguishing the wavelength of the excitation light used to excite the reagent from the wavelength of the fluorescence emitted by the reagent. In this way, the fluorescence emitted by the selected reagent has different light intensities.

[0085] Furthermore, although the amount of fluorescence observed in the target object can be adjusted by adjusting the dosage of the reagent, it is difficult to increase the dosage beyond what is required to achieve minimally invasive treatment.

[0086] That is, it is difficult to adjust the amount of fluorescence of the observed object by selecting the type of reagent and adjusting the dosage. Therefore, it is difficult to adjust the signal value based on the fluorescence of the observed object captured by the image sensor 513 by selecting the type of reagent and adjusting the dosage.

[0087] (2) Observation object

[0088] The amount of fluorescence emitted by the observed object OB varies depending on the location / state of the observed object OB.

[0089] Specifically, when the object of observation OB is a site where reagents tend to stagnate, the amount of fluorescence emitted from the object increases. Conversely, when the object of observation OB is a site where reagents flow freely and do not stagnate, the amount of fluorescence emitted from the object decreases as the afterglow time decreases. Furthermore, when the object of observation OB is a tumor, the amount of fluorescence received by the image sensor 513 varies depending on the extent, size, and depth of the tumor.

[0090] That is, it is difficult to adjust the amount of fluorescence of the observed object based on the type and state of the observed object (OB). Therefore, it is difficult to adjust the signal value of the observed object fluorescence based on the image sensor 513 based on the type and state of the observed object (OB).

[0091] (3) Light source device

[0092] The amount of fluorescence observed in the object varies depending on the amount of excitation light emitted from the light source device 3.

[0093] To increase the intensity of the excitation light, it may be necessary to adjust the power supplied to the light source device 3. However, the power that can be supplied to the light source device 3 is limited by the upper limit of the power required to operate the entire medical observation system 1. Additionally, adjusting the intensity of the excitation light requires consideration of the following conditions: heat generation in the components constituting the optical path of the excitation light (e.g., heat generation between the light guide 4 and the insertion unit 2); compliance with laser safety standards; the light energy received by the observed object OB or a nearby living organism (there is a risk of burns when the light energy is high); or the decay rate of the fluorescence emitted from the reagent from the observed object. The intensity of the excitation light can also be adjusted by changing the number of excitation light sources mounted on the light source device 3. However, the number of light sources affects the size of the light source device 3. The size of the light source device 3 may be limited by the size of the trolley or similar device used to support it.

[0094] In other words, it is difficult to adjust the amount of fluorescence of the observed object by adjusting the amount of excitation light. Therefore, it is difficult to adjust the signal value of the fluorescence of the observed object based on the image sensor 513 by adjusting the amount of excitation light.

[0095] (4) Image sensor

[0096] The signal value generated by the image sensor 513 based on the fluorescence of the observed object varies depending on the amount of fluorescence light received by the image sensor 513.

[0097] To adjust the signal value based on the fluorescence of the observed object, it is desirable to select an image sensor 513 with optimal sensitivity, configuration, etc., for imaging the fluorescence of the observed object. However, the image sensor 513 needs to output images of fluorescence observation not only based on the fluorescence of the observed object received in a predetermined wavelength band, but also images of normal light observation based on visible light such as white light. Furthermore, the image sensor 513 may also need to output images of fluorescence observation corresponding to a wide band or multiple bands including visible and invisible light. Moreover, even using the same reagents, the amount of fluorescence of the observed object may change depending on the process or the observed object OB. Therefore, the image sensor 513 may also need to output images of fluorescence observation corresponding to such changes in the amount of fluorescence of the observed object. In this case, the image sensor 513 needs to be selected with sensor elements capable of supporting these observations, resulting in situations where an image sensor with optimal characteristics for fluorescence imaging in a predetermined wavelength band cannot be used. Furthermore, the image sensor 513 is housed in the camera 5, and since the camera 5 needs to have a suitable size and weight for observation, the type (including its size) of the image sensor 513 may be limited. Not only in the configuration where the image sensor 513 is located in the camera 5, but also in the case where the image sensor 513 is located at the distal end of a rigid or flexible endoscope, the image sensor 513 needs to have a size and weight suitable for observation, which limits the type of image sensor 513 (including its size).

[0098] That is, it is difficult to adjust the signal value based on the fluorescence of the observed object imaged by the image sensor 513 by selecting the type of image sensor 513.

[0099] As described above, the signal value of the fluorescence of the observed object captured by the image sensor 513 is determined within the aforementioned constraints and therefore cannot be easily adjusted.

[0100] In this embodiment, such as Figure 4 As shown in the SPF1 spectrum, the fluorescence of the observed object exhibits spectral characteristics in the band with a center wavelength of approximately 650 nm. In other words, in this embodiment, the fluorescence of the observed object is light in the visible light band. Furthermore, as... Figure 4 The wavelength range of unwanted light that may be generated is shown in the SPU. Unwanted light has spectral characteristics covering the entire visible and invisible light bands. Spectral characteristics such as the fluorescence of the observed object and unwanted light are also called radiation spectral characteristics. In this embodiment, Figure 4The spectral characteristics of the fluorescence and unnecessary light of the observed object are shown. However, spectral characteristics may vary depending on the type and concentration of the fluorophore, the wavelength characteristics and intensity of the excitation light, and the type and number of components constituting the optical path. In other words, the spectral characteristics of the fluorescence and unnecessary light of the observed object can be other spectral characteristics, and are not limited to... Figure 4 Those shown in the image.

[0101] Therefore, based on the spectral characteristics of the R filter group 514r and the spectral characteristics of the fluorescence and unnecessary light of the observed object, the signal value of the pixel signal corresponding to the pixel of the R filter group 514r (hereinafter referred to as the r value) contains components of the fluorescence and unnecessary light of the observed object. Furthermore, based on the spectral characteristics of the G filter group 514g and the spectral characteristics of the fluorescence and unnecessary light of the observed object, the signal value of the pixel signal corresponding to the pixel of the G filter group 514g (hereinafter referred to as the g value) mainly contains components of unnecessary light. Moreover, based on the spectral characteristics of the B filter group 514b and the spectral characteristics of the fluorescence and unnecessary light of the observed object, the signal value of the pixel signal corresponding to the pixel of the B filter group 514b (hereinafter referred to as the b value) also mainly contains components of unnecessary light. In other words, compared to the g value and the b value, the r value contains more components of the fluorescence of the observed object.

[0102] As described above, the r, g, and b values ​​contain components of unnecessary light. In this embodiment, in order to perform fluorescence observation well, image processing (correction matrix processing) performed by the control device 9 (described later) is performed to generate a corrected pixel signal that suppresses the components of unnecessary light that would otherwise become noise during fluorescence observation (i.e., a signal with reduced signal values ​​based on unnecessary light).

[0103] Note that the color filter groups forming the color filter 514 are not limited to the color filter groups 514r, 514g, and 514b that mainly transmit R, G, and B band light. That is to say, the color filter 514 is not limited to color filters with the above-mentioned spectral characteristics, but can also be composed of multiple color filter groups with other spectral characteristics.

[0104] [Control device configuration]

[0105] Next, we will refer to Figure 2 Describe the configuration of control device 9.

[0106] like Figure 2 As shown, the control device 9 includes a communication unit 91, an image processing unit 92, a control unit 93, an input unit 94, an output unit 95, and a storage unit 96.

[0107] The communication unit 91 functions as a receiver for receiving captured images sequentially transmitted from the camera 5 (communication unit 52) ​​via the first transmission cable 6. For example, the communication unit 91 includes a high-speed serial interface that communicates with the communication unit 52 for captured images at a transmission rate of 1 Gbps or higher. Incidentally, for example, if the image sensor 513 is an image sensor with HD resolution or lower, the communication unit 91 can communicate at a transmission rate of less than 1 Gbps.

[0108] Under the control of the control unit 93, the image processing unit 92 processes the captured images sequentially output from the camera 5 (communication unit 52) ​​and received by the communication unit 91. For example... Figure 2 As shown, the image processing unit 92 includes a WB processing unit 921, a demosaic processing unit 922, a correction matrix processing unit 923, a gamma processing unit 924, and a YC converter 925.

[0109] The WB processing unit 921 performs white balance (WB) processing, multiplying each of the r, g, and b values ​​in the captured image by a specific gain. In the following text, the r, g, and b values ​​in the captured image output from the WB processing unit 921 are respectively determined by R... 921 G 921 and B 921 express( Figure 2 ).

[0110] The demosaic processing unit 922 performs demosaic processing on the captured image after WB processing, assigning r, g, and b values ​​(R, g, and b) to each pixel in the captured image by interpolation. 922 G 922 and B 922 The signal value of ).

[0111] Here, in this embodiment, as described above, based on the spectral characteristics of the color filter 514 and the spectral characteristics of the observed object's fluorescence and unnecessary light, the r value has more of the observed object's fluorescence component compared to the g value and b value. Therefore, the signal value (R) after demosaicing... 922 G 922 and B 922 In ), with the signal value (G) 922 and B 922 Compared to the signal value (R), 922 It contains more of the fluorescence component of the observed object. Therefore, the signal value (R) 922 ) corresponds to the first pixel signal (signal value) according to this disclosure. Furthermore, the signal value (G) 922 and B 922 This corresponds to the second pixel signal (signal value) according to this disclosure. Signal value (R) 922The signal value (G) is the signal value in which the fluorescence component of the observed object is greater than the non-essential light component and the fluorescence component of the observed object is dominant. On the other hand, the signal value (G) 922 and B 922 () is a signal value where the non-essential light component is dominant.

[0112] In the aforementioned WB processing, the r, g, and b values ​​are multiplied by a specific gain to ensure that the components of unnecessary light contained in the r, g, and b values ​​are essentially the same (a process that adjusts the magnitude relationship of the unnecessary light components). In other words, in WB processing, based on the spectral characteristics of the color filter 514, processing is performed to make the signal value (R... 922 ), signal value (G) 922 ) and signal value (B 922 The components of non-essential light contained in the WB processing unit are substantially the same. Therefore, the WB processing unit 921 corresponds to the spectral sensitivity correction unit according to the present disclosure. The WB processing corresponds to the spectral sensitivity correction processing according to the present disclosure.

[0113] The correction matrix processing unit 923 corresponds to the signal processing unit according to this disclosure. The correction matrix processing unit 923 is based on the correction coefficients and the signal value (R... 922 G 922 and B 922 Generate the corrected pixel signal.

[0114] In this embodiment, the correction matrix processing unit 923 processes the correction matrix M, which serves as the correction coefficient according to the present disclosure. C With each containing signal value (R) 922 G 922 and B 922 The input matrix M is used as a matrix element. 922 Multiplication is used to perform correction matrix processing to generate the corrected pixel signal.

[0115] Specifically, in the correction matrix processing, the correction matrix processing unit 923 processes the signal value (R) of each pixel using the following formula (1). 922 G 922 and B 922 Corrected to signal value (R) 923 G 923 and B 923 Signal value (R) 923 G 923 and B 923 This corresponds to the signal value of the corrected pixel signal.

[0116]

[0117] Correction matrix M C Details will be provided in the following section on "Correction Matrix M"C The description is in the text.

[0118] The gamma processing unit 924 processes the captured image (each pixel (R) after correction matrix processing) 923 G 923 and B 923 The signal value is processed by gamma correction.

[0119] The YC converter 925 performs YC conversion to convert a gamma-processed captured image (RGB signal) into a luminance signal and a chrominance signal (Y, C). B / C R (Signal). Next, the image processing unit 92 outputs the captured image after YC conversion to the display device 7 via the second transmission cable 8. The display device 7 then displays the captured image.

[0120] Note that the image processing unit 92 is not limited to a configuration where demosaicing is performed between WB processing and correction matrix processing. Demosaicing can be performed at any time, as long as it precedes gamma processing. That is, as image processing unit 92, processing units 921 to 925 are capable of performing demosaicing processing in conjunction with... Figure 2 The different sequences shown are arranged in the order shown.

[0121] The control unit 93 is implemented by a controller, such as a CPU or a microprocessor unit (MPU), executing various programs stored in the storage unit 96. The control unit 93 controls the operation of the light source device 3, the camera 5, and the display device 7, and controls the overall operation of the control device 9. The control unit 93 may include integrated circuits, such as application-specific integrated circuits (ASICs) or FPGAs, and is not limited to CPUs or MPUs.

[0122] The input unit 94 includes operating devices such as a mouse, keyboard, and touch panel, and receives user operations performed by users such as physicians. The input unit 94 then outputs an operation signal corresponding to the user operation to the control unit 93.

[0123] The output unit 95 includes a speaker, a printer, etc., and outputs various types of information.

[0124] The storage unit 96 stores the program executed by the control unit 93, the information required for processing by the control unit 93, etc.

[0125] [Correction Matrix M] C ]

[0126] Next, the correction matrix M will be described. C .

[0127] The correction matrix M should be set based at least on the spectral characteristics of the fluorescence of the observed object.C In this embodiment, the correction matrix M is set based on the spectral characteristics of the fluorescence of the observed object, the spectral characteristics of the unnecessary light, and the spectral characteristics of the color filter 514. C .

[0128] Specifically, the correction matrix M C The matrix elements multiplied by the first pixel signal (signal value) are set to 0 or positive values. In this embodiment, as described above, the first pixel signal (signal value) is the signal value (R). 922 ), which is the signal value of the component with a relatively large amount of fluorescence from the observed object. Therefore, the correction matrix M C Neutral signal value (R) 922 The matrix elements “a”, “d” and “g” (refer to formula (1)) of the multiplication are set to 0 or positive values.

[0129] On the other hand, the correction matrix M C The matrix elements multiplied by the second pixel signal (signal value) are set to 0 or negative values. In this embodiment, as described above, the second pixel signal (signal value) and the signal value (G) are multiplied by the signal value (G). 922 and B 922 Correspondingly, and primarily containing the signal value of non-essential light components, the correction matrix M... C Neutral and signal values ​​(G) 922 and B 922 The matrix elements “b”, “c”, “e”, “f”, “h” and “i” (refer to formula (1)) of the multiplication are set to 0 or negative values.

[0130] Furthermore, it is conceivable to set the correction matrix M using the following first to third setting methods. C The matrix elements in the matrix.

[0131] The first setting method is in the correction matrix M C In each row, the sum of the matrix elements is set to 0, so that (a+b+c=d+e+f=g+h+i=0) is set. In this embodiment, the correction matrix M is set by the first setting method. C The correction matrix can be illustrated by the following formula (2).

[0132]

[0133] The second setting method is to use the correction matrix M C The method of setting the sum of matrix elements in each row to be greater than 0, so that (a+b+c=d+e+f=g+h+i>0).

[0134] The third setting method is to set the calibration matrix M. CThe method of setting the sum of matrix elements in each row to be less than 0, so that (a+b+c=d+e+f=g+h+i<0).

[0135] In the correction matrix M C In this case, the spectral sensitivity ratios of each of the color filter groups 514r, 514g, and 514b can be considered to set the ratios of “a”, “d”, and “g” in the first column, “b”, “e”, and “h” in the second column, and “c”, “f”, and “i” in the third column.

[0136] Furthermore, when it is desired to enhance the observed fluorescence components or to represent the observed fluorescence components as false colors, in the correction matrix M C In addition, it is allowed that “a”, “b” and “c” in the first row, “d”, “e” and “f” in the second row, and “g”, “h” and “i” in the third row be multiplied by different gain factors.

[0137] The implementation method described above achieves the following effects: In the control device 9 according to this embodiment, the correction matrix processing unit 923 is based on the correction coefficient (the correction matrix M in this embodiment). C The first pixel signal in which the fluorescence component of the observed object is dominant (signal value (R) in this embodiment) is the dominant component of the observed object. 922 The second pixel signal, in which non-essential light components dominate (signal value (G) in this embodiment), and the second pixel signal, in which the non-essential light components dominate. 922 and B 922 To generate the corrected pixel signal (the signal value (R) in this embodiment). 923 G 923 and B 923 )).

[0138] Therefore, according to the control device 9 of this embodiment, by subtracting the second pixel signal dominated by the non-essential light component from the first pixel signal dominated by the fluorescence component of the observed object, a corrected pixel signal that suppresses the non-essential light component that becomes noise during fluorescence observation can be generated. That is, fluorescence observation can be performed well.

[0139] Specifically, the correction matrix processing unit 923 processes the correction matrix M, which serves as the correction coefficients according to this disclosure. C Each includes a first pixel signal and a second pixel signal (signal value R in this embodiment) as matrix elements. 922 G 922 and B 922 The input matrix M 922 Multiply to generate the corrected pixel signal (signal value (R) in this embodiment) 923 G923 and B 923 )).

[0140] Therefore, the corrected pixel signal can be easily generated through simple processing.

[0141] Correction matrix M C The matrix element multiplied by the signal value of the first pixel signal is set to 0 or a positive value, and the matrix element multiplied by the signal value of the second pixel signal is set to 0 or a negative value. Furthermore, by using the first to third setting methods to set the matrix elements, the following effects can be obtained.

[0142] The correction matrix M is set using the first setting method. C In the case of matrix elements, a corrected pixel signal with unnecessary light components substantially removed or suppressed can be generated. The correction matrix M is set using a second setting method. C In the case of matrix elements, a corrected pixel signal can be generated that has essentially removed or suppressed only some unnecessary light components. The correction matrix M is set via a third setting method. C In the case of matrix elements, it is possible to generate corrected pixel signals that contain variations and substantially remove or suppress unnecessary light components.

[0143] Additionally, the control device 9 includes a WB processing unit 921, which performs WB processing to make the components of unnecessary light contained in the first pixel signal and the components of unnecessary light contained in the second pixel signal approximately the same. The correction matrix processing unit 923 uses the WB-processed first pixel signal and the second pixel signal (signal values ​​(R) in this embodiment)... 922 G 922 and B 922 This is used to generate the corrected pixel signal.

[0144] Therefore, by subtracting the second pixel signal dominated by the non-essential light component from the first pixel signal dominated by the fluorescence component of the observed object, a corrected pixel signal with the non-essential light component effectively suppressed can be generated.

[0145] (Other implementation methods)

[0146] Although the above is a description of how to implement this disclosure, this disclosure should not be limited to the above-described embodiments.

[0147] In the above embodiment, the light source device 3 emits only excitation light. However, the present invention is not limited to this, and also allows for a configuration that emits white light in addition to excitation light. In this case, the camera 5 generates a captured image (hereinafter referred to as a fluorescence image) obtained by capturing the fluorescence of the observed object OB generated in response to the excitation light, and a captured image (hereinafter referred to as a white light image) obtained by capturing the white light reflected from the observed object OB. In addition, the control device 9 overlays the fluorescence image on the white light image, displays the white light image and the fluorescence image in parallel, displays the white light image and the fluorescence image in a picture-in-picture (PinP) manner, or displays the white light image and the fluorescence image on different monitors.

[0148] Even in this configuration, effects similar to those of the embodiments described above can be achieved.

[0149] In the above embodiments, the following first to tenth modifications may be adopted.

[0150] (First variation)

[0151] The first variation will describe the case where ALM-488 is used as a reagent.

[0152] Figures 5 to 8 This is a diagram illustrating a first variation of the embodiment. Specifically, Figure 5 This is a diagram showing the absorption and emission spectra of ALM-488. Figure 5 In the diagram, the horizontal axis represents wavelength [nm], and the vertical axis represents the absorbance of the absorption spectrum and the fluorescence emission intensity of the emission spectrum. Figure 5 In the diagram, the absorption spectrum of ALM-488 is represented by a dashed line, and the emission spectrum of ALM-488 is represented by a solid line. Figure 6 It corresponds to Figure 4 The illustration. Figure 7 The captured image F1 is shown. This captured image was obtained by capturing the area of ​​the reagent DR, which is the subject, with the test tube TU containing reagent DR containing ALM-488 placed in front of a light-shielding curtain and the reagent DR illuminated with excitation light. Figure 8 (a) is shown Figure 7 The diagram shows the frequency distribution of r, g, and b values ​​in the first region Ar1 of the captured image F1. Figure 8 (b) is shown Figure 7 A diagram illustrating the frequency distribution of r, g, and b values ​​in the second region Ar2 of the captured image F1. Figure 8 In the diagram, the horizontal axis represents the r, g, and b values, and the vertical axis represents the number of pixels (frequency) with each r, g, and b value.

[0153] like Figure 5 As shown, the peak wavelength of the absorption spectrum of ALM-488 is approximately 480 nm. Therefore, in order to increase the intensity of the fluorescence of the observed object emitted from ALM-488, the excitation light emitted from the light source device 3 is preferably excitation light in a wavelength band including approximately 480 nm.

[0154] In addition, such as Figure 5 As shown, the peak wavelength of the emission spectrum of ALM-488 is approximately 530 nm. Therefore, the fluorescence of the observed object emitted from ALM-488 has spectral characteristics with a center wavelength of approximately 530 nm, as... Figure 6 The spectrum of SPF2 is shown. Based on the spectral characteristics of the G filter group 514g and the spectral characteristics of the observed object's fluorescence and unnecessary light, the g value includes components of the observed object's fluorescence and unnecessary light. Similarly, based on the spectral characteristics of the R filter group 514r and the spectral characteristics of the observed object's fluorescence and unnecessary light, the r value mainly includes components of unnecessary light. Likewise, based on the spectral characteristics of the B filter group 514b and the spectral characteristics of the observed object's fluorescence and unnecessary light, the b value also mainly includes components of unnecessary light in the same manner. In other words, compared to the r and b values, the g value contains a greater amount of the observed object's fluorescence component.

[0155] The following experiments also revealed that "when using ALM-488, the g value contains a greater amount of the observed fluorescence component compared to the r and b values."

[0156] In this experiment, a test tube TU containing reagent DR (ALM-488) was placed in front of a light-shielding curtain and irradiated with excitation light. Under these conditions, imaging unit 51 captured the region including reagent DR as the subject, and the frequency distributions of r, g, and b values ​​in the first region Ar1 and the second region Ar2 of the captured image F1 generated by image capture were compared with each other. Figure 8 Here, as Figure 7 As shown, the first region Ar1 is the area where reagent DR appears in the captured image F1. Conversely, the second region Ar2 is the background (light-blocking curtain) area in the captured image F1 excluding reagent DR. That is, the first region Ar1 is brighter than the second region Ar2 because the observed fluorescence is emitted from reagent DR by the illumination of excitation light.

[0157] exist Figure 8 In the figures, the reference numeral "FR" indicates the frequency distribution of the r value. The reference numeral "FG" indicates the frequency distribution of the g value. The reference numeral "FB" indicates the frequency distribution of the b value.

[0158] Therefore, as Figure 8 As shown in (a), the frequency distribution FG of the g value appears in the bright part ( Figure 8 The part with larger horizontal axis values), and the frequency distributions FR and FB of r and b values ​​appear in the darker part than the frequency distribution FG. Figure 8 (The portion of the horizontal axis with smaller values). This has revealed that "when using ALM-488, the g value contains a larger component of the observed object's fluorescence compared to the r and b values." Note that in Figure 8 The frequency distributions of r and b values ​​in the first region Ar1 shown in (a) are FR and FB, and in Figure 8 The frequency distributions of r, g, and b values ​​in the second region Ar2 shown in (b) FR, FG, and FB mainly contain components of non-essential light.

[0159] As mentioned above, compared to the r and b values, the g value contains a larger component of the observed object's fluorescence, and therefore, for the same signal value (R... 922 G 922 and B 922 In ), with (R) 922 and B 922 Compared to the signal value (G), 922 It contains a larger proportion of the fluorescence component of the observed object. Therefore, the signal value (G) 922 ) corresponds to the first pixel signal (signal value) according to this disclosure. Furthermore, the signal value (R) 922 and B 922 This corresponds to the second pixel signal (signal value) according to this disclosure. Signal value (G) 922 The signal value (R0) is the signal value in which the fluorescence component of the observed object is greater than the non-essential light component and the fluorescence component of the observed object is dominant. On the other hand, the signal value (R0) 922 and B 922 () is a signal value where the non-essential light component is dominant.

[0160] Furthermore, when using ALM-488, similar to the implementation described above, the correction matrix M... C The matrix element multiplied by the first pixel signal (signal value) is set to 0 or a positive value. Specifically, when using ALM-488, as described above, the first pixel signal (signal value) is the signal value (G) of the component with a relatively larger amount of fluorescence from the observed object. 922 Therefore, the correction matrix M C Neutral and signal values ​​(G) 922 The matrix elements “b”, “e” and “h” (refer to formula (1)) of the multiplication are set to 0 or positive values.

[0161] Furthermore, when using ALM-488, similar to the implementation described above, the correction matrix M... CThe matrix elements multiplied by the second pixel signal (signal value) are set to 0 or negative values. Specifically, when using ALM-488, as described above, the second pixel signal (signal value) is a signal value (R) that mainly contains components of unnecessary light. 922 and B 922 Therefore, the correction matrix M C Neutral signal value (R) 922 and B 922 The matrix elements “a”, “d”, “g”, “c”, “f” and “i” (refer to formula (1)) of the multiplication are set to 0 or negative values.

[0162] When using ALM-488, the correction matrix M can be set using a first to a third setting method similar to the embodiments described above. C When using ALM-488, the correction matrix M is set using the first setting method. C The correction matrix can be illustrated by the following formula (3).

[0163]

[0164] Even when ALM-488 is used as a reagent in the first variation described above, similar effects to those of the embodiments described above can be obtained.

[0165] (Second variation)

[0166] The second variation will describe the case where LUM-015 is used as a reagent.

[0167] Figure 9 and Figure 10 This is a diagram illustrating a second variation of the embodiment. Specifically, Figure 9 This is a diagram showing the absorption and emission spectra of LUM-015. Figure 9 In the diagram, the horizontal axis represents wavelength [nm], and the vertical axis represents the absorbance of the absorption spectrum and the fluorescence emission intensity of the emission spectrum. Figure 9 In the diagram, the absorption spectrum of LUM-015 is represented by a dashed line, and the emission spectrum of LUM-015 is represented by a solid line. Figure 10 Is with Figure 4 Corresponding illustrations.

[0168] like Figure 9 As shown, the peak wavelength of the absorption spectrum of LUM-015 is approximately 650 nm. Therefore, in order to increase the intensity of the fluorescence of the observed object emitted from LUM-015, the excitation light emitted from the light source device 3 is preferably excitation light in the wavelength range of approximately 650 nm.

[0169] like Figure 9As shown, the peak wavelength of the emission spectrum of LUM-015 is approximately 670 nm. Therefore, the fluorescence of the observed object emitted from LUM-015 has spectral characteristics in the band with a center wavelength of approximately 670 nm, as... Figure 10 The spectrum of SPF3 is shown. Based on the spectral characteristics of the R filter group 514r and the spectral characteristics of the observed object's fluorescence and unnecessary light, the r value contains components of the observed object's fluorescence and unnecessary light. Similarly, based on the spectral characteristics of the G filter group 514g and the spectral characteristics of the observed object's fluorescence and unnecessary light, the g value primarily contains components of unnecessary light. Likewise, based on the spectral characteristics of the B filter group 514b and the spectral characteristics of the observed object's fluorescence and unnecessary light, the b value also primarily contains components of unnecessary light in the same manner. In other words, compared to the g and b values, the r value contains a greater amount of the observed object's fluorescence.

[0170] As mentioned above, compared to the g and b values, the r value contains a larger component of the observed object's fluorescence, and therefore, also in the signal value (Rg). 922 G 922 and B 922 In ), with the signal value (G) 922 and B 922 Compared to the signal value (R), 922 It contains a larger component of the fluorescence of the observed object. Therefore, the signal value (R) 922 ) corresponds to the first pixel signal (signal value) according to this disclosure. Furthermore, the signal value (G) 922 and B 922 This corresponds to the second pixel signal (signal value) according to this disclosure. Signal value (R) 922 The signal value (G) is the signal value in which the fluorescence component of the observed object is greater than the non-essential light component and the fluorescence component of the observed object is dominant. On the other hand, the signal value (G) 922 and B 922 () is a signal value where the non-essential light component is dominant.

[0171] In other words, similar to the above implementation, when using LUM-015, the signal value (R) 922 ) corresponds to the first pixel signal (signal value) according to this disclosure, the signal value (G) 922 and B 922 This corresponds to the second pixel signal (signal value) according to this disclosure. Therefore, when using LUM-015, the correction matrix described in the above embodiments can be used as the correction matrix M. C .

[0172] Even when LUM-015 is used as a reagent in the second variation described above, similar effects to those of the embodiments described above can be obtained.

[0173] (Third variation)

[0174] The third variation will describe the case where multiple reagents are used.

[0175] Figure 11 This is a diagram illustrating a third variation of the embodiment. Specifically, Figure 11 It corresponds to Figure 1 The illustration. Figure 11 A medical observation system 1 is shown in the case where two reagents are to be used. In the following text, for ease of description, the number of reagents to be used is two.

[0176] When two reagents are to be used, such as Figure 11 As shown, the medical observation system 1 needs to include a first light source 31 that emits excitation light for exciting the first of two reagents, and a second light source 32 that emits excitation light for exciting the second reagent. When the number of reagents to be used is three or more, three or more light sources can be set up according to the number of reagents.

[0177] The medical observation system 1 activates the first light source 31 and the second light source 32 in a time-division manner for the observation subject OB who has received the first and second reagents. When performing correction matrix processing on the captured image generated by the image sensor 513 at the timing of the activation of the first light source 31, the medical observation system 1 uses the correction matrix M corresponding to the first reagent. C On the other hand, when the medical observation system 1 performs correction matrix processing on the captured image generated by the image sensor 513 at the lighting timing of the second light source 32, it uses the correction matrix M corresponding to the second reagent. C .

[0178] Even when multiple reagents are used as in the third variation described above, effects similar to those of the embodiments described above can be obtained.

[0179] (Fourth variation)

[0180] Figures 12 to 14 This is a diagram illustrating a fourth variation of the embodiment. Specifically, Figure 12 Is with Figure 4 Corresponding illustrations. Figure 13 This shows the calculation of the correction matrix M. C The flowchart of the method. Figure 14 It is shown in Figure 13 The calculation correction matrix M is shown in the figure. C The method uses a white chart (WC) diagram.

[0181] Although the above implementation method uses 3 A matrix of 3 is used as the correction matrix M C This is an example case, but the matrix is ​​not limited to this. For example, other matrices can also be used, such as the 3 matrix represented by equation (4) below. 8 correction matrix M C 3, represented by the following formula (5) The correction matrix M of 7 C 3, represented by the following formula (6) The correction matrix M of 6 C 3, represented by the following formula (7) The correction matrix M of 5 C , or 3 as expressed by the following formula (8) 4. Correction matrix M C .

[0182]

[0183] The input matrix M shown in formula (4) 922 Including signal value (R) 922 G 922 and B 922 ) as matrix elements, and also includes using the signal value (R) as a matrix element. 922 ) and signal value (G 922 The "R" obtained by multiplying by ) 922 G 922 ", by using the signal value (G) 922 ) and signal value (B 922 The "G" obtained by multiplying by ) 922 B 922 ", by the signal value (R 922 ) and signal value (B 922 The "R" obtained by multiplying by ) 922 B 922 ", by the signal value (R 922 ), signal value (G) 922 ) and signal value (B 922 The "R" obtained by multiplying by ) 922 G 922 B 922 " and "1" are used as matrix elements.

[0184] The input matrix M shown in formula (5)922 By omitting the input matrix M shown in formula (4) 922 In the matrix element "R 922 G 922 B 922 "To obtain."

[0185] Although formula (6) represents the three input matrices M 922 However, three input matrices M can be used. 922 Any one of them. By omitting the input matrix M shown in formula (5) 922 The "R" in 922 G 922 “G” 922 B 922 "and "R 922 B 922 Use any matrix element from the given set to obtain the three input matrices M. 922 .

[0186] Although formula (7) represents the three input matrices M 922 However, three input matrices M can be used. 922 Any one of them. By omitting the input matrix M shown in formula (5) 922 The "R" in 922 G 922 “G” 922 B 922 "and "R 922 B 922 Use any two matrix elements from the given set to obtain the three input matrices M. 922 .

[0187] The input matrix M shown in formula (8) 922 This is achieved by adding the matrix element "1" to the input matrix M shown in formula (1). 922 It was obtained through this process.

[0188] Meanwhile, the input matrix M 922 Preferably, the matrix elements include signal values ​​dominated by the fluorescence component of the observed object and signal values ​​dominated by the non-essential light component. For example, such as... Figure 12 As shown in the SPF4 spectrum, when the fluorescence spectrum of the observed object has spectral characteristics with a center wavelength of approximately 580 nm, the signal value (R) is... 922 ) and signal value (G 922(B) is the signal value where the fluorescence component of the observed object is dominant. On the other hand, the signal value (B) 922 The signal value (R) is dominated by non-essential light components. In this case, the signal value (R) 922 ) and signal value (G 922 ) corresponds to the first pixel signal (signal value) according to this disclosure. Furthermore, the signal value (B) 922 ) corresponds to the second pixel signal (signal value) according to this disclosure. In this case, by using the signal value (R) 922 ) and signal value (G 922 The product of these two products is called "R". 922 G 922 "R" represents the dominant component of the fluorescence observed in the object. 922 G 922 "Included in the input matrix M" 922 In the matrix elements (e.g., formulas (4) to (7)), a corrected pixel signal that effectively suppresses unnecessary light components can be generated.

[0189] In the fourth variation, such as Figure 13 The correction matrix M is calculated as shown in the flowchart. C .

[0190] First, the operator places the distal end of insertion unit 2 against the whiteboard WC ( Figure 14 Approaching the white plate WC, with the first light emitted from the light source device 3 illuminating the white plate WC, the imaging unit 51 captures the white plate WC (step S1). Here, the white plate WC is a white plate with a light reflectivity of approximately 1. That is, the first light (excitation light) reflected by the white plate WC and the unnecessary light generated in the observation light path P0 are incident on the imaging unit 51. Among them, the first light (excitation light) is partially, substantially, or completely suppressed by the excitation light cutoff color filter 511 in the imaging unit 51. That is, only the unnecessary light passes through the color filter 514 and is incident on the image sensor 513.

[0191] After step S1, the operator uses image processing unit 92 to perform WB processing (step S2) and demosaic processing (step S3) on the captured image generated by imaging unit 51, thereby obtaining the signal value (R) of each pixel in the captured image. 922 G 922 B 922 Subsequently, regarding the signal value (R) 922 ), signal value (G) 922 ) and signal value (B 922 For each of the areas in Ar (where the operator is located), the operator is responsible for the area containing the whiteboard WC at its center. Figure 14 The signal value (R) of each pixel in ) 922 G922 and B 922 The average signal value (R) is obtained by averaging the signals. 922_AVE G 922_AVE and B 922_AVE (Step S4).

[0192] After step S4, the operator selects the input matrix M. 922 The type (step S5).

[0193] Specifically, in calculating 3 3 Correction matrix M C At that time, the operator selects the input matrix M of formula (1). 922 In calculating 3 8 correction matrix M C At that time, the operator selects the input matrix M of formula (4). 922 In calculating 3 The correction matrix M of 7 C At that time, the operator selects the input matrix M of formula (5). 922 When calculating 3 The correction matrix M of 6 C At that time, the operator selects one of the three input matrices M from formula (6). 922 One of them. When calculating 3 The correction matrix M of 5 C At that time, the operator selects one of the three input matrices M from formula (7). 922 One of them. In calculating 3 4. Correction matrix M C At that time, the operator selects the input matrix M of formula (8). 922 .

[0194] After step S5, the operator uses the average signal value (R) obtained in step S4. 922_AVE G 922_AVE and B 922_AVE ) Calculate the input matrix M selected in step S5 922 (Step S6).

[0195] After step S6, the operator will use the input matrix M calculated in step S6. 922 Input into the following equation (9) (step S7). That is, step S7 generates equation (9), where the correction matrix M C For variables.

[0196]

[0197] After step S7, the operator uses the least squares error method to calculate the correction matrix M based on equation (9) of the variables generated in step S7. CWhere the correction matrix M C The variable is (step S8). The correction matrix M in step S8 C The calculation method is not limited to the least squares error method; other methods can be used.

[0198] As described above, only non-essential light passes through color filter 514 and enters image sensor 513. Therefore, the input matrix M... 922 The matrix elements in the formula contain only the components of non-essential light. By setting the left side of formula (9) to "0", step S8 calculates the correction matrix M used to remove only the components of non-essential light. C .

[0199] In addition to achieving the same effects as the above-described embodiments, the fourth variation also has the following effects.

[0200] In the fourth variation, besides 3 3 Correction matrix M C In addition, 3 can also be used 8.3 7.3 6, 3 5 and 3 4. Correction matrix M C That is, by increasing the number of correction parameters, it becomes possible to generate corrected pixel signals that effectively suppress unnecessary light components. Furthermore, this is achieved by including the values ​​multiplied by each of the at least two first pixel signals in the input matrix M. 922 The matrix elements can generate corrected pixel signals that effectively suppress unnecessary light components.

[0201] In the above embodiments and the fourth variation, the second pixel signal according to this disclosure has two signal values ​​(G). 922 ) and (B 922 ), and therefore adopted 3 The correction matrix M of n (n is 3 or greater) C However, when the number of second pixel signals is only one, it is also permissible to use 2. The correction matrix M (where m is 2 or greater than 2) C .

[0202] (Fifth variation)

[0203] Figure 15 and Figure 16 The diagrams shown all illustrate the fifth variation of the embodiment. Specifically, Figure 15 This is a diagram showing the configuration of the prism 516 and image sensors 513r, 513g, and 513b according to the fifth modification. Figure 15In the diagram, light in the red band dispersed by the spectrum of prism 516 is represented as "R", light in the green band dispersed by the spectrum of prism 516 is represented as "G", and light in the blue band dispersed by the spectrum of prism 516 is represented as "B". Figure 16 This is a diagram showing the spectral characteristics of prism 516. The spectral characteristics of prisms such as prism 516 are also called wavelength spectral characteristics. Figure 16 In the graph, the horizontal axis represents wavelength [nm], and the vertical axis represents relative response [%]. Additionally, in... Figure 16 In the diagram, the spectrum of light in the red band dispersed by the spectrum of prism 516 is represented by curve CR, the spectrum of light in the green band dispersed by the spectrum of prism 516 is represented by curve CG, and the spectrum of light in the blue band dispersed by the spectrum of prism 516 is represented by curve CB. Note that, similarly in the fifth variation, as... Figure 16 As shown in the spectrum SPF1, the spectral characteristics of the observed fluorescence are the same as those in the above embodiment, but the spectral characteristics can be other spectral characteristics. Similarly, the wavelength range SPU capable of generating unnecessary light can be a different wavelength range than that in the above embodiment.

[0204] In the above embodiments, it is possible to adopt according to Figure 15 The fifth modified example shown in the figure uses prism 516 and image sensors 513r, 513g and 513b to replace image sensor 513 and color filter 514.

[0205] Prism 516 corresponds to the spectral dispersion component according to this disclosure. Prism 516 disperses the spectrum of the second light (subject image) that has passed through the excitation light cutoff filter 511 and been converged by the lens unit 512 into red, green, and blue band R, G, and B beams.

[0206] Image sensor 513r has a configuration similar to that of image sensor 513 and captures light R in the red band dispersed by the spectrum of prism 516.

[0207] Image sensor 513g has a similar configuration to image sensor 513 and captures light G in the green band dispersed by the spectrum of prism 516.

[0208] Image sensor 513b has a similar configuration to image sensor 513 and captures light B in the blue band dispersed by the spectrum of prism 516.

[0209] Based on the spectral characteristics (curve CR) of prism 516 and the spectral characteristics of fluorescence and unnecessary light of the observed object, the signal value (r value) of the pixel signal output from image sensor 513r contains components of fluorescence and unnecessary light of the observed object. Furthermore, based on the spectral characteristics (curve CG) of prism 516 and the spectral characteristics of fluorescence and unnecessary light of the observed object, the signal value (g value) of the pixel signal output from image sensor 513g mainly contains components of unnecessary light. Moreover, based on the spectral characteristics (curve CB) of prism 516 and the spectral characteristics of fluorescence and unnecessary light of the observed object, the signal value (b value) of the pixel signal output from image sensor 513b mainly contains components of unnecessary light. That is, compared to the g value and b value, the r value contains a greater amount of fluorescence component of the observed object. Based on the above, the pixel signal output from image sensor 513r corresponds to the first pixel signal according to this disclosure. Furthermore, the pixel signals output from image sensors 513g and 513b correspond to the second pixel signal according to this disclosure.

[0210] According to the fifth variation, the image processing unit 92 does not need to include the demosaicing processing unit 922 described in the above embodiments. Similar to the above embodiments, the correction matrix processing unit 923 according to the fifth variation uses the signal value (R) of each pixel signal from the corresponding pixels of the image sensors 513r, 513g, and 513b. 922 G 922 and B 922 The signal values ​​(r, g, b values) obtained from WB processing are then subjected to correction matrix processing to generate the corrected pixel signal. The correction matrix M used in the correction matrix processing... C The settings are based on the spectral characteristics of the fluorescence of the observed object, the spectral characteristics of the unnecessary light, and the spectral characteristics of the prism 516.

[0211] Even with the configuration of the fifth variation described above, the same effect as the above-described implementation can be obtained.

[0212] Alternatively, in the fifth variation, the configuration of the fourth variation described above can also be used.

[0213] (Sixth variation)

[0214] Figure 17 and Figure 18 The diagrams shown all illustrate a sixth variation of the implementation method. Specifically, Figure 17 This is a perspective view showing the image sensor 513 according to the sixth modified example. Figure 18 This is a side view showing the image sensor 513 according to the sixth variant. Figure 18 In this design, light in the red band is represented as "R", light in the green band as "G", and light in the blue band as "B".

[0215] In the above embodiments, it is possible to adopt according to Figure 17 and Figure 18 The sixth variant shown replaces the image sensor 513 and color filter 514 with an image sensor 513.

[0216] According to the sixth variation, the image sensor 513 is a Foveon-type image sensor (e.g., organic CMOS). Specifically, as Figure 17 and Figure 18 As shown, the image sensor 513 has a configuration of stacked first light-receiving layer 5131b, first light-receiving layer 5131g, and third light-receiving layer 5131r.

[0217] The first light-receiving layer 5131b receives only blue-band light and converts it into an electrical signal. The first light-receiving layer 5131b is disposed on the top layer and receives blue-band light B contained in the second light (subject image) that is focused by the lens unit 512 after passing through the excitation light cutoff filter 511, and converts light B into an electrical signal.

[0218] The second light-receiving layer 5131g receives only green band light and converts the light into an electrical signal. The second light-receiving layer 5131g is disposed below the first light-receiving layer 5131b, receives green band light G contained in the light transmitted through the first light-receiving layer 5131b, and converts light G into an electrical signal.

[0219] The third light-receiving layer 5131r is disposed at the bottom layer, receiving the red band light R transmitted through the first light-receiving layer 5131b and the second light-receiving layer 5131g, and converting the light R into an electrical signal.

[0220] For example, if the spectral characteristics of the fluorescence of the observed object are those of a band included in the red band of light R, the electrical signal (pixel signal (r value)) output from the third light-receiving layer 5131r mainly contains the component of the fluorescence of the observed object, and therefore corresponds to the first pixel signal according to this disclosure. On the other hand, the electrical signals (pixel signals (g value and b value)) output from the first light-receiving layer 5131b and the second light-receiving layer 5131g mainly contain the component of non-essential light, and therefore correspond to the second pixel signal according to this disclosure.

[0221] According to the sixth variation, the image processing unit 92 need not include the demosaicing processing unit 922 described in the above embodiments. Similar to the above embodiments, the correction matrix processing unit 923 according to the sixth variation uses the signal value (R) of each pixel signal from the corresponding pixels of the first light-receiving layer to the third light-receiving layers 5131b, 5131g, and 5131r. 922 G 922 and B922 The signal values ​​(r value, g value, b value) obtained by performing WB processing are then subjected to correction matrix processing to generate corrected pixel signals. According to the sixth variation, the correction matrix M used in the correction matrix processing is set based on the spectral characteristics of the fluorescence of the observed object, the spectral characteristics of unnecessary light, and the spectral characteristics of the image sensor 513. C The spectral characteristics of a Foveon-type image sensor, such as the image sensor 513 according to the sixth modification, are also called light-receiving spectral characteristics.

[0222] Even with the structure of the sixth variation described above, the same effect as the above-described implementation can be obtained.

[0223] Alternatively, in the sixth variation, the configuration of the fourth variation described above can also be used.

[0224] (Seventh variation)

[0225] The seventh modified medical observation system is a medical observation system that uses a device called a video endoscope (flexible endoscope) that includes an imaging unit on the distal side of the insertion unit. Hereinafter, for ease of explanation, the medical observation system 1 of the seventh modified example will be referred to as medical observation system 1A.

[0226] Figure 19 This is a diagram illustrating the seventh variation of the embodiment.

[0227] like Figure 19 As shown, the medical observation system 1A includes: an endoscope 100 that captures in vivo images of the observation area by inserting an insertion unit 2A into a living body and outputs the captured images; a light source device 3 that emits a first light from the distal end of the endoscope 100; a control device 9 that processes the captured images output from the endoscope 100; and a display device 7 that is connected to the control device 9 via a second transmission cable 8 and is configured to display images based on video signals processed by the control device 9.

[0228] like Figure 19 As shown, the endoscope 100 includes: an insertion unit 2A, which is a flexible elongated portion; an operation unit 101 connected to the proximal end of the insertion unit 2A and receiving various operation signals; and a universal cable 102 extending from the operation unit 101 in a direction different from the extension direction of the insertion unit 2A, and having built-in various cables for connecting to the light source device 3 and the control device 9.

[0229] like Figure 19As shown, the insertion unit 2A includes: a distal end portion 22; a curved portion 23, which is a curved portion connected to the proximal side of the distal end portion 22 and has a plurality of curved blocks formed thereon; and a flexible tube portion 24, which is a flexible elongated portion connected to the proximal side of the curved portion 23.

[0230] Although specific details are omitted, the distal end 22 includes a configuration substantially the same as that of the camera 5 described in the above embodiments. The captured image captured by the distal end 22 (imaging unit) is output to the control device 9 via the operation unit 101 and the universal cable 102.

[0231] Even with the configuration of the seventh variation described above, the same effect as the above-described implementation can be obtained.

[0232] (Eighth variation)

[0233] The medical observation system of Modification 8 is a medical observation system using a surgical microscope, which magnifies and captures a predetermined field of view inside the subject (within the biological body) or on the surface of the subject (on the surface of the biological body). Hereinafter, for ease of explanation, the medical observation system 1 according to the seventh modification will be referred to as medical observation system 1B.

[0234] Figure 20 This is a diagram illustrating the eighth variation of the embodiment.

[0235] like Figure 20 As shown, the medical observation system 1B includes: a surgical microscope 12 that captures images for observing a subject and outputs the captured images; a control device 9 that processes the captured images output from the surgical microscope 12; and a display device 7 that is connected to the control device 9 via a second transmission cable 8 and displays images based on video signals processed by the control device 9.

[0236] like Figure 20 As shown, the surgical microscope 12 includes: a microscope unit 121 that captures magnified images of minute parts of a subject and outputs the captured images; a support 122 that is connected to the proximal end of the microscope unit 121 and includes an arm that pivotally supports the microscope unit 121; and a base 123 that pivotally holds the proximal end of the support 122 and is movable on the ground.

[0237] like Figure 20 As shown, the control device 9 is mounted in the base 123. Furthermore, although specific details are omitted, the light source device 3 for generating illumination light to irradiate the subject from the surgical microscope 12 is also mounted on the base 123.

[0238] The base 123 can also be fixed to the ceiling, wall, etc. to support the support part 122, instead of being movably set on the ground.

[0239] Although specific details are omitted, the microscope unit 121 has a configuration substantially similar to that of the camera 5 described in the above embodiments. The captured image, which has been imaged on the microscope unit 121 (imaging unit), is output to the control device 9 via a first transmission cable 6 wired along the support portion 122.

[0240] Even with the configuration of the eighth variation described above, the same effect as the above-described implementation can be obtained.

[0241] (Ninth variation)

[0242] Figure 21 and Figure 22 The diagrams shown all illustrate the ninth variation of the embodiment. Specifically, Figure 21 This is a side view of the ring lamp 15. The ring lamp 15 corresponds to an open-field observation device for observing biological tissues. This open-field observation device is used in laparotomy. Figure 22 From the front ( Figure 21 (Observe the diagram of ring light 15 on the left side).

[0243] In the ninth variation, in addition to the insertion unit 2 in the above embodiments, Figure 21 and Figure 22 The ring light 15 shown is detachably connected to the camera 5. That is, as... Figure 21 As shown, depending on the user's usage status, the insertion unit 2 can be connected to the camera 5 or the ring light 15 can be connected to the camera 5.

[0244] Unlike insertion unit 2, which is inserted into the object OB, ring light 15 provides first light to the surgical site and captures second light (the image of the subject) as a return light from the first light from the surgical site. Figure 21 and Figure 22 As shown, the ring light 15 includes an illumination unit 151 and a subject image capture unit 152 for capturing subject images.

[0245] like Figure 21 and Figure 22 As shown, the lighting unit 151 includes a housing 1511 and a plurality of lighting lenses 1512.

[0246] The housing 1511 has an annular shape centered on the optical axis Ax. The other end of the light guide 4 is detachably connected to the housing 1511.

[0247] like Figure 22As shown, a plurality of illumination lenses 1512 are arranged at predetermined intervals on the front end face of the housing 1511 in the circumferential direction centered on the optical axis Ax. Then, the plurality of illumination lenses 1512 illuminate the surgical site with first light provided from the light source device 3 and guided into the housing 1511 via the light guide 4.

[0248] The subject image acquisition unit 152 extends along the optical axis Ax. The subject image acquisition unit 152 internally includes an optical system formed with one or more lenses and configured to converge second light (subject image) emitted from a plurality of illumination lenses 1512 and passing through the surgical site. Furthermore, on the proximal side of the subject image acquisition unit 152 ( Figure 21 A connecting unit 1521 is provided at the right end of the insertion unit 2. The connecting unit 1521 is designed (shaped) to be compatible with the eyepiece 21 in the insertion unit 2 and is detachably connected to the camera 5.

[0249] Even with the configuration of the ninth variation described above, the same effect as the above-described implementation can be obtained.

[0250] (Tenth variation)

[0251] In the above embodiments, the fluorescence of the observed object is visible light, but the fluorescence of the observed object is not limited to this and can also be invisible light.

[0252] At this time, as an image sensor according to the tenth variation, such as the image sensor 513 in the above embodiment, as long as the image sensor has the sensitivity to image not only visible light but also invisible light, and can also image the fluorescence of the observed object in invisible light, the number of image sensors can be one.

[0253] Furthermore, as an image sensor according to the tenth variation, it is also permissible to employ an image sensor that captures only invisible light (such as an infrared image sensor). In this case, as long as the image sensor has sensitivity for each of the multiple bands and is each capable of outputting a first pixel signal primarily containing the component of fluorescence of the observed object in invisible light and a second pixel signal primarily containing the component of unnecessary light in invisible light, a configuration including an image sensor that captures only invisible light is also permissible. Furthermore, it is also permissible to have a configuration in which two image sensors are provided: an image sensor that captures invisible light and an image sensor that captures visible light, wherein the image sensor that captures invisible light captures the fluorescence of the observed object in invisible light, and the image sensor that captures visible light captures unnecessary light in visible light.

[0254] Even with the configuration of the tenth variation described above, the same effect as the above-described implementation can be obtained.

[0255] The following configurations also fall within the technical scope of this disclosure.

[0256] (1) A medical image processing apparatus, comprising an image processing unit configured to process pixel signals acquired from pixels of an image sensor, wherein the pixel signals comprise: a first pixel signal containing a component of non-essential light and a component of fluorescence of an object being observed, the non-essential light comprising spontaneous fluorescence generated from a component of an observation optical path constituting the light as light emitted from a light source device propagates through an observation optical path, the component of fluorescence of the object being observed being emitted from a substance contained in the object being observed and excited by light; and a second pixel signal containing at least the component of non-essential light, and the image processing unit comprising a signal correction unit configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal and the second pixel signal, the correction coefficient being set based on the spectral characteristics of fluorescence of the object being observed.

[0257] (2) The medical image processing apparatus according to (1), wherein the amount of the fluorescent component of the observed object contained in the first pixel signal is greater than the amount of the fluorescent component of the observed object contained in the second pixel signal.

[0258] (3) The medical image processing apparatus according to (1) or (2), wherein the signal correction unit is configured to generate a corrected pixel signal by multiplying a correction matrix, which is a correction coefficient, with an input matrix, the input matrix including the signal values ​​of the first pixel signal and the signal values ​​of the second pixel signal as matrix elements.

[0259] (4) The medical image processing apparatus according to (3), wherein the first pixel signal is provided as at least two first pixel signals, and the input matrix includes values ​​obtained by multiplying the signal values ​​of at least two first pixel signals as matrix elements.

[0260] (5) The medical image processing apparatus according to (3) or (4), wherein the matrix element in the correction matrix multiplied with the signal value of the first pixel signal is set to 0 or a positive value, and the matrix element in the correction matrix multiplied with the signal value of the second pixel signal is set to 0 or a negative value.

[0261] (6) The medical image processing apparatus according to (5), wherein in each row of the correction matrix, each number obtained by adding the matrix elements is 0.

[0262] (7) The medical image processing apparatus according to (5), wherein in each row of the correction matrix, each number obtained by adding the matrix elements is greater than 0.

[0263] (8) The medical image processing apparatus according to (5), wherein in each row of the correction matrix, each number obtained by adding the matrix elements is less than 0.

[0264] (9) A medical image processing apparatus according to any one of (1) to (8), wherein the image sensor is configured to capture each beam of light obtained by spectrally dispersing light from the object of observation through a spectral dispersion component, and the correction coefficient is set based on the spectral characteristics of the fluorescence of the object of observation and the spectral characteristics of the spectral dispersion component.

[0265] (10) A medical image processing apparatus according to any one of (1) to (9), wherein the correction coefficient is set based on the spectral characteristics of fluorescence of the observed object and the spectral characteristics of unnecessary light.

[0266] (11) The medical image processing apparatus according to (9), wherein the image processing unit further comprises a spectral sensitivity correction unit configured to perform spectral sensitivity correction processing on a first pixel signal and a second pixel signal based on the spectral characteristics of the spectral dispersion component, and the signal correction unit is configured to generate a corrected pixel signal by using the first pixel signal and the second pixel signal after the spectral sensitivity correction processing performed by the spectral sensitivity correction unit.

[0267] (12) A medical image processing apparatus according to any one of (1) to (11), wherein the first pixel signal and the second pixel signal are acquired from the same image sensor.

[0268] (13) A medical image processing apparatus according to any one of (1) to (11), wherein a plurality of image sensors are provided, and a first pixel signal and a second pixel signal are acquired from different image sensors.

[0269] (14) A medical image processing apparatus according to any one of (1) to (13), wherein the fluorescence of the observed object is generated by a reagent or fluorescent dye applied to the observed object, or by a fluorescent substance derived from the observed object.

[0270] (15) A medical image processing apparatus according to any one of (1) to (14), wherein the component constituting the observation optical path and configured to generate autofluorescence is an optical fiber, a lens, glass, an adhesive material or a color filter.

[0271] (16) A medical image processing apparatus according to any one of (1) to (15), wherein the corrected pixel signal is a signal whose signal value based on unnecessary light has been reduced.

[0272] (17) A medical observation system, comprising: a light source device including a first light source configured to emit first light comprising narrowband light that excites a substance contained in an object of observation; an imaging device including an image sensor configured to capture second light that is a return light from the first light from the object of observation; and a medical image processing device including an image processing unit configured to process pixel signals corresponding to pixels of the image sensor, wherein the pixel signals include: a first pixel signal comprising a component of non-essential light and a component of fluorescence of the object of observation, the non-essential light comprising spontaneous fluorescence generated from a component constituting the observation optical path as the first light emitted from the light source device propagates through the observation optical path, the fluorescence of the object of observation being emitted from a substance contained in the object of observation excited by the first light; and a second pixel signal comprising at least a component of non-essential light, and the image processing unit including a signal correction unit configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal and the second pixel signal, the correction coefficient being set based on the spectral characteristics of the fluorescence of the object of observation.

[0273] (18) The medical observation system according to (17), wherein the first light source is set as one or more light sources.

[0274] (19) The medical observation system according to (17) or (18), wherein the first light source is set as an LED or a semiconductor laser.

[0275] (20) A medical observation system according to any one of (17) to (19), wherein the first light is set to broadband light or narrowband light.

[0276] Reference number list

[0277] 1. 1A, 1B Medical Observation System

[0278] 2.2A Insertion Unit

[0279] 3. Light source device

[0280] 4. Optical guide

[0281] 5 cameras

[0282] 6 First transmission cable

[0283] 7 Display devices

[0284] 8 Second transmission cable

[0285] 9. Control device

[0286] 10 Third transmission cable

[0287] 12. Surgical microscope

[0288] 15 Ring lights

[0289] 21 Eyepiece

[0290] 22 Distal end

[0291] 23. Bend

[0292] 24 Flexible Tubing Section

[0293] 31 First Light Source

[0294] 51 imaging units

[0295] 52 Communication Units

[0296] 91 Communication Unit

[0297] 92 Image Processing Units

[0298] 93 Control Unit

[0299] 94 Input Units

[0300] 95 Output Unit

[0301] 96 storage units

[0302] 100 Endoscopes

[0303] 101 Operation Unit

[0304] 102 General Purpose Cable

[0305] 121 Microscope Unit

[0306] 122 Support section

[0307] 123 base

[0308] 151 Lighting Units

[0309] 152 Subject Image Capture Unit

[0310] 511 Excitation cutoff color filter

[0311] 512 lens unit

[0312] 513, 513b, 513g, 513r image sensors

[0313] 514 Color Filter

[0314] 514b B color filter group

[0315] 514g G color filter set

[0316] 514r R color filter set

[0317] 515 Signal Processing Unit

[0318] 516 Prism

[0319] 921 WB processing unit

[0320] 922 De-mosaic Processing Unit

[0321] 923 Correction Matrix Processing Unit

[0322] 924 Gamma Processing Unit

[0323] 925 YC Converter

[0324] 1511 casing

[0325] 1512 Illumination Lens

[0326] 5131b First light-receiving layer

[0327] 5131g Second light-receiving layer

[0328] 5131r Third light-receiving layer

[0329] Ar region

[0330] Ar1 First Region

[0331] Ar2 Second Region

[0332] Ax optical axis

[0333] CB, CG, CR curves

[0334] CN1 and CN2 connectors

[0335] DG reagent

[0336] F1 capture image

[0337] FR, FG, FB frequency distribution

[0338] OB Observation Object

[0339] P0 Observation of the optical path

[0340] P1 First optical path

[0341] P2 Second Optical Path

[0342] The fluorescence spectra of the observed objects under SPF1, SPF2, SPF3, and SPF4.

[0343] SPUs can generate wavelengths of non-essential light.

[0344] TU test tube

[0345] WC Whiteboard.

Claims

1. A medical image processing device, comprising: The image processing unit is configured to process pixel signals acquired from the pixels of the image sensor. The pixel signal includes: The first pixel signal contains a component of non-essential light and a component of fluorescence from the object being observed. The non-essential light includes autofluorescence, which is generated by components of the observation optical path constituting the light as it propagates through the observation optical path. The component of fluorescence from the object being observed is emitted from a substance contained within the object being observed and excited by the light. The second pixel signal contains at least the component of the unnecessary light, and The image processing unit includes a signal correction unit, which is configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal, and the second pixel signal. The correction coefficient is set based on the spectral characteristics of the fluorescence of the observed object.

2. The medical image processing apparatus according to claim 1, wherein, The amount of the component of the observed object fluorescence contained in the first pixel signal is greater than the amount of the component of the observed object fluorescence contained in the second pixel signal.

3. The medical image processing apparatus according to claim 1, wherein, The signal correction unit is configured to generate the corrected pixel signal by multiplying a correction matrix, which serves as the correction coefficient, with an input matrix, wherein the input matrix includes the signal values ​​of the first pixel signal and the second pixel signal as matrix elements.

4. The medical image processing apparatus according to claim 3, wherein, The first pixel signal is provided in the form of at least two first pixel signals, and The input matrix includes values ​​obtained by multiplying the signal values ​​of the at least two first pixel signals as matrix elements.

5. The medical image processing apparatus according to claim 3, wherein, Set the matrix element in the correction matrix that is multiplied by the signal value of the first pixel signal to 0 or a positive value, and Set the matrix element in the correction matrix that is multiplied by the signal value of the second pixel signal to 0 or a negative value.

6. The medical image processing apparatus according to claim 5, wherein, In each row of the correction matrix, each number obtained by adding the matrix elements is 0.

7. The medical image processing apparatus according to claim 5, wherein, In each row of the correction matrix, each number obtained by adding the matrix elements is greater than 0.

8. The medical image processing apparatus according to claim 5, wherein, In each row of the correction matrix, each number obtained by adding the matrix elements is less than 0.

9. The medical image processing apparatus according to claim 1, wherein, The image sensor is configured to capture each beam of light obtained by spectrally dispersing the light from the observed object through a spectral dispersion component, and The correction coefficient is set based on the spectral characteristics of the fluorescence of the observed object and the spectral characteristics of the spectral dispersion component.

10. The medical image processing apparatus according to claim 1, wherein, The correction coefficient is set based on the spectral characteristics of the fluorescence of the observed object and the spectral characteristics of the unnecessary light.

11. The medical image processing apparatus according to claim 9, wherein, The image processing unit further includes a spectral sensitivity correction unit, which is configured to perform spectral sensitivity correction processing on the first pixel signal and the second pixel signal based on the spectral characteristics of the spectral dispersion component. The signal correction unit is configured to generate the corrected pixel signal by using the first pixel signal and the second pixel signal after the spectral sensitivity correction process has been performed by the spectral sensitivity correction unit.

12. The medical image processing apparatus according to claim 1, wherein, The first pixel signal and the second pixel signal are acquired from the same image sensor.

13. The medical image processing apparatus according to claim 1, wherein, Multiple image sensors are provided, and The first pixel signal and the second pixel signal are acquired from different image sensors.

14. The medical image processing apparatus according to claim 1, wherein, The fluorescence of the observed object is generated by a reagent or fluorescent dye applied to the observed object, or by a fluorescent substance derived from the observed object.

15. The medical image processing apparatus according to claim 1, wherein, The components constituting the observation optical path and configured to generate the autofluorescence are optical fibers, lenses, glass, adhesive materials, or color filters.

16. The medical image processing apparatus according to claim 1, wherein, The corrected pixel signal is a signal whose signal value based on the unnecessary light has been reduced.

17. A medical observation system, comprising: A light source device includes a first light source configured to emit first light, the first light comprising narrowband light that excites a substance contained in an object being observed; An imaging device includes an image sensor configured to capture a second light, the second light being a return light from the first light from the object being observed; as well as A medical image processing device includes an image processing unit configured to process pixel signals corresponding to pixels of the image sensor. The pixel signal includes: The first pixel signal includes a component of non-essential light and a component of fluorescence of the observed object. The non-essential light includes spontaneous fluorescence generated from components of the observation optical path constituting the first light as the first light emitted from the light source device propagates through the observation optical path. The component of fluorescence of the observed object is emitted from a substance contained in the observed object excited by the first light. The second pixel signal contains at least the component of the unnecessary light, and The image processing unit includes a signal correction unit, which is configured to generate a corrected pixel signal based on a correction coefficient, the first pixel signal, and the second pixel signal. The correction coefficient is set based on the spectral characteristics of the fluorescence of the observed object.

18. The medical observation system according to claim 17, wherein, The first light source is configured as one or more light sources.

19. The medical observation system according to claim 17, wherein, The first light source is set as an LED or a semiconductor laser.

20. The medical observation system according to claim 17, wherein, The first light is set to broadband light or narrowband light.

Images