Medical control device and medical vision system

Abstract

To generate images suitable for observation. [Solution] The medical control device 9 includes a drive mode switching unit 94 that switches the drive mode of a rolling shutter type image sensor 521 in which multiple pixels are arranged in a two-dimensional manner in units of horizontal lines, and a light control unit 94 that controls a light-emitting element 31 that emits light according to the supplied current. The light control unit 94 switches the control of the light-emitting element 31 according to the drive mode.

Description

Technical Field

[0005] , ,

[0004] ,

[0001] The present disclosure relates to a medical control device.

Background Art

[0002] Conventionally, a medical observation system that images an observation target such as inside a living body using an imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) and observes the observation target is known (see, for example, Patent Document 1). In the medical observation system described in Patent Document 1, the control of the light emitting element is switched according to the type of the imaging device such as CCD and CMOS in the imaging device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when using a CMOS imaging device, if the drive mode of the imaging device is changed, the all-line exposure period in which all horizontal lines of the effective pixel region in the imaging device are simultaneously exposed and the readout period for reading the charges accumulated in a plurality of pixels in the imaging device are also changed. And when the light emitting element is controlled identically in all drive modes, stripe patterns due to exposure unevenness may occur in the obtained captured image. Therefore, there is a demand for a technology that can suppress the occurrence of stripe patterns and generate an image suitable for observation even when the drive mode of the imaging device is changed.

[0005] This disclosure has been made in view of the above, and aims to provide a medical control device capable of generating images suitable for observation. [Means for solving the problem]

[0006] To solve the above-mentioned problems and achieve the objective, the medical control device according to this disclosure includes a drive mode switching unit that switches the drive mode of a rolling shutter type image sensor in which a plurality of pixels are arranged in a two-dimensional manner in units of horizontal lines, and a light control unit that performs light control including a first control that adjusts the supply time of the current supplied to a light-emitting element that emits light according to the supplied current, a second control that adjusts the current value supplied to the light-emitting element, and a third control that adjusts the electronic shutter of the image sensor. The light control unit performs the light control by fixing the electronic shutter in the third control when the drive mode is a first drive mode in which pixels in the region that contributes to image generation of the image sensor are read out in a first period shorter than the period for reading out all of the pixels by a method including binning, and when the drive mode is a second drive mode in which exposure is performed within 1 field and pixels in the region that contributes to image generation of the image sensor are read out in a second period longer than the first period, the light control unit performs the light control by adjusting the electronic shutter in the third control. [Effects of the Invention]

[0007] The medical control device relating to this disclosure can generate images suitable for observation. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows the configuration of the medical observation system according to Embodiment 1. [Figure 2] Figure 2 is a block diagram showing the configuration of the camera head and control device. [Figure 3] Figure 3 illustrates the dimming control in the first driving mode. [Figure 4] Figure 4 illustrates the dimming control in the first driving mode. [Figure 5] Figure 5 illustrates the dimming control in the second drive mode. [Figure 6] Figure 6 illustrates dimming control in the third drive mode. [Figure 7] Figure 7 shows the configuration of the medical observation system according to Embodiment 2. [Figure 8] Figure 8 illustrates the functions of the control unit (drive mode switching unit and dimming control unit) in the second observation mode. [Modes for carrying out the invention]

[0009] The embodiments for implementing this disclosure (hereinafter referred to as "embodiments") will be described below with reference to the drawings. However, the embodiments described below do not limit this disclosure. Furthermore, the same parts are denoted by the same reference numerals in the drawings.

[0010] (Embodiment 1) [Outline configuration of a medical observation system] Figure 1 shows the configuration of the medical observation system 1 according to Embodiment 1. Medical observation system 1 is a system used in the medical field to image (observe) the inside of a living body (object of observation). As shown in Figure 1, medical observation system 1 comprises an insertion unit 2, a light source device 3, a light guide 4, a camera head 5, a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

[0011] In this embodiment 1, the insertion section 2 is made of a rigid endoscope. That is, the insertion section 2 has an elongated shape that is either entirely rigid or partially flexible with other parts rigid, and is inserted into the body. Inside this insertion section 2, there is an optical system made up of one or more lenses that focuses light from the subject.

[0012] The light source device 3 is connected to one end of the light guide 4 and supplies light for irradiating the living body to one end of the light guide 4 under the control of the control device 9. As shown in FIG. 1, this light source device 3 includes a first light source 31. The first light source 31 corresponds to the light-emitting element and the first light-emitting element according to the present disclosure. This first light source 31 emits (emits) normal light in a visible wavelength band (first wavelength band) according to the supplied current. In the first embodiment, the first light source 31 is composed of an LED (Light Emitting Diode) that emits white light. In the first embodiment, the light source device 3 is configured separately from the control device 9, but it is not limited to this, and a configuration provided inside the control device 9 may be adopted.

[0013] One end of the light guide 4 is detachably connected to the light source device 3, and the other end is detachably connected to the insertion portion 2. Then, the light guide 4 transmits the light (normal light) supplied from the light source device 3 from one end to the other end and supplies it to the insertion portion 2. The normal light irradiated into the living body and reflected by the living body is condensed by the optical system in the insertion portion 2.

[0014] The camera head 5 corresponds to the imaging device according to the present disclosure. This camera head 5 is detachably connected to the proximal end (eyepiece portion 21 (FIG. 1)) of the insertion portion 2. Then, the camera head 5 captures the light condensed by the insertion portion 2 under the control of the control device 9 and generates a captured image. Details of the configuration of the camera head 5 will be described in the "Configuration of Camera Head" described later.

[0015] One end of the first transmission cable 6 is detachably connected to the control device 9 via the connector CN1 (FIG. 1), and the other end is detachably connected to the camera head 5 via the connector CN2 (FIG. 1). Then, the first transmission cable 6 transmits the captured image and the like output from the camera head 5 to the control device 9, and also transmits the control signal, synchronization signal, clock, and power output from the control device 9 to the camera head 5 respectively. Incidentally, the transmission of imaging images and the like from the camera head 5 to the control device 9 via the first transmission cable 6 may be performed by transmitting the imaging images and the like as optical signals, or may be transmitted as electrical signals. The same applies to the transmission of control signals, synchronization signals, and clocks from the control device 9 to the camera head 5 via the first transmission cable 6.

[0016] The display device 7 is composed of a display such as a liquid crystal or an organic EL (Electro Luminescence), and displays an image based on the video signal from the control device 9 under the control of the control device 9. One end of the second transmission cable 8 is detachably connected to the display device 7, and the other end is detachably connected to the control device 9. Then, the second transmission cable 8 transmits the video signal processed by the control device 9 to the display device 7.

[0017] The control device 9 corresponds to the medical control device according to the present disclosure. This control device 9 is composed of a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), etc., and comprehensively controls the operations of the light source device 3, the camera head 5, and the display device 7. The detailed configuration of the control device 9 will be described in the "Configuration of the Control Device" described later. One end of the third transmission cable 10 is detachably connected to the light source device 3, and the other end is detachably connected to the control device 9. Then, the third transmission cable 10 transmits the control signal from the control device 9 to the light source device 3.

[0018] 〔Configuration of the Camera Head〕 Next, the configuration of the camera head 5 will be described. FIG. 2 is a block diagram showing the configurations of the camera head 5 and the control device 9. As shown in FIG. 2, the camera head 5 includes a lens unit 51, an imaging unit 5, and a communication unit 53. The lens unit 51 is composed of one or more lenses and focuses the light (normal light) collected in the insertion section 2 onto the imaging surface of the imaging unit 52 (image sensor 521). For the sake of explanation, in the following, the normal light (white light) directed from the lens unit 51 to the image sensor 521 will be referred to as the subject image.

[0019] The imaging unit 52 images the inside of a living body under the control of the control device 9. As shown in Figure 2, the imaging unit 52 comprises an image sensor 521 and a signal processing unit 522. The image sensor 521 receives the subject image and converts it into an electrical signal (analog signal). In this embodiment 1, the image sensor 521 is composed of a CMOS, which is a rolling shutter type image sensor in which multiple pixels are arranged in a two-dimensional manner in units of horizontal lines. Here, although not shown in detail in the illustration, the image sensor 521 consists of an electrically unguaranteed invalid region, an optical black region (OB region), and an effective pixel region that converts the subject image formed by the lens unit 51 into an imaging signal for output. For the sake of explanation, in the following, the image generated by capturing a subject image (in normal light) with the image sensor 521 will be referred to as a normal light image. The signal processing unit 522 performs signal processing on the analog signal normal light image generated by the image sensor 521 and outputs a digital signal normal light image.

[0020] The communication unit 53 functions as a transmitter that transmits the normal optical image output from the imaging unit 52 to the control device 9 via the first transmission cable 6. This communication unit 53 is configured, for example, as a high-speed serial interface that communicates the normal optical image with the control device 9 via the first transmission cable 6 at a transmission rate of 1 Gbps or more.

[0021] [Control device configuration] Next, the configuration of the control device 9 will be explained with reference to Figure 2. As shown in Figure 2, the control device 9 includes a communication unit 91, a memory 92, an observation image generation unit 93, a control unit 94, an input unit 95, an output unit 96, and a storage unit 97. The communication unit 91 functions as a receiver that receives the normal optical image output from the camera head 5 (communication unit 53) via the first transmission cable 6. This communication unit 91 is configured, for example, with a high-speed serial interface that communicates the normal optical image with the communication unit 53 at a transmission rate of 1 Gbps or more. The memory 92 is composed of, for example, DRAM (Dynamic Random Access Memory). This memory 92 can temporarily store multiple frames of normal light images that are sequentially output from the camera head 5 (communication unit 53).

[0022] The observation image generation unit 93 processes the normal light images that are sequentially output from the camera head 5 (communication unit 53) and received by the communication unit 91, under the control of the control unit 94. As shown in Figure 2, this observation image generation unit 93 comprises a memory controller 931, a normal light image processing unit 932, and a display control unit 933.

[0023] The memory controller 931 controls the writing and reading of normal light images to and from the memory 92. More specifically, the memory controller 931 sequentially writes the normal light images output from the camera head 5 (communication unit 53) and received by the communication unit 91 to the memory 92. The memory controller 931 also reads the normal light images from the memory 92 at specific timings and inputs the read normal light images to the normal light image processing unit 932.

[0024] The normal light image processing unit 932 performs a first image processing operation on the input normal light image. Examples of this first image processing include optical black subtraction, white balance adjustment, demosaicing, color correction matrix processing, gamma correction processing, and YC processing, which converts RGB signals (normal light images) into luminance chrominance signals (Y, Cb / Cr signals).

[0025] The display control unit 933, under the control of the control unit 94, generates a video signal for displaying the normal light image after the first image processing has been performed by the normal light image processing unit 932. The display control unit 933 then outputs the video signal to the display device 7 via the second transmission cable 8.

[0026] The control unit 94 is configured using, for example, a CPU or FPGA, and controls the operation of the light source device 3, camera head 5, and display device 7, as well as the operation of the entire control device 9, by outputting control signals via the first to third transmission cables 6, 8, and 10. This control unit 94 has the functions of a drive mode switching unit and a dimming control unit according to this disclosure. The functions of the drive mode switching unit and the dimming control unit will be explained later in the section "Operation of the Control Device".

[0027] The input unit 95 is configured using operating devices such as a mouse, keyboard, and touch panel, and accepts user operations from users such as doctors. The input unit 95 then outputs an operation signal corresponding to the user operation to the control unit 94. The output unit 96 is configured using a speaker, printer, etc., and outputs various types of information. The memory unit 97 stores programs executed by the control unit 94, as well as information necessary for the processing performed by the control unit 94.

[0028] [Operation of the control device] Next, the operation of the control device 9 described above will be explained. For the sake of clarity, the following explanation will primarily focus on the functions of the drive mode switching unit and the dimming control unit in the control unit 94. The control unit 94 controls the image sensor 521 as follows. Specifically, the control unit 94 performs exposure control using a so-called rolling shutter method, which sequentially starts exposure of the image sensor 521 for each horizontal line during one field period, and sequentially reads out each horizontal line after a predetermined period (so-called shutter speed) has elapsed since the start of exposure. The control unit 94 (drive mode switching unit) then switches the control pattern (drive mode) of the exposure control described above. In this embodiment 1, the control unit 94 (drive mode switching unit) switches the drive mode to one of the first to third drive modes in response to a user operation on the input unit 95 by a user such as a surgeon. Details of the first to third drive modes will be described later.

[0029] Furthermore, the control unit 94 (dimming control unit) performs dimming control to adjust the normal light image to a reference brightness based on the brightness (average value of luminance, etc.) of a specific area (detection area) in the normal light image. In this embodiment 1, the control unit 94 is capable of performing a first control to adjust the supply time (current application pulse width) of the current supplied to the first light source 31, a second control to adjust the current value supplied to the first light source 31, and a third control to adjust the electronic shutter of the image sensor 521 as part of the dimming control. The control unit 94 then switches the dimming control according to the drive mode of the image sensor 521. The dimming control in the first drive mode, the dimming control in the second drive mode, and the dimming control in the third drive mode will be explained below in order.

[0030] [Dimming control in the first drive mode] Figures 3 and 4 illustrate the dimming control in the first drive mode. Specifically, Figures 3(a) and 4(a) show the exposure timing of the image sensor 521, with the vertical axis representing the horizontal line of the image sensor 521 (the top row represents the uppermost horizontal line (the first horizontal line), and the bottom row represents the lowermost horizontal line (the final line)), and the horizontal axis representing time. The parallelogram region is the region that contributes to the generation of a 1-field normal light image. Figures 3(b) and 4(b) show the dimming control, with the vertical axis representing the current value supplied to the first light source 31, and the horizontal axis representing time (the supply time of the current supplied to the first light source 31). In Figures 3(b) and 4(b), the third control (adjustment of the electronic shutter) of the dimming control is represented by a diagonal line. Furthermore, Figure 3 shows the case where the brightness of the normal light image is increased using dimming control, and Figure 4 shows the case where the brightness of the normal light image is decreased using dimming control.

[0031] The first drive mode, as shown in Figures 3(a) and 4(a), is a drive mode in which the readout period TR for reading the charge accumulated in multiple pixels of the image sensor 521 is long, and a full line exposure period cannot be obtained. Here, the full line exposure period is the period in which all horizontal lines of the effective pixel area of ​​the image sensor 521 are exposed simultaneously.

[0032] Then, when the drive mode of the image sensor 521 is the first drive mode, the control unit 94 (light control unit) performs the following light control. First, let's consider the case where the brightness of a normal light image is increased using dimming control. In this case, as shown in Figure 3(b), the control unit 94 (dimming control unit) fixes the supply time of the current supplied to the first light source 31 for each field (the supply time: 1 field), and while fixing the electronic shutter of the image sensor 521 (NTSC system: 1 / 60 [seconds]), it increases the current value supplied to the first light source 31 in the second control, thereby increasing the brightness of the normal light image.

[0033] Next, let's consider the case where the brightness of a normal light image is reduced using dimming control. In this case, as shown in Figure 4(b), the control unit 94 (dimming control unit) fixes the supply time of the current supplied to the first light source 31 for each field (the supply time: 1 field), and while fixing the current value supplied to the first light source 31 at the minimum rated current value Imin, it adjusts the electronic shutter with the third control to darken the brightness of the normal light image. In Figure 4(b), the shaded parallelogram region represents the area where charge is swept away by the electron shutter. The open-circle parallelogram region represents the effective exposure period. As described above, when the drive mode of the image sensor 521 is the first drive mode, the control unit 94 (light control unit) executes only the second and third controls out of the first to third controls as light control.

[0034] [Dimming control in the second drive mode] Figure 5 illustrates dimming control in the second drive mode. Specifically, Figure 5 corresponds to Figures 3 and 4. Here, we consider a scenario where the readout period is long and the entire line exposure period cannot be achieved, as shown by the dashed line in Figure 5(a). The second driving mode is a mode called long exposure or long exposure, in which multiple fields (two in the example of Figure 5(a)) are pseudo-combined into a single field. That is, when the driving mode of the image sensor 521 is switched from the driving mode shown by the dashed line in Figure 5(a) to the second driving mode, a pseudo-total line exposure period TE is generated for each pseudo-combined field, as shown by the solid line in Figure 5(a).

[0035] Then, when the drive mode of the image sensor 521 is the second drive mode, the control unit 94 (light control unit) performs the following light control. As shown in Figure 5(b), the control unit 94 (light control unit) fixes the electronic shutter of the image sensor 521 for each pseudo-grouped field and sets the current value supplied to the first light source 31 for the entire line exposure period TE, which is pseudo-generated by the second drive mode, to the maximum rated current value Imax. Then, for each pseudo-grouped field, the control unit 94 (light control unit) adjusts the brightness of the normal light image by adjusting the supply time of the current supplied to the first light source 31 in the first control during the readout period TR, which is pseudo-generated by the second drive mode. Here, the amount of light emitted from the first light source 31 during the entire line exposure period TE (current value supplied to the first light source 31 × current supply time) is higher than the amount of light emitted from the first light source 31 during the readout period TR. As described above, when the drive mode of the image sensor 521 is the second drive mode, the control unit 94 (light control unit) executes only the first and second controls out of the first to third controls as light control.

[0036] [Dimming control in the third drive mode] Figure 6 illustrates dimming control in the third drive mode. Specifically, Figure 6 corresponds to Figures 3 through 5. Note that in Figure 6(a), Here, we consider a scenario where the readout period is long and it is not possible to expose the entire line, as shown by the dashed line in Figure 6(a). The third drive mode is a drive mode that includes binning. This binning is not limited to adding adjacent pixels, but also includes reading out while downsampling some pixels. In other words, when the drive mode of the image sensor 521 is switched from the drive mode shown by the dashed line in Figure 6(a) to the third drive mode, the readout speed can be increased as shown by the solid line in Figure 6(a), and the full line exposure period TE is pseudo-generated for each field.

[0037] Then, when the drive mode of the image sensor 521 is the third drive mode, the control unit 94 (light control unit) performs the following light control. As shown in Figure 6(b), the control unit 94 (light control unit) fixes the electronic shutter of the image sensor 521 for each field and sets the current value supplied to the first light source 31 for the entire line exposure period TE, which is artificially generated by the third drive mode, to the maximum rated current value Imax. Then, for each field, the control unit 94 (light control unit) adjusts the brightness of the normal light image by adjusting the supply time of the current supplied to the first light source 31 in the first control during the readout period TR, which is artificially generated by the third drive mode. Here, the amount of light emitted from the first light source 31 during the entire line exposure period TE (current value supplied to the first light source 31 × current supply time) is higher than the amount of light emitted from the first light source 31 during the readout period TR. As described above, when the drive mode of the image sensor 521 is the third drive mode, the control unit 94 (light control unit) executes only the first and second controls out of the first to third controls as light control.

[0038] According to the first embodiment described above, the following effects are achieved. The control device 9 according to this embodiment 1 switches between the first to third control (dimming control) according to the drive mode of the image sensor 521. Specifically, when the drive mode of the image sensor 521 is the first drive mode, the control device 9 performs only the second and third controls out of the first to third controls as dimming control. Here, when increasing the brightness of the normal light image, a second control is performed. That is, the difference in brightness between the upper and lower positions in the normal light image increases in proportion to the increase in the amount of light emitted between adjacent fields (current value supplied to the first light source 31 × current supply time). In other words, by reducing the increase in the amount of light emitted between adjacent fields, the difference in brightness within the normal light image can be suppressed. Also, when decreasing the brightness of the normal light image, the amount of light emitted from the first light source 31 is fixed while adjusting the electronic shutter of the image sensor 521. That is, since the electronic shutter sweeps away the charge of all lines of the image sensor 521 for the same amount of time, no difference in brightness occurs within the normal light image.

[0039] Furthermore, when the drive mode of the image sensor 521 is the second or third drive mode, the control device 9 performs only the first and second controls out of the first to third controls as dimming control. Here, the control device 9 sets the current value supplied to the first light source 31 for the entire pseudo-generated line exposure period TE to the maximum rated current value Imax, and performs the first control during the pseudo-generated readout period TR. In other words, by making the first light source 31 emit light at full power for the entire line exposure period TE, the brightness difference in the normal light image caused by the first control during the readout period TR can be made less noticeable. Based on the above, even when the drive mode of the image sensor 521 is changed, it is possible to suppress the brightness difference in the normal light image and generate a normal light image suitable for observation.

[0040] (Embodiment 2) Next, Embodiment 2 will be described. In the following description, components similar to those in Embodiment 1 described above are denoted by the same reference numerals, and their detailed descriptions are omitted or simplified. Figure 7 shows the configuration of the medical observation system 1A according to Embodiment 2. In the medical observation system 1A according to this second embodiment, a first observation mode and a second observation mode are made executable. The first observation mode is similar to the medical observation system 1 of Embodiment 1 described above, in which the inside of a living body irradiated with normal light is imaged by the image sensor 521 to obtain a normal light image (first image). The second observation mode is an observation mode in which, in addition to obtaining a normal light image, the inside of a living body is irradiated with light in a second wavelength band, which is a different wavelength band from normal light, and the image sensor 521 captures the image to obtain a second image (hereinafter referred to as a fluorescence image).

[0041] Furthermore, in the medical observation system 1A, as shown in Figure 7, the configuration of the light source device 3, camera head 5, and control device 9 has been changed compared to the medical observation system 1 described in Embodiment 1 above. For the sake of explanation, the light source device 3, camera head 5, and control device 9 according to Embodiment 2 will be referred to as light source device 3A, camera head 5A, and control device 9A below.

[0042] As shown in Figure 7, the light source device 3A has a second light source 32 added to the light source device 3 described in the embodiment 1 above. The second light source 32 corresponds to the light-emitting element and the second light-emitting element according to this disclosure. This second light source 32 is composed of a semiconductor laser or LED that emits near-infrared excitation light in the near-infrared wavelength band (for example, a wavelength band of about 750 nm to 800 nm), which is the second wavelength band. This near-infrared excitation light is excitation light that excites a fluorescent substance such as indocyanine green. Furthermore, when the fluorescent substance such as indocyanine green is excited by this near-infrared excitation light, it emits fluorescence in a wavelength band other than the visible region (for example, a wavelength band around 830 nm) whose center wavelength is at a longer wavelength than the center wavelength of the wavelength band of the near-infrared excitation light. Note that the wavelength band of the near-infrared excitation light and the wavelength band of the fluorescence may be set to partially overlap, or they may be set to not overlap at all.

[0043] In the light source device 3A according to this second embodiment, in the first observation mode, under the control of the control device 9A, only the first light source 31 is driven, as shown in Figures 3 to 6 in the first embodiment described above. Furthermore, in the light source device 3A according to this second embodiment, in the second observation mode, under the control of the control device 9A, the first light source 31 is driven during the first period of the first and second periods which are alternately repeated. That is, during the first period, the light source device 3 emits normal light (white light). Also, in the light source device 3A, under the control of the control device 9A, the second light source 32 is driven during the second period. That is, during the second period, the light source device 3A emits near-infrared excitation light. When near-infrared excitation light is irradiated into the living body via the light guide 4 and the insertion section 2, the near-infrared excitation light transmitted through the living body (near-infrared excitation light reflected from the living body) and the fluorescence emitted from fluorescent substances such as indocyanine green that accumulate in the lesion in the living body, which are excited by the near-infrared excitation light, are focused by the optical system in the insertion section 2.

[0044] As shown in Figure 7, camera head 5A has an excitation light cut filter 523 added to the imaging unit 52 compared to camera head 5 described in Embodiment 1 above. The excitation light cut filter 523 is provided between the lens unit 51 and the image sensor 521 and is composed of a band stop filter that removes a specific wavelength band. For the sake of explanation, in the following, the wavelength band that is cut (removed) by the excitation light cut filter 523 will be referred to as the cut band, the wavelength band that is shorter than the cut band and passes through the excitation light cut filter 523 will be referred to as the short-wave side transmission region, and the wavelength band that is longer than the cut band and passes through the excitation light cut filter 523 will be referred to as the long-wave side transmission region. Here, the cut-off band includes at least a portion of the wavelength band of the near-infrared excitation light. In this embodiment 1, the cut-off band includes the entire wavelength band of the near-infrared excitation light. Furthermore, the long-wave side transmission band includes the fluorescence wavelength band. In addition, the short-wave side transmission region includes the wavelength band of normal light (white light).

[0045] In other words, the excitation light cut filter 523 transmits the subject image (normal light) traveling from the lens unit 51 to the image sensor 521. On the other hand, the excitation light cut filter 523 removes the near-infrared excitation light and transmits the fluorescence traveling from the lens unit 51 to the image sensor 521. For the sake of explanation, in the following, the fluorescence that passes through the excitation light cut filter 523 and travels to the image sensor 521 will be referred to as the fluorescence image. The image sensor 521 generates a normal light image by capturing an image of the subject, and a fluorescence image by capturing an image of the fluorescence.

[0046] As shown in Figure 7, in the control device 9A, compared to the control device 9A described in Embodiment 1 above, a fluorescence image processing unit 934 and a superimposed image generation unit 935 are added to the observation image generation unit 93. In this case, the memory controller 931 operates in the first observation mode in the same manner as described in Embodiment 1 above. Furthermore, the memory controller 931 operates as follows in the second observation mode. The memory controller 931 sequentially writes the captured images (normal light images and fluorescence images) output from the camera head 5A (communication unit 53) and received by the communication unit 91 to the memory 92. The memory controller 931 also reads the normal light images from the memory 92 at specific timings and inputs these read normal light images to the normal light image processing unit 932. Furthermore, the memory controller 931 reads the fluorescence images from the memory 92 at specific timings and inputs these read fluorescence images to the fluorescence image processing unit 934.

[0047] In the second observation mode, the fluorescence image processing unit 934 performs a second image processing operation on the input fluorescence image, which is different from the first image processing operation normally performed by the optical image processing unit 932. Examples of the second image processing include optical black subtraction, white balance adjustment, demosaicing, color correction matrix processing, gamma correction, and YC processing, which converts RGB signals (fluorescent images) into luminance chrominance signals (Y, Cb / Cr signals), similar to the first image processing.

[0048] In the second observation mode, the superimposed image generation unit 935 performs a superimposition process to generate a superimposed image by superimposing a fluorescence image, which has been processed in a second image by the fluorescence image processing unit 934, onto the normal light image, which has been processed in a first image by the normal light image processing unit 932. Here, the superposition process can be exemplified by the first and second superposition processes shown below. In the following, in a fluorescence image, the region consisting of pixels whose brightness value is above a certain threshold will be referred to as the fluorescence region. The first superposition process involves replacing areas in a normal light image that have the same pixel positions as the fluorescent regions with images of the fluorescent regions in the fluorescence image. The second superposition process is a process (so-called alpha blending) that changes the brightness of the fluorescent color applied to each pixel in the region that corresponds to the same pixel position in the fluorescent region of the normal light image, according to the brightness value of each pixel position in the fluorescent region of the fluorescent image.

[0049] Then, in the first observation mode, the display control unit 933 generates a video signal for displaying the normal light image after the first image processing has been performed by the normal light image processing unit 932, as described in Embodiment 1 above. Furthermore, in the second observation mode, the display control unit 933 generates a video signal for displaying the superimposed image generated by the superimposed image generation unit 935.

[0050] Furthermore, the control unit 94 according to this embodiment 2 has the functions of a drive mode switching unit and a dimming control unit according to this disclosure, as well as the function of an observation mode switching unit.

[0051] Next, the operation of the control device 9A described above will be explained. For the sake of clarity, the following explanation will primarily focus on the functions of the drive mode switching unit, the dimming control unit, and the observation mode switching unit in the control unit 94. The control unit 94 (observation mode switching unit) switches the observation mode to either the first or second observation mode in response to user operations on the input unit 95 by a user, such as a surgeon. In the first observation mode, the control unit 94 (drive mode switching unit and dimming control unit) operates in the same manner as described in the "dimming control in the first drive mode," "dimming control in the second drive mode," and "dimming control in the third drive mode" sections of Embodiment 1 described above. Therefore, only the functions of the control unit 94 (drive mode switching unit and dimming control unit) in the second observation mode will be described below.

[0052] Figure 8 is a diagram illustrating the functions of the control unit 94 (drive mode switching unit and dimming control unit) in the second observation mode. Specifically, Figure 8 corresponds to Figures 3 to 6. Here, the dashed line shown in Figure 8(a) indicates the drive mode before switching. The solid line shown in Figure 8(b) indicates the drive mode after switching (second drive mode). Here, we assume a case where the readout period is long and the total line exposure period TE0 is short, as shown by the dashed line in Figure 8(a). In this case, when the observation mode is switched to the second observation mode, the control unit 94 (drive mode switching unit) switches the drive mode of the image sensor 521 to the second drive mode. Then, when the drive mode of the image sensor 521 is switched from the drive mode shown by the dashed line in Figure 8(a) to the second drive mode, a pseudo-total line exposure period TE1, which is longer than the total line exposure period TE0, is pseudo-generated for each pseudo-grouped field, as shown by the solid line in Figure 8(a).

[0053] Subsequently, the control unit 94 (dimming control unit) alternately repeats the emission of light from the first light source 31 and the emission of light from the second light source 32 for each pseudo-combined field, as shown in Figure 8(b). In Figure 8(a), the solid parallelogram region that contributes to the generation of the normal light image is labeled as the WLI (White Light Imaging) field, and the solid parallelogram region that contributes to the generation of the fluorescence image is labeled as the IR (Infra-Red) field. In Figure 8(b), the emission of normal light is labeled as WLI, and the emission of excitation light is labeled as IR.

[0054] Specifically, the control unit 94 (dimming control unit) causes the first light source 31 to emit light for the entire first period TE11' within the total line exposure period TE11, which is artificially generated by the second drive mode, in the WLI field of the alternating WLI field and IR field. In Figure 8(b), an example is shown where the current supply time is maximized and the current value is set to the maximum rated current value Imax. Therefore, the first period TE11' is the same period as the total line exposure period TE11. Meanwhile, the control unit 94 (dimming control unit) causes the second light source 32 to emit light for the entire second period TE12' within the total line exposure period TE12, which is artificially generated by the second drive mode, in the IR field of the alternating WLI field and IR field. In Figure 8(b), an example is shown where the current supply time is maximized and the current value is set to the maximum rated current value Imax. Therefore, the second period TE12' is the same period as the total line exposure period TE12.

[0055] According to this embodiment 2 described above, in addition to the same effects as in embodiment 1 described above, the following effects are achieved. In this second embodiment, the control device 9A switches the drive mode of the image sensor 521 to the second drive mode when the observation mode is the second observation mode. The control device 9A also causes the first and second light sources 31 and 32 to emit light for each full line exposure period TE that is artificially generated by the second drive mode. In other words, in a configuration where the first and second light sources 31 and 32 are emitted during each full line exposure period TE0, the brightness of the normal light image and the fluorescence image is insufficient. However, by artificially generating a full line exposure period TE1 that is longer than the full line exposure period TE0, and emitting the first and second light sources 31 and 32 during each full line exposure period TE1, sufficient brightness can be ensured for both the normal light image and the fluorescence image. Therefore, images suitable for observation can be generated.

[0056] (Other embodiments) While we have described the forms for implementing this disclosure, this disclosure should not be limited solely to embodiments 1 and 2 described above. In the embodiments 1 and 2 described above, the light control unit according to this disclosure included a function to adjust the electronic shutter of the image sensor 521. However, it is not limited to this, and may also have a function to control only the first light source 31 (or the second light source 32). In embodiments 1 and 2 described above, the number of drive modes for the image sensor 521 was set to three, but it is not limited to this; it may be set to only two, or to four or more.

[0057] In the embodiments 1 and 2 described above, the light source control device according to this disclosure was mounted on a medical observation system 1 in which the insertion section 2 was composed of a rigid endoscope, but the invention is not limited to this. For example, the light source control device according to this disclosure may be mounted on a medical observation system in which the insertion section 2 is composed of a flexible endoscope. Furthermore, the light source control device according to this disclosure may be mounted on a medical observation system such as a surgical microscope (see, for example, Japanese Patent Application Publication No. 2016-42981) that magnifies and observes a predetermined field of view area inside a subject (inside a living body) or on the surface of a subject (on a living body surface). In the embodiments 1 and 2 described above, some of the components of the camera head 5 and some of the components of the control device 9 may be provided in, for example, connector CN1 and connector CN2.

[0058] Incidentally, photodynamic diagnosis (PDD) is a known cancer diagnostic method that has been used to detect cancer cells. In this photodynamic diagnostic method, a photosensitive substance such as 5-aminolevulinic acid (hereinafter referred to as 5-ALA) is used. 5-ALA is a naturally occurring amino acid found in the bodies of plants and animals. After administration to the body, 5-ALA is taken up into cells and biosynthesized into protoporphyrin in the mitochondria. In cancer cells, this protoporphyrin accumulates excessively. Furthermore, the protoporphyrin that accumulates excessively in cancer cells is photoactive. Therefore, when this protoporphyrin is excited by excitation light (for example, blue visible light in the wavelength range of 375 nm to 445 nm), it emits fluorescence (for example, red fluorescence in the wavelength range of 600 nm to 740 nm). This cancer diagnostic method, which uses a photosensitive substance to cause cancer cells to fluoresce, is called photodynamic diagnostics. Furthermore, in the above-described embodiment 2, the first light source 31 may be configured with an LED that emits white light, and the second light source 32 may be configured with a semiconductor laser that emits excitation light (for example, blue visible light in the wavelength range of 375 nm to 445 nm) that excites protoporphyrin. Even in this configuration, the same effects as in embodiment 2 described above will be achieved.

[0059] In the above-described embodiment 2, the first and second periods were set to alternate in the second observation mode. However, the system is not limited to this, and it is also possible to configure the system so that at least one of the first and second periods is consecutive, and the frequency ratio of the first and second periods is not 1:1.

[0060] Furthermore, the following configurations also fall within the technical scope of this disclosure. (1) The system includes a drive mode switching unit that switches the drive mode of a rolling shutter type image sensor in which multiple pixels are arranged in a two-dimensional manner in units of horizontal lines, and a light control unit that controls a light-emitting element that emits light according to the supplied current, wherein the light control unit switches the control of the light-emitting element according to the drive mode. (2) The light control unit is capable of performing light control including a first control for adjusting the supply time of the current supplied to the light-emitting element, a second control for adjusting the value of the current supplied to the light-emitting element, and a third control for adjusting the electronic shutter of the image sensor, and the light control is switched according to the drive mode, as described in (1). (3) The medical control device according to (2) above, wherein the light control unit executes only the second and third controls out of the first, second, and third controls when the drive mode is a first drive mode in which there is no full line exposure period in which all horizontal lines of the effective pixel area of ​​the image sensor are exposed simultaneously. (4) The medical control device according to (2) or (3) above, wherein the dimming control unit performs only the first and second controls among the first, second, and third controls when the drive mode is a second drive mode for long exposure in which multiple fields are pseudo-composed into one field. (5) The light control unit, when the drive mode is the second drive mode, increases the amount of light emitted by the light-emitting element during the full line exposure period, which is a period artificially generated by the second drive mode and in which all lines of the effective pixel area of ​​the image sensor are exposed simultaneously, to be higher than during other periods, as described in (4). (6) The dimming control unit, when the drive mode is a third drive mode including binning, executes only the first control and the second control among the first control, the second control and the third control, as described in any one of (2) to (5) above, for the medical control device. (7) The medical control device according to (6), wherein the dimming control unit, when the drive mode is the third drive mode, increases the amount of light emitted by the light-emitting element during the full-line exposure period, which is a period artificially generated by the third drive mode, in which all lines of the effective pixel area of ​​the image sensor are exposed simultaneously, compared to other periods. (8) The light-emitting element comprises a first light-emitting element that emits light in a first wavelength band and a second light-emitting element that emits light in a second wavelength band different from the first wavelength band, and the medical control device further comprises an observation mode switching unit that switches the observation mode to a first observation mode or a second observation mode, the first observation mode is an observation mode in which an observation target irradiated with light in the first wavelength band is imaged with the image sensor to obtain a first image, and the second observation mode is an observation mode in which the first image is obtained and the observation target irradiated with light in the second wavelength band is imaged with the image sensor to obtain a second image A medical control device according to any one of (1) to (7), wherein the observation mode is obtained, and the drive mode switching unit switches the drive mode to a second long-exposure drive mode in which multiple fields are pseudo-combined into one field when the observation mode is the second observation mode, and the light control unit causes the first light-emitting element and the second light-emitting element to emit light for each full-line exposure period in which all lines of the effective pixel area of ​​the image sensor are simultaneously exposed during a period pseudo-generated by the second drive mode. (9) A medical observation system comprising: an imaging device including a rolling shutter type image sensor in which a plurality of pixels are arranged in a two-dimensional manner in units of horizontal lines; a light source device including an element that emits light in accordance with a supplied current; and a medical control device that controls the operation of the imaging device and the light source device, wherein the medical control device comprises a drive mode switching unit that switches the drive mode of the image sensor and a light control unit that controls the element, and the light control unit switches the control of the element according to the drive mode. [Explanation of Symbols]

[0061] 1.1A Medical Observation System 2 Insertion part 3,3A light source device 4 Light Guide 5.5A Camera Head 6. First transmission cable 7 Display device 8. Second transmission cable 9,9A Control device 10. Third transmission cable 21 Eyepiece 31 First light source 32 Second light source 51 Lens Unit 52 Imaging Department 53 Communications Department 91 Communications Department 92 memory 93 Observation Image Generation Unit 94 Control Unit 95 Input section 96 Output section 97 Memory section 521 Image sensor 522 Signal Processing Unit 523 Excitation light cut filter 931 Memory Controller 932 Normal Optical Image Processing Unit 933 Display Control Unit 934 Fluorescence Image Processing Unit 935 Superimposed Image Generation Unit CN1, CN2 connectors TE0, TE1, TE11, TE12 Exposure period for all lines TE11' First period TE12' Second period TR Readout Period

Claims

[Claim 1] A drive mode switching unit that switches the drive mode of a rolling shutter type image sensor in which multiple pixels are arranged in a two-dimensional manner in units of horizontal lines, The system includes a dimming control unit that performs dimming control, which includes a first control that adjusts the supply time of the current supplied to a light-emitting element that emits light in accordance with the supplied current, a second control that adjusts the current value supplied to the light-emitting element, and a third control that adjusts the electronic shutter of the image sensor. The dimming control unit, In the case of a first drive mode in which the pixels in the region contributing to image generation of the image sensor are read out in a first period shorter than the period for reading out all of the pixels, the third control fixes the electronic shutter and performs the dimming control. In the case where the drive mode is a second drive mode in which exposure is performed within one field and pixels in the region contributing to image generation of the image sensor are read out over a second period longer than the first period, the medical control device adjusts the electronic shutter with the third control to perform the dimming control.

Images