Endoscope system

Abstract

To provide a technology to make unevenness of brightness and horizontal stripes inconspicuous even when a change in a pulse emission period continues to a rolling shutter period while avoiding distortion attributed to the roller shutter and occurrence of artifacts.SOLUTION: An endoscope system includes: a plurality of semiconductor light emitting elements; an image pickup device of a rolling shutter system; a main control unit for controlling a light emitting profile of the plurality of semiconductor light emitting elements on the basis of an image signal; a light source control unit for driving the plurality of semiconductor light emitting elements by a driving signal according to the light emitting profile; and a photometric unit for detecting an exposure level of the image pickup device. The light emitting profile is composed of a two-step light emission period including a strong light emission period for emitting light with predetermined light intensity and a weak light emission period only for emitting light with light intensity weaker than the predetermined light intensity. The main control unit executes reduction control processing for replacing a light emission amount in the weak light emission period with a light emission amount in the strong light emission period while maintaining a total amount of the light emission amount in the strong light emission period and the light emission amount in the weak light emission period at a fixed amount.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to an endoscope system.

Background Art

[0002] In a normal endoscope device equipped with an image sensor of the rolling shutter method, the light source is turned off during the effective pixel readout period (rolling shutter period) of the image sensor, and the light source is turned on during the other period (pseudo global exposure period) (pulse light emission control), thereby performing pseudo global exposure and avoiding undesirable phenomena caused by the rolling shutter, such as distortion and artifacts.

[0003] On the other hand, if the light source is completely turned off during the rolling shutter period, the amount of light may be insufficient depending on the subject (observation target site), and a good image cannot be obtained. For example, Patent Documents 1 to 3 show light source control that includes a part of the rolling shutter period in the pulse light emission period in order to eliminate this light amount shortage.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when performing light source control as in Patent Documents 1 to 3, uneven brightness or horizontal stripes may occur on the screen due to the exposure time difference for each line in adjacent frames. And, due to the change in the pulse emission period for each frame, there is a problem that such uneven brightness and horizontal stripes move up and down on the display screen, which is annoying. Also, when offset emission is performed during the rolling shutter period to eliminate insufficient light quantity, if the offset emission becomes strong to a certain extent, unnatural images such as double exposure of a long exposure image and a high-speed exposure image will be generated.

[0006] The present disclosure has been made in view of such a situation, and proposes a technique for ensuring a sufficient light quantity while avoiding the occurrence of distortion and artifacts caused by the rolling shutter, and making uneven brightness and horizontal stripes less noticeable even when changes in the pulse light emitting device extend over the rolling shutter period.

Means for Solving the Problems

[0007] In order to solve the above problems, the present disclosure is an endoscope system that inserts an endoscope into an observation target and acquires an image of a subject, a plurality of semiconductor light emitting elements that emit light having different wavelength bands respectively, a rolling shutter type imaging element that irradiates the subject with illumination light, detects reflected light from the subject, and generates an image signal, a processor that processes the image signal to generate an image of the subject and displays it on a monitor, a main control unit that generates a control signal for controlling the light emission profile of the plurality of semiconductor light emitting elements based on the image signal, a light source control unit that receives the control signal from the main control unit and drives the plurality of semiconductor light emitting elements with a drive signal corresponding to the light emission profile, a photometry unit that detects the exposure level of the imaging element, and the light emission profile is composed of a two-stage light emission period including only a strong light emission period in which light is emitted at a predetermined light intensity and a weak light emission period in which light is emitted at a light intensity weaker than the predetermined light intensity. The main control unit executes a reduction control process of replacing the light emission amount in the weak light emission period with the light emission amount in the strong light emission period while maintaining the total amount of the light emission amount in the strong light emission period and the light emission amount in the weak light emission period constant, and determines the light emission profile. An endoscope system is proposed in which, before and after the reduction control process, the light intensity in the strong light emission period is always constant, and only the light intensity in the weak light emission period changes before and after the reduction control process.

[0008] Further features related to the present disclosure will become apparent from the description in this specification and the accompanying drawings. Also, the present disclosure is achieved and realized by elements and combinations of various elements and the aspects of the following detailed description and the appended claims. It should be understood that the description in this specification is merely a typical example and does not limit the claims or application examples in any sense.

Effect of the Invention

[0009] According to the present disclosure, while avoiding the occurrence of distortion and artifacts caused by the rolling shutter, it is possible to secure a sufficient amount of light, and even if the change in the pulse light emitting device extends over the rolling shutter period, it is possible to make uneven brightness and horizontal stripes less noticeable.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following, an endoscope system will be described as an example of an embodiment of the present disclosure.

[0012] The target site to be observed in the endoscope system is, for example, the respiratory organs such as the respiratory tract, the digestive organs such as the digestive tract. The respiratory organs such as the respiratory tract are, for example, the lungs, bronchi, and otorhinolaryngology. The digestive organs such as the digestive tract are, for example, the large intestine, small intestine, stomach, esophagus, duodenum, uterus, bladder, and the like. When observing the target site as described above, it is more effective to utilize an image that emphasizes a specific biological structure.

[0013] <Configuration of Endoscope System> FIG. 1 is a diagram showing an example of the overall external appearance of the endoscope system of the present embodiment, and FIG. 2 is a diagram showing an example of the schematic internal configuration of the endoscope system of the present embodiment. The endoscope system 1 includes an endoscope device (electronic endoscope) 100, a processor 200, and a monitor 300. A scope connector (hereinafter, sometimes simply referred to as a "connector") 400 including a connector circuit according to the features of the present embodiment is provided at the processor side end of the endoscope device 100.

[0014] The endoscope device 100 includes an elongated tubular insertion portion 11 to be inserted into the interior of a subject. The endoscope device 100 includes, for example, an LCB (Light Carrying Bundle) 101 for guiding irradiation light from a light source device 201 described later, a light distribution lens 102 provided at the emission end of the LCB 101, an imaging unit 103 that receives return light from an irradiated portion (observation site) via an objective lens (not shown), a driver signal processing circuit 105 that drives the imaging unit 103, and a first memory 106.

[0015] The irradiation light from the light source device 201 enters the LCB 101 and propagates by repeating total reflection within the LCB 101. The irradiation light (illumination light) that has propagated within the LCB 101 is emitted from the emission end of the LCB 101 disposed within the distal end portion 12 of the insertion portion 11 and irradiates the observation site via the light distribution lens 102. The return light from the irradiated portion forms an optical image at each pixel on the light receiving surface of the imaging unit 103 via the objective lens.

[0016] The imaging unit 103 is disposed within the distal end portion 12 of the insertion portion 11, and a CMOS (Complementary Metal Oxide Semiconductor) image sensor, which is an image sensor of the rolling shutter method, can be used. The imaging unit 103 accumulates the optical image (return light from the biological tissue) formed on each pixel on the light receiving surface as charges corresponding to the amount of light, and generates and outputs R, G, and B image signals. Note that the imaging unit 103 is not limited to a CMOS image sensor, and may be replaced with other types of imaging devices as long as they are based on the rolling shutter method. The signal output from the imaging unit 103 is processed by a scope connector circuit 401 provided in a scope connector 400, as will be described later.

[0017] The processor 200 is a device that integrally includes a signal processing device that processes signals from the endoscope device 100 and a light source device that irradiates a body cavity where natural light does not reach through the endoscope device 100. In another embodiment, the signal processing device and the light source device may be configured separately. The processor 200 includes a light source device 201, a system controller 202, a photometry unit 203, a front-stage signal processing circuit 205, a color conversion circuit 206, a rear-stage signal processing circuit 207, and a second memory 208.

[0018] The processor 200 may include an operation panel (not shown). The configuration of the operation panel has various forms. Specific configurations of the operation panel include, for example, hardware keys for each function mounted on the front surface of the processor 200, a touch panel type GUI (Graphical User Interface), a combination of a hardware key and a GUI, and the like. An operator (surgeon) can perform a mode switching operation described later using the operation panel.

[0019] The photometric unit 203 acquires the luminance information of the image signal obtained by imaging from the gain circuit included in the color conversion circuit 206, compares it with a predetermined appropriate luminance value (for example, the information of the appropriate luminance value can be stored in advance in an internal memory (not shown) of the photometric unit 203), and notifies the system controller 202 of the comparison result (whether the current luminance value is appropriate, high, or low).

[0020] The system controller 202 executes various programs stored in a memory (not shown) and integrally controls the entire endoscope system 1. The system controller 202 uses a control signal to control the operations and timings of various circuits in the processor 200 so that appropriate processing suitable for the endoscope device 100 connected to the processor 200 is performed. Note that the system controller 202 may be connected to the above-described operation panel.

[0021] In addition, the system controller 202 receives the comparison result with the appropriate luminance value from the photometric unit 203, determines whether to maintain the current exposure, increase the exposure (including the level value to increase), or decrease the exposure (including the level value to decrease), and outputs it to the light source device 201 as an exposure control signal.

[0022] Furthermore, the system controller 202 changes each operation of the endoscope system 1 and the parameters for each operation in response to an instruction from an operator input from the operation panel. For example, when the operator selects an observation mode (mode switching operation) using the operation panel, the system controller 202 outputs a mode selection signal for causing the light source corresponding to the observation mode to emit light to the light source device 201. As will be described later, as the light source device 201, for example, a plurality of LEDs (Light Emitting Diodes) that emit light in different wavelength bands can be used (see FIG. 3). When the operator selects an observation mode (for example, a normal observation mode, a special light observation mode, a SatO2 mode, etc.) by operating a mode selection switch provided in the processor 200, for example, the system controller 202 generates a mode selection signal corresponding to the selected mode and supplies this to the light source control unit 2016 of the light source device 201 (see FIG. 3). Based on the mode selection signal, the light source control unit 2016 determines the combination of LEDs to be emitted, their intensities, and the amount of light (for example, the combination of light-emitting LEDs corresponding to the mode selection signal, etc. are stored in advance in an internal memory not shown), and outputs necessary LED control signals to each of the LEDs 2011 to 2015. When each of the LEDs 2011 to 2015 emits light in each wavelength band based on the LED control signal supplied from the light source control unit 2016, each emitted light is synthesized by the cross prism, and irradiation light (synthetic light) is generated.

[0023] Data communication between the endoscope device 100 and the processor 200 may use a wired electrical communication method or an optical wireless communication method.

[0024] As shown in FIG. 2, the endoscope device 100 and the processor 200 are connected via a scope connector 400. The connector 400 includes an LCB that forms a part of the LCB 101 extending from the processor 200 to the endoscope device 100, and a scope connector circuit 401. In this embodiment, the scope connector circuit 401 is provided within the scope connector 400, but it does not necessarily have to be provided inside the scope connector 400. For example, a circuit corresponding to the scope connector circuit 401 may be provided in the connector portion on the processor 200 side or inside the processor 200.

[0025] <Internal configuration example of the light source device 201> FIG. 3 is a diagram showing an internal configuration example of a light source device 201 provided inside the processor 200, for example.

[0026] The light source device 201 includes a green LED 2011 that emits green light, a blue LED 2012 that emits blue light, a red LED 2013 that emits red light, an amber LED 2014 that emits amber light, a UV LED 2015 that emits UV light, a light source control unit 2016 that controls the light emission of each of the LEDs 2011 to 2015, and cross prisms 2017 and 2018.

[0027] When the light source control unit 2016 receives an exposure control signal from the system controller 202, it changes the light emission profile of each LED by controlling the light emission period and the applied current value of each currently emitting LED (the combination of LEDs to be emitted depends on the observation mode), and performs exposure adjustment (light amount adjustment) (see FIGS. 11 and 13 described later). For example, after changing the light emission profile by one step, the light source control unit 2016 determines whether to change the light emission profile again based on the exposure control signal determined by the light measurement result (comparison result with the appropriate luminance value) by the light measurement unit 203 to perform exposure adjustment.

[0028] Also, the light source control unit 2016 determines the combination of LEDs to emit light based on a mode selection signal indicating the observation mode selected by the operator. At the light emission start stage, the light source control unit 2016 controls the light emission of each LED based on, for example, a predetermined light emission profile (default light emission period and drive current value), and then performs the exposure adjustment as described above.

[0029] <Regarding each LED light source> FIG. 4 is a diagram showing the spectra (wavelength characteristics) of each of the LEDs 2011 to 2015. FIG. 5 is a diagram showing the characteristics of the illumination light (light for illuminating the observation site) generated by transmitting each LED through the cross prisms 2017 and 2018.

[0030] The transmission wavelength band of the green LED 2011 is from 540 nm to 575 nm, the peak wavelength is 550 nm, and the full width at half maximum is 30 nm. A phosphor is mounted on the green LED 2011, and by this phosphor, as shown in FIG. 4, light in the transmission wavelength band from about 400 nm to 780 nm is emitted. That is, substantially white light is emitted by the green LED and the phosphor, but this white light is an intermediate product, and as will be described later, the transmission wavelength band is narrowed by the cross prism 2018, and the observation site is irradiated with green light. The transmission wavelength band of the blue LED 2012 is from 460 nm to 490 nm, the peak wavelength is 456 nm, and the full width at half maximum is 21 nm. The transmission wavelength band of the red LED 2013 is from 630 nm to 1000 nm, the peak wavelength is 650 nm, and the full width at half maximum is 20 nm. The transmission wavelength band of the amber LED 2014 is from 600 nm to 615 nm, the peak wavelength is 613 nm, and the full width at half maximum is 19 nm. The transmission wavelength band of the UV LED 2015 is from 385 nm to 425 nm, the peak wavelength is 405 nm, and the full width at half maximum is 14 nm.

[0031] Each light emitted from each of the LEDs 2011 to 2015, including the green LED 2011 equipped with a phosphor (white light, blue light, red light, amber light, UV light as intermediate products), when passing through the cross prisms 2017 and 2018, will become each light with the characteristics shown in Fig. 5 and irradiate the observation site. Specifically, the white light emitted from the green LED 2011 + phosphor will have its transmission wavelength band limited by the cross prism 2018 and become green light with a wavelength of 520 nm to 595 nm. The blue light emitted from the blue LED 2012 will become blue light with a wavelength of 440 nm to 500 nm by the cross prisms 2017 and 2018. Also, the red light emitted from the red LED 2013 will become red light with a wavelength of 620 nm to 630 nm by the cross prisms 2017 and 2018. The amber light emitted from the amber LED 2014 will become amber light with a wavelength of 580 nm to 630 nm by the cross prisms 2017 and 2018. Furthermore, the UV light emitted from the UV LED 2015 will become UV light with a wavelength of 380 nm to 450 nm by the cross prism 2018.

[0032] <Correction of the linearity difference of each LED> When the light source device 201 is composed of a plurality of LEDs, not only the wavelengths of the light emitted from each of the LEDs 2011 to 2015 but also the light distribution (luminance distribution in each direction) may be different (see Fig. 6: Configuration example of a light source using LEDs with different wavelengths), and there is a possibility that the color and light distribution of the emitted light from each of the LEDs 2011 to 2015 may change. Also, depending on the type of LED, when the forward voltage is lowered to reduce the drive current value, there may be a case where the drive current value drops rapidly and the LED stops emitting light, so the drive current value cannot be significantly reduced. To address such a situation, it becomes necessary to dynamically correct the difference in linearity of the emitted light amount / current ratio of each of the LEDs 2011 to 2015 in accordance with the drive current control of each of the LEDs 2011 to 2015.

[0033] However, since the process of dynamically correcting the linearity difference is complex, it is preferable to determine the drive current value in advance so that there is no linearity difference. Therefore, in the present embodiment, a correction table for correcting the linearity of the emitted light amount / current ratio is prepared in advance, and the drive current values of the respective LEDs 2011 to 2015 are determined using this. FIG. 7 is a diagram showing a graph of the emitted light amount / current ratio of each LED. In FIG. 7, only the relationship between two LEDs (LED1 and LED2) is shown as an example, but the same applies when using the five LEDs 2011 to 2015 as shown in the present embodiment. The relationship of the emitted light amount / current ratio of each LED as shown in FIG. 7 can be obtained by measuring each LED in advance. For this reason, as a correction value, a correction table having the reciprocal of the relationship of the emitted light amount / current ratio as a correction parameter is prepared in advance (stored in a memory), and the light source control unit 2016 calculates a corrected drive current value by multiplying the correction parameter corresponding to the desired emitted light amount (the target emitted light amount obtained by exposure adjustment), and drives each LED. By doing so, even when the wavelengths and light distributions of the emitted light of the respective LEDs are different, it becomes possible to appropriately control the linearity of the emitted light amount / current ratio.

[0034] <Configuration Example of Imaging Surface of Image Sensor> FIG. 8 is a diagram showing an effective pixel region and an invalid region of a rolling shutter type image sensor using a CMOS sensor as an example. The CMOS sensor includes an effective pixel region capable of imaging and an invalid region that cannot be imaged. Also, a part (peripheral part) of the effective pixel region is masked and is a region where an image signal cannot be substantially obtained. When imaging is performed using such an image sensor (in the case of global exposure), various phenomena (characteristics) will appear in the captured image. Note that in the present embodiment, the period not displayed on the screen is defined as the global exposure period, but the idea of the present embodiment is not limited to this case.

[0035] <General Dimming Control Process> FIG. 9 is a diagram showing a phenomenon (feature) that appears in an image captured using an image sensor having the imaging surface shown in FIG. 8 when performing general dimming control processing. As shown in FIG. 9a, when pulse light emission occurs during the readout period of lines not shown on the screen, pseudo global exposure can be realized. Also, when pulse light emission occurs as shown in FIG. 9b, the top line of the effective pixel region has less exposure than the other lines by the period from readout to reset, and the top line appears slightly darker, but if the period from readout to reset is sufficiently shorter than the pseudo global exposure period (for example, less than 1%), the darkness is not noticeable. Further, when pulse light emission occurs as shown in FIG. 9c, the upper half of the effective pixel region becomes slightly darker, but the darkness of that region becomes less noticeable as much as the total exposure of each line increases. Thus, there is a feature that as the pulse light emission period extends, the region where the exposure amounts differ expands, but the difference in brightness due to the different exposure amounts becomes less noticeable. Furthermore, as shown in FIG. 9d, when the pulse light emission period is further extended and the previous pulse component increases, the ratio smoothly changes from the lower part to the upper part of the screen. For this reason, artifacts and distortion (undesirable phenomena) become less noticeable.

[0036] FIG. 10 is a diagram showing the operation when performing elongation control (time control only) of the pulse light emission period and the state of the acquired image corresponding thereto. FIG. 10 is a diagram showing the effect of the above-described pulse light emission period elongation in more detail. As shown in FIG. 10A, when the pulse light emission period is changed between adjacent frames, the difference in exposure amount becomes large in the lines where readout is performed before and after the pulse edge, and a phenomenon occurs where horizontal stripes appear to move up and down on the image. On the other hand, as shown in FIG. 10B, when the weak light emission intensity is changed between adjacent frames, the horizontal stripes on the image become less noticeable.

[0037] However, simply extending the exposure period as shown in FIGS. 9 and 10 is not sufficient. This is because when the exposure amount increases, depending on the surrounding environment of the observation location, a white-out phenomenon (where bright parts of the image become white and disappear) may occur in the image. Therefore, it is necessary to avoid the occurrence of distortion and artifacts while maintaining an appropriate exposure amount (where the image is not too dark and does not cause white-out).

[0038] <Improved Dimming Control Processing> FIG. 11 is a diagram showing an overview of the improved dimming control processing according to the present embodiment. Also, FIG. 12 is a diagram for explaining the influence due to the intensity difference between strong emission and weak emission.

[0039] When controlling the light emission operation of the light source so that the intensity of the light (strong emission) during the pseudo-global exposure period is equal to or greater than the intensity of the light (weak emission) during the rolling shutter period, and performing dimming control by adjusting the weak emission intensity, if the weak emission during the rolling shutter period is below a predetermined threshold, it is common to perform dimming control by changing the strong emission intensity or the strong emission period within the range of the pseudo-global exposure period.

[0040] However, as shown in FIG. 12A, when the intensity difference between strong emission and weak emission is large and the ratio (integral value ratio) of the "product of the intensity and period of strong emission (strong emission integral value)" to the "product of the intensity and period of weak emission (weak emission integral value)" is not sufficiently large, an unnatural image like a double exposure of a short-time exposure image and a long-time exposure image is observed, and this phenomenon is particularly likely to cause discomfort to the operator for a moving subject. Also, when the light source is composed of a plurality of LEDs with different wavelengths or different light distributions (such as in the present embodiment), if the emitted light amounts of each LED are not set to a certain ratio, the color and light distribution of the emitted light will change. For this reason, as described above, when controlling the drive current value in the case where there are variations in the linearity of the emission amount / current ratio of each LED, it is necessary to correct the difference in the linearity of the emission amount / current ratio of each LED, which makes the processing complicated.

[0041] Therefore, in order to reduce the complexity of the process, in addition to the correction process for the linearity of the emission light amount / current ratio of each LED by the above-described correction table, when the intensity of weak emission is lower than a predetermined threshold value, any one of the following Processes 1 to 3 is executed. Process 1: A process of extending the strong emission period while reducing the weak emission intensity Process 2: A process of extending the strong emission period while shortening the weak emission period Process 3: A process of increasing the weak emission intensity while shortening the weak emission period

[0042] That is, any of Processes 1 to 3 is a process of sequentially replacing weak emission with strong emission over time while keeping the total emission integrated value (strong emission integrated value + weak emission integrated value) the same before and after the process, so that changes in the brightness of the observation screen, phenomena such as horizontal stripes moving up and down in the image, and dimming (decrease in brightness) of the screen do not occur (reduction control process). By any of these processes, it becomes possible to eliminate the inconvenience that an unnatural image such as double exposure is formed over time, or the color and light distribution of the emitted light change. For example, it becomes possible to change from the state of FIG. 12A (an image with double exposure and horizontal stripes) to the state of FIG. 12E (an image without double exposure, brightness / color unevenness, and horizontal stripes).

[0043] With reference to FIG. 11, Processes 1 to 3 will be specifically described. First, for each of the LEDs 2011 to 2015, the light source control unit 2016 performs dimming control until the exposure reaches an appropriate level, with the pseudo global exposure period being strong emission and the rolling shutter period being weak emission. As shown in FIG. 11(1), when reducing the emission intensity from a state where light is emitted with uniform intensity throughout the entire period (pseudo global exposure period and rolling shutter period), the emission intensity during the rolling shutter period (weak emission period) is reduced. When the rolling shutter period becomes a non-emission state, the pseudo global exposure period is shortened, or the emission intensity during the pseudo global exposure period is reduced. Thus, dimming control is executed. On the other hand, as shown in FIG. 11(2), when increasing the emission intensity, the emission period during the pseudo global exposure period is extended, or the emission intensity during the pseudo global exposure period is increased. When the maximum emission period and emission intensity are reached during the pseudo global exposure period, the emission intensity during the rolling shutter period (weak emission period) is sequentially increased. Thus, dimming control is executed.

[0044] For example, assume that through the dimming control of FIG. 11(1), the appropriate exposure level (e.g., a predetermined appropriate level ±α: α is a margin) is achieved with the emission pattern of P1. If imaging is continuously performed in this state, an image that appears double-exposed as shown in FIG. 12A will be generated. This is because images with different brightnesses are obtained from the image due to strong emission and the image due to weak emission, and they overlap. Therefore, while maintaining the exposure level of the emission pattern P1, the dimming control process is shifted to any one of FIGS. 11(3-1) to (3-3). Here, FIG. 11(3-1) corresponds to the above Process 1, FIG. 11(3-2) corresponds to the above Process 2, and FIG. 11(3-3) corresponds to the above Process 3. In Processes 1 to 3, when the weak emission intensity is below a predetermined threshold value, a process of gradually replacing the weak emission intensity with the strong emission period is performed for each frame. This makes it possible to eliminate the weak emission period without generating horizontal stripes in the captured image, and to avoid the generation of unnatural images and changes in the color and light distribution of light from each light source.

[0045] More specifically, in Process 1 (Fig. 11(3-1)), the value of the light emission intensity × light emission period (weak light emission area) in weak light emission is gradually replaced by the strong light emission period so that the total area (total light emission integration value) of the strong light emission period and the weak light emission period becomes the same, and dimming control is executed by the light emission pattern Q1 (reduction process). For example, for the light emission from the reduction parts 1001 to 1004 in Fig. 11, in the frame where the corresponding reduction process is executed, the area below the top line of the effective pixel area (see Fig. 8) of the image sensor (CMOS sensor) is exposed. Therefore, in this frame, the image part corresponding to the top line becomes dark, but in the next frame, the image signal corresponding to the top line can be acquired.

[0046] Process 2 (Fig. 11(3-2)) and Process 3 (Fig. 11(3-3)) are dimming control processes according to forms (embodiments) different from Process 1. That is, similar to Process 1, in Process 2 and Process 3 as well, the value of the light emission intensity × light emission period (weak light emission area) in weak light emission is gradually replaced by the strong light emission period so that the total area (total light emission integration value) of the strong light emission period and the weak light emission period becomes the same (reduction process). In Process 2, while keeping the weak light emission level constant, the weak light emission period is shortened, and the light emission integration value corresponding to the weak light emission area reduced by the period shortening is gradually assigned to the strong light emission period, thereby extending the strong light emission period. Also, in Process 3, the weak light emission period is shortened, the light emission integration value corresponding to the weak light emission area reduced by the period shortening is assigned to the weak light emission, and the weak light emission intensity is gradually increased, thereby extending the strong light emission period. By Processes 1 to 3, the light emission patterns Q1 to Q3 are executed. Note that the light emission patterns Q1 to Q3 are all the same pattern.

[0047] If the change in light emission due to the dimming control of any one of Processes 1 to 3 (Fig. 11(3-1) to (3-3)) is executed sufficiently slowly, it is possible to avoid generating an unnatural image such as double exposure without giving the operator a sense of discomfort. For example, in the state where the dimming control according to the light emission patterns Q1 to Q3 is being executed, the obtained image is as shown in Fig. 12E, and the double exposure is eliminated.

[0048] When irradiating the subject with light from each light source in the light emission patterns Q1 to Q3, if the subject moves (including camera movement: relative movement) and the exposure level increases and becomes too bright (the exposure level deviates from the appropriate value), the appropriate exposure level is searched for by the process shown in FIG. 11(4-1). For example, when it is determined that the exposure level is appropriate in the light emission profile P2 (when the overly bright exposure level becomes appropriate: as described above, it is determined whether the exposure level is appropriate based on the photometric result by the photometric unit 203), a reduction process (any one of processes 1 to 3) is executed from the state of the light emission profile P2, and the weakly emitting part is reduced to strongly emit light. On the other hand, when emitting light in the state of the light emission profile P2, if the subject moves (including camera movement: relative movement) and the exposure level decreases and becomes too dark (the exposure level deviates from the appropriate value), it is corrected to the appropriate exposure level by the process shown in FIG. 11(4-2) (the process of increasing the weak light emission level shown in FIG. 11(4-2) is sequentially executed until the appropriate exposure level is reached). For example, when the appropriate exposure level is reached with the light emission profile Q4, as shown in FIG. 11(5), a reduction process of any one of processes 1 to 3 is executed from the exposure level state of the light emission profile Q4.

[0049] <Details of the dimming control process and response to sudden movement of the endoscope> FIG. 13 is a diagram showing the dimming control process of FIG. 11 in more detail, and is a diagram for explaining an inappropriate dimming control process example.

[0050] In FIG. 13, for example, when the subject is irradiated with the light of the emission profile P11 and imaged, and it is determined that the image is overexposed (when the exposure level of P11 is inappropriate), the intensity of the weak emission during the rolling shutter period is sequentially decreased (moving sequentially to the right profile from the profile P11). Here, for example, if the appropriate exposure level (an exposure level without overexposure and with good visibility) is achieved with the emission profile P16, subsequently, the above-described reduction process (any one of processes 1 to 3) is executed, and the weak emission period is replaced with the strong emission period. Then, the captured image changes from the state of FIG. 12A (an unnatural image as if double-exposed when irradiating the subject with the emission profile P16) to the state of FIG. 12E, and phenomena such as overexposure and double exposure are also eliminated. If there is no change in the brightness of the captured image (if the change is within a predetermined threshold), the light emission from each of the light sources 2011 to 2015 is performed with the emission profile P66.

[0051] In FIG. 13, the dimming control is performed such that the emission profile sequentially moves to the right from P11 to P16, and subsequently to P66 by the reduction process. However, it is also possible to follow other paths (for example, a path that moves diagonally (but only slowly in the vertical direction), such as P11 → P12 → P13 → P24 → P25 → P36 → P46 → P56 → P66, that is, a path that moves while performing an operation of simultaneously performing dimming control and reduction process). However, it is not possible to execute the dimming control process along the path of emission profile P11 → P22 → P33 → P44 → P55 → P66. This is because only the strong emission period is simply time-controlled, and a phenomenon (reduction of horizontal stripe movement) in which the bright part of the image moves vertically as shown in FIG. 10A appears. In order not to cause this phenomenon, it is necessary to slowly reduce the intensity distribution of the weak emission to the strong emission over time (reduction process).

[0052] For example, when it is necessary to make it brighter from the state of the emission profile P66 in FIG. 13, if the intensity distribution of the strong emission is extended in the time direction, the above-described horizontal stripe movement phenomenon will be caused. Therefore, first, the intensity level of the weak emission is increased. For example, the state is changed from the emission profile P66 to the emission profile P56, and then the dimming control is adjusted to the state of the emission profile P55. However, if the emission profile is rapidly shifted in the vertical direction, it is equivalent to rapidly shifting the profile in the time direction. Therefore, for a sharp movement of the endoscope (image sensor), it is important to move the profile horizontally to adjust to an appropriate exposure level and then perform a reduction process in the vertical direction.

[0053] On the other hand, when it is necessary to make it darker from the state of the emission profile P66, it is not possible to immediately shift to the emission profile P77. For this reason, once the intensity level of the strong emission during the rolling shutter period is lowered to make it the weak emission period and the state of the emission profile P76 is set, and then a reduction process is performed to set it to the state of the emission profile P77.

[0054] Note that after an appropriate exposure level is determined, the dimming control process is executed until the emission profile (Pkk; k = 1, 2, ···, n: n is an integer of 1 or more) in the state where the up arrow and the down arrow meet.

[0055] Also, in FIG. 13, the current value Imin indicates the correction limit of the linearity of the emission amount / current ratio of each of the above-described light sources, or the current value at which each LED goes out. Therefore, in the present embodiment, when performing dimming control to lower the emission intensity of the weak emission, after lowering it to Imin, instead of lowering the emission intensity, the brightness is adjusted by shortening the emission period. Note that in this case, if Imin is sufficiently small with respect to the strong emission intensity × time, no horizontal stripes will be visible.

[0056] <Offset Emission Process> Figures 14, 15, and 16 are diagrams for explaining offset light emission processing during a non-light-emitting period (or a weak light-emitting period where the light emission intensity is so weak that light emission cannot be visually recognized). Figure 14 shows offset light emission by weak pulses. Figure 15 shows offset light emission by weak continuous light. Figure 16 is a diagram showing the differences in captured images that appear depending on the presence or absence of offset light emission when the imaging device rapidly approaches the subject. Here, "weak" means that the light emission intensity is sufficiently lower than the light emission intensity during the strong light-emitting period and does not result in a double-exposure image, but when the subject and the endoscope tip are close, it means that the subject can be imaged with a light emission intensity that allows visual recognition.

[0057] The offset light emission processing is executed separately (in the background of the dimming control processing) from the conventional dimming control processing (such as in Figure 9) and the dimming control processing according to this embodiment (refer to Figures 11 and 13), and is a process of emitting weak offset light during the non-light-emitting period. Weak offset light emission is considered to be in a pulsed light emission form (refer to Figure 14) and a continuous light emission form (refer to Figure 15), but other light emission patterns may also be used. For example, a light emission pattern may be configured by combining pulsed light emission and continuous light emission, or a light emission pattern with an irregular pulse width may be configured. Such weak offset light emission can be regarded as 0 (zero) when the light emission intensity by the dimming control processing (regardless of which dimming control processing) is equal to or greater than a predetermined value. On the other hand, when the light emission intensity by the dimming control processing becomes less than the predetermined value (or when the light emission intensity is zero), the subject will be irradiated only by the offset light emission. As a result, events that would originally occur during the non-light-emitting period cannot be acquired as an image, but with offset light emission, events during the non-light-emitting period can also be captured.

[0058] FIG. 16 is a diagram showing differences in captured images depending on the presence or absence of weak offset light emission (pulsed light, continuous light) when the imaging device approaches the subject rapidly. When there is no weak offset light emission, the captured image of frame Fk is the same image as that of frame Fk-1. On the other hand, when weak offset light emission is performed, in either the case of pulsed light or continuous light, the captured image of frame Fk is clearly different from that of frame Fk-1, and it can be seen that the event when the imaging device approaches the subject rapidly at frame Fk is captured. Also, it can be seen that there is no difference in the captured image of frame Fk+1 depending on the presence or absence of weak offset light emission.

[0059] <Light control processing: flowchart> FIG. 17 is a flowchart for explaining the light control processing according to the present embodiment. The processing of each of the following steps is mainly described with the system controller 202 as the main operating entity, but it is not limited to this, and a control unit (processor) for performing operation control and arithmetic processing may be provided separately and made to execute it. Also, the function of the system controller 202 may be provided in the light source control unit 2016 of the light source device 201. Therefore, the light control (reduction) control processing can be made a part of the overall operation of the endoscope system 1, or can be made a part of the operation of the light source device 201. In the latter case, the light source control unit 2016 becomes the main operating entity of the processing of each step.

[0060] (i) Step 1701 The light source control unit 2016 receives a mode selection signal corresponding to the observation mode selected by the operator from the system controller, and corrects the linearity of the emission light amount / current ratio of each light source (any combination from the green LED 2011 to the UV LED 2015) using the above correction table for each light source to be emitted.

[0061] (ii) Step 1702 The light source control unit 2016 drives each light source with a drive current after linearity correction of the emission light amount / current ratio to cause light emission and generate illumination light, and irradiates this illumination light onto the subject. Note that the light emission profile (the period of strong light emission, the level and period of weak light emission) at this time can be set to a predetermined value (default value), or the light emission profile used in the last operation during the previous endoscope use can also be used.

[0062] (iii) Step 1703 The imaging element (e.g., a CMOS sensor) of the imaging unit 103 detects the reflected light from the subject generated by irradiating the illumination light generated in step 1702 onto the subject (observation site), and transmits an imaging image signal to the processor 200 via the scope connector circuit 401. The photometry unit 203 acquires the luminance information of the current imaging image signal from the gain circuit included in the color conversion circuit 206, compares it with a predetermined appropriate luminance value (e.g., takes a difference value), and passes the comparison result to the system controller 202. Note that in the photometry unit 203, only the luminance information of the current imaging image signal is acquired from the gain circuit, and the comparison with the appropriate luminance value may be executed by another processing unit such as the system controller 202.

[0063] (iv) Step 1704 The system controller 202 compares the comparison result received from the photometry unit 203 (or the system controller 202 may calculate the comparison result (difference value)) with a predetermined threshold value (a threshold value for determining whether the exposure level is appropriate), and determines whether the current exposure level is appropriate. For example, if the comparison result (difference value) is equal to or less than the predetermined threshold value, it can be determined as appropriate. When it is determined that the current exposure level is not appropriate (No in step 1704), the process proceeds to step 1705. On the other hand, when it is determined that the current exposure level is appropriate (Yes in step 1704), the process proceeds to step 1706.

[0064] (v) Step 1705 The system controller 202 changes the current emission profile (information representing the strong emission period, the weak emission period, and the weak emission level) used for irradiating illumination light to the subject.

[0065] For example, when the current exposure level is high and the image is too bright (such as when the image is overexposed), the system controller 202 generates an exposure control signal that adjusts the exposure level by reducing the level of weak emission in the emission profile. Also, when the weak emission level in the current emission profile is zero, the system controller 202 further generates an exposure control signal that adjusts the exposure level by shortening the strong emission period.

[0066] On the other hand, when the current exposure level is low and the image is too dark, the system controller 202 extends the strong emission period when the strong emission period in the emission profile has not reached the pseudo-global period width. Also, when the strong emission period in the current emission profile has reached the pseudo-global period width, the system controller 202 further generates an exposure control signal that adjusts the exposure level by increasing the weak emission level.

[0067] When the emission profile is changed, the process proceeds to step 1702. Here, the emission profile is gradually changed step by step to adjust to the appropriate exposure level. However, for example, information (such as a table) indicating the relationship between the difference value (comparison value) between the luminance value of the captured image and the appropriate luminance value (luminance value corresponding to the appropriate exposure level) and the change width of the emission profile (information indicating how many steps the emission profile is changed) may be stored in an internal memory (not shown) of the system controller 202, and an appropriate emission profile may be obtained directly from the above difference value (comparison value).

[0068] (vi) Step 1706 The system controller 202 determines whether it is necessary to execute a reduction control process for a light emission profile presenting an appropriate exposure level. The determination of the necessity of the reduction control process can be based on, for example, whether there has been a fact of changing the light emission profile after the execution of the previous reduction control process. If there is no change in the light emission profile, since the subject is being imaged at the currently appropriate exposure level (appropriate brightness) and appropriate dimming (dimming without moiré), there is no need to execute the reduction control process. On the other hand, if there is a change in the light emission profile, since there may be a weak light emission component to be replaced with strong light emission, the necessity of the reduction control process is determined based on whether there is a weak light emission period in the changed light emission profile.

[0069] When it is determined that it is necessary to execute the reduction control process (YES in step 1706), the process proceeds to step 1707. On the other hand, when it is determined that there is no need to execute the reduction control process (NO in step 1706), the process proceeds to step 1708.

[0070] (vii) Step 1707 The system controller 202 executes any one of processes 1 to 3 in FIG. 11, and gradually replaces the weak light emission component with the strong light emission component while keeping the light emission integral value (the area of the light emission profile) represented by "strong light emission period × strong light emission intensity + weak light emission period × weak light emission intensity" constant. For example, the reduction control process is executed over a time period of about 1 second (the time for several tens of frames), and is changed to the light emission profile in FIG. 10 represented by Q1 to Q3, for example. Thereby, it becomes possible to obtain an image (FIG. 12E) without such occurrences from an image (FIG. 12A) that is double-exposed and has moiré when only the exposure level is made appropriate.

[0071] (viii) Step 1708 The light source control unit 2016 receives information on the emission profile to be applied from the system controller 202, and based on the received emission profile and the mode selection signal, causes any one of the LEDs 2011 to 2015 to emit light to generate illumination light and irradiate the subject. Further, the imaging element (CMOS sensor) of the imaging unit 103 detects reflected light from the subject irradiated with the illumination light, generates an imaging image signal, and transmits it to the processor. Further, the processor 200 executes predetermined image processing on the imaging image signal to generate display image data, and displays the display image data on the screen of the monitor (display device) 300.

[0072] (ix) Step 1709 The system controller 202 determines whether an instruction to end the observation, such as the end of imaging or the turning off of the illumination light, has been input from the operator. When an instruction to end the observation is input (YES in step 1709), the dimming control process ends. When an instruction to end the observation is not input (the instruction is not detected) (NO in step 1709), the process proceeds to step 1703, and the determination and monitoring of whether the current exposure level is appropriate, the dimming control process, etc. are continuously performed. The imaging unit 103 is installed at the tip 12 of the endoscope device 100 and moves within the body cavity of the subject. Therefore, since the subject (observation site) approaches or moves away, the exposure level may change. Therefore, the operation of the light source device 201 is controlled so as to constantly monitor the luminance level of the imaging image and maintain an appropriate exposure level.

[0073] <Effects of the present embodiment> According to this embodiment, it becomes possible to image a subject while ensuring a sufficient amount of light while avoiding rolling shutter distortion and artifacts. Further, even if a change in the pulse emission period extends to the rolling shutter period, it is possible to make the vertical movement of the horizontal stripes less noticeable. Furthermore, when a plurality of LEDs are used simultaneously as the light source, if the emission intensity changes, the ratio of the emission light amount / current of each LED changes due to differences in the linearity of the emission light amount / current ratio of each LED, and the ratio of the light amounts of the LEDs changes, causing changes in hue and color. However, according to this embodiment, it is possible to return the emission intensity to the original level in a short time and solve such problems.

[0074] <Specific Matters of the Present Disclosure> (1) Specific Matter 1 An endoscope system that inserts an endoscope into an observation target and acquires an image of a subject, comprising: a plurality of semiconductor light-emitting elements that emit light having different wavelength bands; a rolling shutter type imaging element that irradiates the subject with illumination light, detects reflected light from the subject, and generates an image signal; a processor that processes the image signal to generate an image of the subject and displays it on a monitor; a main control unit that generates a control signal for controlling the emission profile of the plurality of semiconductor light-emitting elements based on the image signal; a light source control unit that receives the control signal from the main control unit and drives the plurality of semiconductor light-emitting elements with a drive signal corresponding to the emission profile; a photometric unit that detects the exposure level of the imaging element, wherein the emission profile is composed of a two-stage emission period including only a strong emission period in which light is emitted at a predetermined light intensity and a weak emission period in which light is emitted at a light intensity weaker than the predetermined light intensity, the main control unit executes a reduction control process of replacing the emission amount in the weak emission period with the emission amount in the strong emission period while maintaining the total emission amount in the strong emission period and the emission amount in the weak emission period constant, and determines the emission profile, In an endoscope system, before and after the reduction control process, the light intensity during the strong light emission period is always constant, and only the light intensity during the weak light emission period before and after the reduction control process changes. (2) Specific matter 2 In specific matter 1, The main control unit generates the control signal for extending the strong light emission period by reducing the light intensity during the weak light emission period while keeping the length of the weak light emission period constant until the light intensity during the weak light emission period becomes zero, and replacing the light emission amount corresponding to the reduction with the light emission amount during the strong light emission period. (3) Specific matter 3 In specific matter 1, The main control unit generates the control signal for extending the strong light emission period by shortening the weak light emission period while keeping the light intensity during the weak light emission period constant, and replacing the light emission amount corresponding to the shortening with the light emission amount during the strong light emission period. (4) Specific matter 4 In specific matter 1, The main control unit generates the control signal for extending the strong light emission period by shortening the length of the weak light emission period and increasing the light intensity during the weak light emission period by the light emission amount corresponding to the shortening, and replacing the light emission amount during the weak light emission period with the light emission amount during the strong light emission period. (5) Specific matter 5 In specific matter 1, The main control unit executes the reduction control process when the exposure level of the imaging element by the photometric unit deviates from a predetermined appropriate value.

Explanation of reference signs

[0075] 1 Endoscope system 100 Endoscope device 103 Imaging unit 200 Processor 201 Light source device 2011 Green LED 2012 Blue LED 2013 Red LED 2014 Amber LED 2015 UV LED 2016 Light source control unit 2017, 2018 Cross prism 202 System controller 203 Photometry unit 300 Monitor

Claims

1. An endoscope system for inserting an endoscope into an object to be observed and acquiring an image of the object, A plurality of semiconductor light emitting elements each emitting light of a different wavelength band; a rolling shutter type image sensor that irradiates the subject with illumination light and detects reflected light from the subject to generate an image signal; a processor that processes the image signal to generate an image of the subject and displays the image on a monitor; a main control unit that generates a control signal for controlling a light emission profile of the plurality of semiconductor light emitting elements based on the image signal; a light source control unit that receives the control signal from the main control unit and drives the semiconductor light emitting elements with drive signals corresponding to the light emission profile; a photometry unit that detects an exposure level of the image sensor; the light emission profile is composed of two stages of light emission periods including only a strong light emission period in which light is emitted at a predetermined light intensity and a weak light emission period in which light is emitted at a light intensity weaker than the predetermined light intensity, the main control unit determines the light emission profile by performing a reduction control process in which the amount of light emitted during the weak light emission period is replaced with the amount of light emitted during the strong light emission period while maintaining a total amount of light emitted during the strong light emission period and the weak light emission period constant; An endoscope system in which the light intensity in the strong light emission period is always constant before and after the reduction control process, and only the light intensity in the weak light emission period changes before and after the reduction control process.

2. In claim 1, the main control unit reduces the light intensity during the weak light emission period while keeping the length of the weak light emission period constant until the light intensity during the weak light emission period becomes zero, and replaces the amount of light emission corresponding to the decrease with the amount of light emission during the strong light emission period, thereby generating the control signal for extending the strong light emission period.

3. In claim 1, the main control unit generates the control signal for shortening the weak light emission period while keeping the light intensity of the weak light emission period constant, and for extending the strong light emission period by replacing the amount of light emission of the shortened weak light emission period with the amount of light emission of the strong light emission period.

4. In claim 1, the main control unit generates the control signal for shortening the length of the weak light emission period and allocating the shortened light emission amount to the weak light emission period to increase the light intensity of the weak light emission period, thereby replacing the light emission amount of the weak light emission period with the light emission amount of the strong light emission period, thereby extending the strong light emission period.

Images