Endoscopic device, endoscopic system, and image processing method

Abstract

It ensures a wide dynamic range for light metering and suppresses image flicker near the maximum aperture setting of the electronic shutter. [Solution] The endoscope device includes a processor that processes the image signal output from the image sensor. The processor acquires exposure correction information based on a target value for exposure time and the amount of exposure control actually applied to the image sensor, and adjusts the gain of the image signal acquired from the image sensor based on the exposure correction information.

Description

Technical Field

[0001] The present invention relates to an endoscope apparatus, an endoscope system, and an image processing method for acquiring and processing an image signal from an image pickup device having an electronic shutter function.

Background Art

[0002] Conventionally, an aperture, an electronic shutter, a gain, etc. have been used for dimming control. The aperture changes the brightness of an optical image for dimming control. The electronic shutter changes the exposure time, that is, the charge accumulation time, for dimming control. The gain performs dimming control by amplifying an image signal.

[0003] For example, Japanese Patent Laid-Open No. 2018-14680 describes a technique for achieving a wide dimming dynamic range in a digital still camera. In the technique of this publication, the following dimming control is performed while the subject changes from a dark state to a bright state.

[0004] When the subject is darkest, the aperture is opened, the gain is maximized, and then the degree to which the electronic shutter is slower than the frame rate is controlled. When the subject is slightly dark, the aperture is opened, the electronic shutter is set to the frame rate, and then the gain is controlled between maximum and 0. When the subject is slightly bright, the electronic shutter is set to the frame rate, the gain is set to 0, and then the aperture is controlled. When the subject is bright, the gain is set to 0, the aperture is closed to the maximum, and then the degree to which the electronic shutter is faster than the frame rate is controlled.

[0005] Thus, in the above publication, as the subject changes from a dark state to a bright state, dimming control is performed in the order of "electronic shutter control" → "gain control" → "aperture control" → "electronic shutter control". That is, it is a technique for changing which of the electronic shutter, gain, and aperture is used according to the brightness of the subject when performing dimming control using only any one of the electronic shutter, gain, and aperture. Therefore, the above publication does not describe controlling using two or more simultaneously.

[0006] Incidentally, exposure control using an electronic shutter generally involves discrete exposure times. In an electronic shutter, the shading state is achieved, for example, by sweeping away unwanted charges from pixels. The exposure time is the time from when sweeping stops and charge accumulation begins until the charge of the pixel is read out. In an image sensor where pixels are arranged in lines, exposure control by the electronic shutter is performed by determining how many lines are exposed and how many lines are left shaded at a given time.

[0007] In other words, exposure control using an electronic shutter quantizes the exposure time into units of one line, resulting in quantization errors relative to the desired brightness. The proportion of quantization error in the image signal increases as the exposure time decreases. That is, the effect of quantization error is greater when there are fewer exposed lines than when there are more lines in the exposed state (exposure lines). The state in which the number of exposed lines is minimized (the state in which the number of light-blocking lines is maximized) is called aperture fully closed.

[0008] Endoscopic systems observe subjects with a wide dynamic range of light control, from the dark areas deep within the lumen to the mucosa at close range. In particular, when observing the surface of the mucosa at close range (i.e., when the subject is bright) in video, the electronic shutter may be controlled near its maximum aperture. Also, when a laser is irradiated onto a lesion, for example, the amount of light in the field of view may increase rapidly. In this case, the electronic shutter may be controlled near its maximum aperture to suppress halation in the image. However, as mentioned above, the accuracy of light control becomes rough near the maximum aperture, causing the brightness of the video to fluctuate from frame to frame, resulting in flickering and hindering image observation.

[0009] On the other hand, one method to suppress flicker is to increase the number of exposure lines corresponding to the full aperture of the electronic shutter, thereby lowering the upper limit of the proportion of quantization error in the image signal. For example, if the number of exposure lines corresponding to the full aperture of the electronic shutter is set to 20 lines, even if a quantization error of 1 line occurs, the brightness variation from frame to frame can be suppressed to 5% or less. However, using this method results in a narrower dynamic range for dimming.

[0010] The present invention has been made in view of the above circumstances, and aims to provide an endoscope device, an endoscope system, and an image processing method that can ensure a wide dynamic range for dimming and suppress image flickering near the maximum aperture of the electronic shutter. [Overview of the Initiative] [Means for solving the problem]

[0011] An endoscope according to one aspect of the present invention includes a processor that processes an image signal output from an image sensor, the processor is configured to acquire exposure correction information based on a target value for exposure time and an exposure control amount actually applied to the image sensor, and to adjust the gain of the image signal acquired from the image sensor based on the exposure correction information.

[0012] An endoscope system according to one aspect of the present invention comprises an endoscope and a processor that processes an image signal output from an image sensor provided in the endoscope, wherein the processor is configured to acquire exposure correction information based on a target value for exposure time and an exposure control amount actually applied to the image sensor, and to adjust the gain of the image signal acquired from the image sensor based on the exposure correction information.

[0013] An image processing method according to one aspect of the present invention is an image processing method for processing an image signal output from an image sensor provided in an endoscope, comprising the steps of: acquiring exposure correction information based on a target value for exposure time and an exposure control amount actually applied to the image sensor; and adjusting the gain of the image signal acquired from the image sensor based on the exposure correction information. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows the external appearance of the endoscope system in the first embodiment of the present invention. [Figure 2] This block diagram shows the electrical configuration of the endoscope system in the first embodiment described above. [Figure 3] This block diagram shows an example of the configuration of the endoscope control processing device in the first embodiment described above. [Figure 4] This is a timing chart showing the operation of the endoscope system during video imaging in the first embodiment described above. [Figure 5] This flowchart shows the operation of the endoscope control processing device during video imaging in the first embodiment described above. [Figure 6] This flowchart shows the electronic shutter control process by the endoscope control processing device in the first embodiment described above. [Figure 7] This figure shows an example of a table that shows the correspondence between the automatic dimming brightness level stored in memory and the dimming detection target value in the first embodiment described above. [Figure 8] This graph shows the relationship between exposure time and electronic shutter control value in the first embodiment described above. [Figure 9] This is a diagram showing the relationship between exposure time and electronic shutter control value in the first embodiment described above. [Figure 10] In the first embodiment described above, this is a flowchart showing the flicker correction process in step S3 of Figure 5. [Figure 11]In the above first embodiment, it is a flowchart showing the process of calculating the exposure time ratio in step S22 of FIG. 10. [Figure 12] In the above first embodiment, it is a graph showing the relationship between the brightness control value and the electronic shutter control value. [Figure 13] In the above first embodiment, it is a graph showing an enlarged view near the maximum value of the brightness control value in FIG. 12 and showing it together with the exposure time. [Figure 14] In the above first embodiment, it is a graph showing an enlarged view near the minimum value of the brightness control value in FIG. 12 and showing it together with the exposure time. [Figure 15] In the above first embodiment, it is a graph showing the ratio of the change amount of the exposure time when the electronic shutter control value changes by one line to the exposure time. [Figure 16] In the above first embodiment, it is a graph showing an enlarged view near the minimum value of the brightness control value, showing how the discretely changing exposure time is substantially corrected by the gain. [Figure 17] It is a block diagram showing the electrical configuration of the endoscope system in the second embodiment of the present invention. [Figure 18] In the above second embodiment, it is a flowchart showing the process of shake correction in step S3 of FIG. 5. [Figure 19] In Related Art 1, it is a diagram showing a configuration example of an endoscope control processing device that also serves as a light source device. [Figure 20] In Related Art 1, it is a block diagram showing a configuration example of a control circuit, a light source drive circuit, and a light source. [Figure 21] In Related Art 1, it is a chart showing an example of the light emission mode of an endoscope control processing device that also serves as a light source device. [Figure 22] In Related Art 1, it is a diagram showing a more detailed configuration example of a light source drive circuit and a light source. [Figure 23] In Related Art 1, it is a graph showing an example of the range where the drive current detected by the current detection circuit is normal and the range where it is abnormal in each light emission mode. [Figure 24]This is a timing chart showing an example of the timing at which the current sensing circuit performs current sensing in related technology 1. [Figure 25] This figure shows an example of applying a single current sensing circuit to a configuration in which multiple light-emitting elements are each driven by multiple light-emitting element drivers, as described in Related Technology 1. [Figure 26] This figure shows an example of applying a single current sensing circuit to a configuration in which multiple light-emitting elements are driven by a single light-emitting element driver, as described in Related Technology 1. [Figure 27] This figure shows an example of applying a single current sensing circuit to a configuration in which one light-emitting element is driven by multiple light-emitting element drivers, as described in Related Technology 1. [Figure 28] This figure shows an example configuration of an endoscope control processing device that also functions as a light source device, as described in Related Technology 2. [Figure 29] This graph shows a first example of conversion parameters for converting the current control value I to pulse brightness L, as described in related technology 2. [Figure 30] This graph shows a second example of conversion parameters for converting the current control value I to pulse brightness L, as described in related technology 2. [Figure 31] Related Technology 2 is a timing chart used to explain exposure unevenness between frames when the image sensor uses a global shutter system. [Figure 32] Related Technology 2 is a timing chart used to explain exposure unevenness between lines when the image sensor uses a rolling shutter system. [Figure 33] This figure shows an example configuration of an endoscope control processing device that also functions as a light source device, as described in Related Technology 3. [Figure 34] Related Technology 3 is a graph showing an example of a function formula for calculating color correction values ​​from dimming control values. [Figure 35] This figure shows an example configuration of an endoscope processor that supplies power for the audio input circuit using an isolator element, as described in related technology 4. [Figure 36]This graph illustrates how, in related technology 4, a high-gain amplification line is used when the input sound pressure to the microphone is below a threshold, and a low-gain amplification line is used when it is above the threshold. [Figure 37] This figure shows an example configuration of an endoscope processor equipped with multiple amplification mechanisms with different amplification levels, as described in related technology 4. [Modes for carrying out the invention]

[0015] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to the embodiments described below.

[0016] In the drawings, identical or corresponding elements are appropriately denoted by the same reference numeral. Furthermore, it should be noted that the drawings are schematic, and the relationships between the lengths of each element, the ratios of the lengths of each element, and the quantities of each element within a single drawing may differ from reality for the sake of simplicity. Additionally, there may be differences in the relationships and ratios of lengths between multiple drawings. [First Embodiment]

[0017] Figures 1 to 16 show a first embodiment of the present invention. Figure 1 is a diagram showing the external appearance of the endoscope system 1.

[0018] The endoscope system 1 comprises an endoscope 2, an endoscope control processing device 3, and a monitor 4.

[0019] The endoscope 2 comprises an insertion section 5, an operating section 6, and a universal cable 7.

[0020] The insertion part 5 is a long, slender part that is inserted into the body of the subject. While the subject into which the insertion part 5 is inserted is assumed to be a human body as an example, it is not limited to humans; it may be an animal or other living organism, or even a machine or building.

[0021] The insertion portion 5 comprises, in order from the tip end to the base end, a tip component 5a, a curved portion 5b, and a flexible tube portion 5c.

[0022] The endoscope 2 is configured as an electronic endoscope, and an imaging system is provided within the tip component 5a. The imaging system includes an objective lens 14 (see Figure 19) that forms an optical image of the subject, and an image sensor 11 (see Figure 2, etc.) that converts the optical image formed by the objective lens 14 into an electrical signal. The image sensor 11 generates an image signal frame by frame and transmits it to the endoscope control processing device 3.

[0023] The image sensor 11 is not limited to being located at the tip component 5a of the insertion section 5. For example, a configuration in which a relay optical system is provided in the insertion section 5 and the operation section 6, and the camera head is attached to the operation section 6, may be adopted. In this configuration, the optical image formed by the objective lens 14 is transmitted by the relay optical system and captured by the image sensor 11 in the camera head.

[0024] The curved portion 5b is a part that can be bent in two directions, for example, up and down, or in four directions, up, down, left, and right. The curved portion 5b is located on the base end side of the tip component 5a. When the curved portion 5b is bent, the direction of the tip component 5a changes, and the direction of illumination light and the observation direction of the imaging system change. The curved portion 5b is also bent to improve the insertability of the insertion portion 5 within the subject.

[0025] The flexible tube section 5c is a tubular section that is flexible. The flexible tube section 5c is located on the proximal end side of the curved section 5b. Here, we give an example in which the endoscope 2 is a flexible endoscope having a flexible tube section 5c. However, the endoscope 2 may also be a rigid endoscope in which the part corresponding to the flexible tube section 5c is rigid. Furthermore, the endoscope 2 may be entirely disposable, entirely reprocessed and reused, or partially disposable.

[0026] The operating section 6 is located on the base end side of the insertion section 5. The operating section 6 comprises a gripping section 6a, a bending operation knob 6b, an operation button 6c, and a treatment instrument insertion port 6d.

[0027] The gripping section 6a is the part where the user grasps the endoscope 2 with their palm.

[0028] The bending operation knob 6b is an operating device for bending the bending section 5b. The bending operation knob 6b is operated using, for example, the thumb of the hand that is gripping the gripping section 6a. The bending operation knob 6b is connected to the bending section 5b by a bending wire. When the bending operation knob 6b is operated, the bending wire is pulled, and the bending section 5b is bent.

[0029] The operation buttons 6c include several buttons for operating the endoscope 2. Some examples of operation buttons 6c are the air / water insufflation button, the suction button, and buttons related to imaging.

[0030] The instrument insertion port 6d is the proximal end opening of the instrument channel, which is located inside the device from the insertion section 5 to the operating section 6. When an instrument is inserted through the instrument insertion port 6d, the tip of the instrument protrudes from the tip end opening of the instrument channel provided in the tip component 5a. In this state, various procedures are performed on the subject using the instrument.

[0031] The universal cable 7 extends from, for example, the proximal end side of the operating unit 6 and is connected to the endoscope control processing device 3.

[0032] The endoscope control processing unit 3 receives image signals from the image sensor 11 in frame units. The endoscope control processing unit 3 processes the acquired image signals and outputs the processed image signals to the monitor 4. The endoscope control processing unit 3 also controls the endoscope 2. In other words, the endoscope control processing unit 3 serves as both an endoscope image processing unit that processes image signals acquired by the endoscope 2 and an endoscope control unit that controls the endoscope 2.

[0033] Monitor 4 is a display device that receives image signals and displays endoscopic images. Note that Monitor 4 does not need to be a configuration specific to the endoscope system 1; a separate Monitor 4 may be connected to the endoscope control processing device 3.

[0034] Figure 2 is a block diagram showing the electrical configuration of the endoscope system 1.

[0035] As described above, the endoscope 2 is equipped with an image sensor 11. The image sensor 11 is a two-dimensional image sensor in which multiple pixels are arranged in a two-dimensional manner in the matrix direction. The arrangement of pixels in the row direction is called a line. Examples of image sensors 11 include CCD (Charge Coupled Device) imagers and CMOS (Complementary Metal Oxide Semiconductor) imagers.

[0036] The image sensor 11 generates and outputs an image signal by photoelectric conversion of the optical image of the subject. The image sensor 11 is equipped with an electronic shutter function that electronically controls the exposure time. The image sensor 11 achieves a state in which no charge accumulates, a so-called light-shielding state, by discharging (sweeping out) the charge from the pixels. The electronic shutter stops the discharge of charge from the pixels, and charge accumulation begins. The time from when charge accumulation begins until the accumulated charge is read out is the exposure time. In the electronic shutter function, the discharge of charge from pixels to achieve the light-shielding state is performed, for example, on a line-by-line basis.

[0037] The exposure period in one frame is defined as the longest period that can be used for exposure within the frame period. In the case of an image sensor 11 where the exposure period and readout period are separate and exposure cannot be performed during the readout period, the exposure period becomes the period obtained by subtracting the readout period, etc., from the frame period.

[0038] The exposure period in one frame divided by the total number of lines on the image sensor 11 is called the specific time. The electronic shutter function of the image sensor 11 can control the exposure time in integer multiples of the specific time, using the specific time as the unit. Therefore, the exposure time is controlled discretely by the electronic shutter function.

[0039] Specifically, let Tep be the exposure period in one frame, and N be the total number of lines on the image sensor 11. Then, the specific time unit in which the image sensor 11 can control the exposure time is (Tep / N). Note that the exposure period Tep is determined according to the specifications of the image sensor 11.

[0040] The endoscope control processing device 3 includes a first image processing circuit 21, a flicker correction circuit 22, a second image processing circuit 23, a dimming detection calculation circuit 24, an electronic shutter control value calculation circuit 25, an exposure time ratio calculation circuit 26, a flicker correction gain calculation circuit 27, an electronic shutter control value transmission circuit 28, and a user interface 29.

[0041] Figure 3 is a block diagram showing an example configuration of the endoscope control processing device 3.

[0042] Each circuit of the endoscope control processing device 3 shown in Figure 2 may be composed of electronic circuits. Alternatively, all or part of each circuit of the endoscope control processing device 3 shown in Figure 2 may be composed of a processor 30a and memory 30b as shown in Figure 3. The processor 30a is composed of an ASIC (Application Specific Integrated Circuit) including a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), etc. The memory 30b stores processing programs that realize the functions of each circuit. The functions of each circuit in the endoscope control processing device 3 are realized when the processor 30a reads and executes the processing programs stored in the memory 30b.

[0043] The first image processing circuit 21 receives the image signal output from the image sensor 11 and performs first image processing. The first image processing includes, for example, amplification, demosaicing, and noise reduction.

[0044] The flicker correction circuit 22 amplifies the image signal output from the first image processing circuit 21 using the flicker correction gain received from the flicker correction gain calculation circuit 27, and performs image processing to correct the flicker of the frame image.

[0045] The second image processing circuit 23 performs a second image processing on the image signal output from the flicker correction circuit 22. This second image processing may include, for example, shading correction, white balance correction, contrast correction, gamma conversion, and format conversion. The second image processing circuit 23 may also superimpose various types of information, such as text information and guide information, onto the image signal. The second image processing circuit 23 then outputs the image signal, converted to a format compatible with the monitor 4, to the monitor 4.

[0046] While several examples of the first and second image processing steps were shown above, the processes included in the first and second image processing steps are not limited to these. Furthermore, the examples above do not limit which processes are included in the first image processing step performed before the flicker correction circuit 22, nor which other processes are included in the second image processing step performed after the flicker correction circuit 22.

[0047] The dimming detection calculation circuit 24 performs dimming detection on the image signal output from the first image processing circuit 21 and obtains a dimming detection value.

[0048] The dimming detection calculation circuit 24 calculates the target exposure time for the second frame, which is time-later than the first frame, based on the dimming detection value of the image signal acquired in the first frame. In Figure 4, which will be described later, the first frame is specifically referred to as frame n and the second frame as frame (n+2). The target exposure time for the second frame is calculated so that the dimming detection value of the image signal acquired in the second frame becomes the target dimming detection value. Here, the automatic dimming brightness level that gives the target dimming detection value can be set by the user from the user interface 29, as will be described later.

[0049] The electronic shutter control value calculation circuit 25 receives the exposure time target value from the dimming detection calculation circuit 24 and calculates the electronic shutter control value to be applied to the second frame. The exposure time target value is calculated as an arbitrary, non-discrete value. In contrast, the electronic shutter control value is calculated as a value representing a discrete exposure time (for example, an integer multiple of a specific time, as described above).

[0050] The exposure time ratio calculation circuit 26 obtains the target exposure time value and the electronic shutter control value from the electronic shutter control value calculation circuit 25. The exposure time ratio calculation circuit 26 calculates the ratio of the electronic shutter control value to the target exposure time value as the exposure time ratio. In Figure 2, the exposure time ratio calculation circuit 26 obtains the target exposure time value from the electronic shutter control value calculation circuit 25, but the present invention is not limited to this. The exposure time ratio calculation circuit 26 may also receive the target exposure time value from the dimming detection calculation circuit 24.

[0051] The flicker correction gain calculation circuit 27 obtains the exposure time ratio from the exposure time ratio calculation circuit 26, calculates the reciprocal of the exposure time ratio as the flicker correction gain, and transmits it to the flicker correction circuit 22.

[0052] In the above description, the exposure time ratio was calculated as the ratio of the electronic shutter control value to the target exposure time value. However, it is not limited to this method; it may also be calculated as the ratio of the target exposure time value to the electronic shutter control value. In this case, the flicker correction gain calculation circuit 27 only needs to transmit the exposure time ratio as the flicker correction gain to the flicker correction circuit 22. Furthermore, in this case, the exposure time ratio calculation circuit 26 can also function as the flicker correction gain calculation circuit 27, simplifying the configuration.

[0053] The electronic shutter control value transmission circuit 28 receives the electronic shutter control value from the electronic shutter control value calculation circuit 25 and transmits it to the image sensor 11. The endoscope control processing device 3 also transmits the operating clock and power to the image sensor 11. The image sensor 11 takes images so that the subject is exposed according to the electronic shutter control value and generates an image signal. The image sensor 11 captures the video sequentially frame by frame as described above. As a result, the image signals for each frame of the video are sequentially output from the endoscope 2 to the endoscope control processing device 3.

[0054] The user interface 29 is an interface for the user to perform various settings for the endoscope system 1. For example, the user interface 29 can be used to set the on / off status of exposure time control by the electronic shutter. In this embodiment, the explanation will be based on the assumption that exposure time control by the electronic shutter is set to on. The user can also set the automatic dimming brightness level using the user interface 29.

[0055] Monitor 4 receives the image signal output from the second image processing circuit 23 and displays the endoscopic image.

[0056] Figure 4 is a timing chart showing the operation of the endoscope system 1 during video acquisition. In Figure 4, the frame number is indicated by (A), the acquisition operation by (B), the image processing operation by (C), the electronic shutter control value calculation operation by (D), the flicker correction gain calculation operation by (E), the electronic shutter control value application operation by (F), and the flicker correction gain application operation by (G).

[0057] As shown in Figure 4(A), the time interval t(n) to t(n+1) corresponds to the duration of frame n, the time interval t(n+1) to t(n+2) corresponds to the duration of frame (n+1), the time interval t(n+2) to t(n+3) corresponds to the duration of frame (n+2), and the time interval t(n+3) to t(n+4) corresponds to the duration of frame (n+3).

[0058] As shown in Figure 4(B), in each frame, exposure is performed in the preceding stage of the frame period, and the image signal is read out in the succeeding stage of the frame period.

[0059] In frame n, the image sensor 11 performs exposure of the image in frame n and reads out the image signal. Below, a series of processes performed by the endoscope control processing device 3 based on the image signal acquired in frame n will be described. In Figure 4, the corresponding blocks are hatched and connected by arrows.

[0060] In frame (n+1), the first image processing circuit 21 and the second image processing circuit 23 perform image processing (C) on the image signal of frame n. Furthermore, the dimming detection calculation circuit 24 and the electronic shutter control value calculation circuit 25 calculate the electronic shutter control value (D) for capturing frame (n+2). Finally, the exposure time ratio calculation circuit 26 and the flicker correction gain calculation circuit 27 calculate the flicker correction gain (E) for the image acquired in frame (n+2).

[0061] In frame (n+2), the electronic shutter control value transmission circuit 28 transmits an electronic shutter control value to the image sensor 11 (F). The transmitted value is an electronic shutter control value for frame (n+2) to be applied to the image sensor 11. The image sensor 11 exposes the image of frame (n+2) using the received electronic shutter control value and reads out the generated image signal (B).

[0062] In frame (n+3), the first image processing circuit 21 processes the image signal acquired in frame (n+2) (C). The flicker correction circuit 22 applies a flicker correction gain for frame (n+2) to the processed image signal to correct the image flicker (G). The image signal, from which the flicker has been corrected, is further processed by the second image processing circuit 23 (C).

[0063] Figure 5 is a flowchart showing the operation of the endoscope control processing device 3 during video imaging.

[0064] When video imaging begins, the endoscope control processing device 3 waits to receive image signals frame by frame from the image sensor 11 of the endoscope 2 (step S1).

[0065] Upon receiving an image signal, the first image processing circuit 21 performs a first image processing on the image signal (step S2).

[0066] The flicker correction circuit 22 performs flicker correction on the image signal output from the first image processing circuit 21 (step S3).

[0067] The second image processing circuit 23 performs a second image processing on the image signal output from the flicker correction circuit 22 (step S4).

[0068] The second image processing circuit 23 outputs the processed image signal to the monitor 4 (step S5).

[0069] The endoscope control processing device 3 determines whether or not the user has given an instruction to end video imaging (step S6). If there is no instruction to end, it returns to step S1 and waits to receive the image signal for the next frame. If there is an instruction to end, it terminates this imaging process.

[0070] Figure 6 is a flowchart showing the electronic shutter control process performed by the endoscope control processing device 3. The process in Figure 6 is performed in parallel with the process in Figure 5, which performs image processing for display on the monitor 4.

[0071] When the process shown in Figure 6 is executed from the main process (not shown), the dimming detection calculation circuit 24 waits to receive the flicker-corrected image signal from the flicker correction circuit 22 (step S11).

[0072] Upon receiving an image signal, the dimming detection calculation circuit 24 performs dimming detection based on the image signal and calculates the target exposure time (step S12).

[0073] Figure 7 shows an example of a table that illustrates the correspondence between the automatic dimming brightness level stored in memory 30b and the dimming detection target value.

[0074] The automatic dimming brightness level, set by the user via the user interface 29, is determined by how much brighter (level = +1, +2, +3, ...) or how much darker (level = -1, -2, -3, ...) the image brightness should be compared to the standard brightness (level = 0). The correspondence between the automatic dimming brightness level and the dimming detection target value is pre-stored in memory 30b in the form of a table, graph, function, etc.

[0075] Here, the correspondence between the automatic dimming brightness level and the dimming detection target value differs depending on whether the endoscope 2 connected to the endoscope control processing device 3 is a flexible endoscope or a type that uses a camera head, etc. For this reason, the memory 30b stores the correspondence for each model of endoscope 2. Figure 7 shows an example of the correspondence when endoscope 2 is a flexible endoscope.

[0076] The dimming detection calculation circuit 24 refers to the memory 30b and determines the dimming detection target value based on the automatic dimming brightness level set by the user interface 29. If the automatic dimming brightness level is not set via the user interface 29, the dimming detection target value is automatically set to the value corresponding to the standard brightness (level = 0).

[0077] Furthermore, the dimming detection calculation circuit 24 extracts, for example, a luminance signal from the image signal received from the flicker correction circuit 22, and calculates the average value of the luminance signal values ​​within the dimming area set for the entire image or a part of the image (or a weighted average, peak value of the luminance histogram, median, etc.) as the dimming detection value.

[0078] The dimming detection calculation circuit 24 calculates a first detection ratio by, for example, dividing the dimming detection value by the dimming detection target value, and calculates the exposure time target value by dividing the exposure time target value in the first frame (for example, frame n in Figure 4) by the first detection ratio. The exposure time target value is expressed in units of seconds (sec), for example.

[0079] For example, suppose the target exposure time for the first frame is 0.001 seconds and the first detection ratio is 0.5. Then, the target exposure time is calculated as 0.001 / 0.5 = 0.002 seconds.

[0080] Alternatively, the dimming detection calculation circuit 24 may calculate the second detection ratio by dividing the dimming detection target value by the dimming detection value, instead of using the first detection ratio. In this case, the dimming detection calculation circuit 24 can calculate the exposure time target value by multiplying the exposure time target value in the first frame by the second detection ratio.

[0081] The electronic shutter control value calculation circuit 25 receives the target exposure time value from the dimming detection calculation circuit 24 and calculates the electronic shutter control value (step S13). The relationship between the electronic shutter control value and the exposure time is predetermined according to the specifications of the image sensor 11, as explained below.

[0082] Figure 8 is a graph showing the relationship between exposure time and electronic shutter control value. Figure 9 is a chart showing the relationship between exposure time and electronic shutter control value.

[0083] In this embodiment, including Figures 8 and 9, we will explain using numerical examples where the total number of lines N of the image sensor 11 is 1120 and the frame rate is 60 (fps). However, it goes without saying that we are not limited to these numerical examples.

[0084] As mentioned above, the exposure time is controlled by integer multiples of a specific time (Tep / N). Therefore, when n is a positive integer less than or equal to N, the exposure time is expressed as n × (Tep / N).

[0085] Let Tmax be the maximum exposure time. The maximum exposure time Tmax is the value when n=N, and Tmax = N × (Tep / N) = Tep. In other words, the maximum exposure time Tmax occurs when all lines are exposed and the exposure time matches the exposure period Tep.

[0086] Furthermore, the minimum exposure time is defined as Tmin. The minimum exposure time Tmin is not necessarily the time (Tep / N) corresponding to n=1. Depending on the specifications of the image sensor 11, it is set to a certain integer value of 1 or greater, such as n=4. If the minimum value of n that can be taken according to the specifications of the image sensor 11 is n(min), then the minimum exposure time Tmin is given by Tmin = n(min) × (Tep / N). The minimum exposure time Tmin corresponds to the state where the electronic shutter aperture is fully closed (the state in which the number of exposure lines is minimized: the state in which the number of light-shielding lines is maximized).

[0087] Therefore, the exposure time n × (Tep / N) is controlled within the range of n = n(min), {n(min) + 1}, ..., (N-1), N.

[0088] The endoscope control processing device 3 uses ES = (Nn) as the electronic shutter control value ES for the image sensor 11. This electronic shutter control value ES represents how many lines out of the total number of lines N are in a light-shielded state. Therefore, as shown in Figures 8 and 9, the electronic shutter control value (the number of lines in a light-shielded state) increases as the exposure time decreases. However, it is not limited to this, and the device may be configured to represent the electronic shutter control value as the number of lines in an exposed state.

[0089] The exposure time target value Tgt is defined as the value received by the electronic shutter control value calculation circuit 25 from the dimming detection calculation circuit 24. Furthermore, the minimum value of the electronic shutter control value ES is defined as ESmin, and the maximum value of the electronic shutter control value ES is defined as ESmax.

[0090] The minimum value of the electronic shutter control value ES, ESmin, corresponds to the maximum exposure time Tmax mentioned above, and ESmin = 0.

[0091] The maximum value of the electronic shutter control value ES, ESmax, corresponds to the minimum exposure time Tmin mentioned above, and ESmax = {Nn(min)}.

[0092] As described above, the exposure time for each line in the image sensor 11 is controlled by an integer multiple of a specific time (Tep / N). Therefore, the electronic shutter control value calculation circuit 25 determines the electronic shutter control value ES that can achieve the "exposure time closest to the target exposure time value Tgt" as follows.

[0093] In other words, the electronic shutter control value calculation circuit 25 uses the ceiling function ceil(x) to calculate the electronic shutter control value ES from the exposure time target value Tgt using the following equation 1. Here, the ceiling function ceil(x) is, as is well known, a function that gives the smallest integer greater than or equal to x for a real number x (i.e., a function that rounds up the decimal part). ES=ceil(ESmax-α×{Tgt-Tmin}) …(Equation 1)

[0094] Here, -α represents the slope of the graph shown in Figure 8, and α is given by the following equation 2. α=(ESmax-ESmin) / (Tmax-Tmin) …(Formula 2)

[0095] Note that here we used the ceiling function ceil(x) to round up the decimal part, but you can also use the floor function floor(x) to round down the decimal part instead.

[0096] However, as mentioned above, Tmax = Tepp and ESmin = 0. Therefore, equation 2 can be expressed as equation 2' below. α=ESmax / (Tep-Tmin) …(Equation 2′)

[0097] In the example above, where n(min)=4, the maximum value of the electronic shutter control value ES is ESmax=(N-4), and the minimum exposure time is Tmin=4×(Tep / N).

[0098] Thus, using Tmin, ESmax, Tep, and α determined according to the specifications of the image sensor 11, the electronic shutter control value ES can be calculated from the exposure time target value Tgt using Equation 1.

[0099] Furthermore, by subtracting the electronic shutter control value ES, which represents the number of lines that remain in the light-shielding state, from the total number of lines N, we can obtain the number of lines that remain exposed, n = (N - ES).

[0100] As shown in the numerical example above, if the target exposure time Tgt calculated by the dimming detection calculation circuit 24 is 0.002 (sec), then 986 (lines) is calculated as the electronic shutter control value ES that can achieve an exposure time that does not exceed 0.002 (sec) but is closest to 0.002 (sec) (see the arrow column in Figure 9).

[0101] Therefore, the electronic shutter control value calculation circuit 25 may use the table shown in Figure 9 to determine the electronic shutter control value ES instead of Equation 1. The formulas, etc., that give the table shown in Figure 9 or the graph shown in Figure 8 may be stored in memory 30b beforehand and read by the electronic shutter control value calculation circuit 25.

[0102] The electronic shutter control value transmission circuit 28 receives the electronic shutter control value ES from the electronic shutter control value calculation circuit 25 and transmits it to the image sensor 11 (step S14). As a result, the image sensor 11 performs electronic shutter control based on the electronic shutter control value ES in the next frame (for example, frame (n+2) in Figure 4).

[0103] After completing step S14, the program returns to the main process (not shown in the diagram).

[0104] Figure 10 is a flowchart showing the flicker correction process in step S3 of Figure 5.

[0105] When the endoscope control processing device 3 enters the flicker correction process in step S3 of Figure 5, the process shown in Figure 10 is executed. Then, the exposure time ratio calculation circuit 26 waits to receive the exposure time target value Tgt and the electronic shutter control value ES from the electronic shutter control value calculation circuit 25 (step S21).

[0106] The exposure time ratio calculation circuit 26 calculates the exposure time ratio er after receiving the target exposure time value Tgt and the electronic shutter control value ES (step S22).

[0107] Figure 11 is a flowchart showing the process for calculating the exposure time ratio in step S22 of Figure 10.

[0108] The exposure time ratio calculation circuit 26 calculates the actual exposure time Texp from the electronic shutter control value ES and the above-mentioned α using the following equation 3 (step S31). Texp = -ES / α + Tmax …(Equation 3)

[0109] As mentioned above, Tmax = Tep, so equation 3 can be expressed as equation 3' below. Texp = -ES / α + Tep …(Equation 3')

[0110] Furthermore, the exposure time ratio calculation circuit 26 is not limited to calculating the actual exposure time Texp using a mathematical formula; it may also calculate it by referring to the diagram in Figure 9 (or the graph in Figure 8).

[0111] As shown in the numerical example above, when the electronic shutter control value ES = 986 (line), the actual exposure time Texp is 0.001990134 (sec), as can be seen by referring to the arrow column in Figure 9, for example.

[0112] The exposure time ratio calculation circuit 26 calculates the exposure time ratio er from the target exposure time Tgt and the actual exposure time Texp using the following equation 4 (step S32). er = Texp / Tgt …(Equation 4)

[0113] As shown in the numerical example above, if the target exposure time Tgt is 0.002 (sec) and the actual exposure time Texp is 0.001990134 (sec), the exposure time ratio is: er = 0.001990134 / 0.002 = 0.995067 Therefore, the actual exposure time Texp is 0.4933% less than the target exposure time Tgt.

[0114] The exposure time ratio calculation circuit 26 transmits the calculated exposure time ratio er to the flicker correction gain calculation circuit 27 (step S33). After step S33 is completed, the process returns to Figure 10.

[0115] In the process shown in Figure 10, the flicker correction gain calculation circuit 27 calculates the reciprocal of the exposure time ratio er received from the exposure time ratio calculation circuit 26 as the flicker correction gain Ga, as shown in the following equation 5 (step S23). Ga = 1 / er …(Equation 5)

[0116] As shown in the numerical example above, when the exposure time ratio er is 0.995067, the flicker correction gain Ga is: Ga = 1 / 0.995067 = 1.004957 This is the result.

[0117] The flicker correction gain calculation circuit 27 transmits the calculated flicker correction gain Ga to the flicker correction circuit 22.

[0118] Figure 12 is a graph showing the relationship between the brightness control value and the electronic shutter control value ES. The brightness control value is a value proportional to the exposure time used to control the brightness of the image. Figure 12 shows an example where the brightness control value is represented by a 16-bit (0 to 65535) numerical value.

[0119] As described above, the electronic shutter control value ES represents how many lines out of the total number of lines N (in the example above, N = 1120 lines) are in a light-shielding state. Therefore, the smaller the electronic shutter control value ES, the larger the brightness control value. When the electronic shutter control value ES is at its minimum value ESmin = 0, the brightness control value takes the maximum value of 65535.

[0120] Furthermore, as mentioned above, depending on the specifications of the image sensor 11, the minimum value of n, n(min), is set to, for example, n(min)=4. In this case, for a total number of lines N=1120, the maximum value of the electronic shutter control value ES is ESmax=1116. The brightness control value at this time is, for example, 118.

[0121] Figure 13 is a graph showing an enlarged view of the area around the maximum brightness control value in Figure 12, along with the exposure time. In Figure 13 (and Figures 14 and 16 described later), the solid line graph represents the electronic shutter control value ES, and the dashed line graph represents the exposure time.

[0122] As mentioned above, exposure time is controlled by the electronic shutter, for example, on a line-by-line basis. Therefore, the exposure time changes discretely (in steps).

[0123] Figure 14 is a graph showing a magnified view of the area around the minimum brightness control value in Figure 12, along with the exposure time.

[0124] The discrete variation in exposure time is the same as in Figure 13. However, the amount of change in image brightness when the electronic shutter control value ES changes by one line is significantly different between the state near the open aperture of the electronic shutter shown in Figure 13 and the state near the fully closed aperture shown in Figure 14. This point will be explained with reference to Figure 15.

[0125] Figure 15 is a graph showing the ratio of the change in exposure time to the total exposure time when the electronic shutter control value ES changes by one line. In Figure 15, the area enclosed by the double-dotted line represents the vicinity of the electronic shutter aperture being fully open, and the area enclosed by the single-dotted line represents the rest of the exposure.

[0126] When the electronic shutter control value ES is near its minimum value ESmin, for example, when the electronic shutter control value ES is 120 (lines), the number of lines exposed will be 1000 (lines). If the electronic shutter control value ES is changed by +1 line (or -1 line), the number of lines exposed will change by -1 line (or +1 line), becoming 999 (lines) (or 1001 (lines)). The absolute value of the rate of change in brightness at this time is 1 / 1000 = 0.1 (%).

[0127] In contrast, when the electronic shutter control value ES is near its maximum value ESmax, for example, when the electronic shutter control value ES is 1115 (lines), the number of lines exposed is 5 (lines). If the electronic shutter control value ES is changed by +1 (or -1) lines, the number of lines exposed changes by -1 (or +1) lines, becoming 4 (or 6) (lines). The absolute value of the rate of change in brightness at this time is 1 / 5 = 20 (%).

[0128] Therefore, the variation in image brightness between frames due to discrete exposure time control is negligible except near the maximum aperture of the electronic shutter. On the other hand, near the maximum aperture of the electronic shutter, even a change of just one line in the electronic shutter control value ES causes a large change in image brightness, resulting in image flickering between different frames.

[0129] In Figure 15, the dotted line shows an example of an electronic shutter control value ES that determines whether or not image flicker due to discrete exposure time control is noticeable. The electronic shutter control value ESa is the total number of lines N minus "100 / target dimming accuracy" lines. For example, if the total number of lines N = 1120 and the target dimming accuracy = 5%, the electronic shutter control value ESa = 1120 - 100 / 5 = 1100.

[0130] However, it goes without saying that the above values ​​may be changed as appropriate depending on the total number of lines N of the image sensor 11, specifications, etc.

[0131] Figure 16 is a magnified graph showing the area around the minimum value of the brightness control, illustrating how discretely varying exposure times are effectively corrected by the gain. In Figure 16, the dotted line graph represents the effective exposure time after flicker correction.

[0132] The flicker correction circuit 22 receives the flicker correction gain Ga from the flicker correction gain calculation circuit 27, amplifies the image signal received from the first image processing circuit 21 by the flicker correction gain Ga, and performs flicker correction (step S24).

[0133] In the numerical example above, the image signal before flicker correction was 0.4933% underexposed, but by amplifying the image signal with a flicker correction gain of Ga = 1.004957, the underexposure is eliminated.

[0134] The flicker correction circuit 22 performs image processing to correct flicker, which changes the stepped exposure time shown by the dashed line in Figure 16 to a smooth linear exposure time shown by the dotted line, thereby correcting image flicker between frames caused by discrete exposure time control.

[0135] In this first embodiment, the error in dimming control (quantization error of exposure time) caused by the discrete control of exposure time by the electronic shutter is corrected by gain adjustment in image processing. This suppresses the flickering of image brightness that occurs near the maximum aperture of the electronic shutter, making the image displayed on the monitor 4 easier to see. Consequently, the diagnosis and treatment performed by the physician observing the monitor 4 will not be hindered by image flickering.

[0136] Furthermore, since gain adjustment is performed using the exposure time ratio er calculated based on the target exposure time Tgt and the actual exposure time Texp, variations in brightness from frame to frame can be suppressed with good accuracy.

[0137] Because image flicker is suppressed near the maximum aperture of the electronic shutter, the electronic shutter control value ES corresponding to the maximum aperture of the electronic shutter can be set to a value close to the total number of lines N. This makes it possible to ensure a wide dynamic range for exposure control. [Second Embodiment]

[0138] Figures 17 and 18 show a second embodiment of the present invention. In the second embodiment, the same reference numerals are used for parts that are the same as in the first embodiment, and their descriptions are omitted as appropriate. In the second embodiment, the differences from the first embodiment will be mainly described.

[0139] Figure 17 is a block diagram showing the electrical configuration of the endoscope system 1.

[0140] The endoscope control processing device 3 of this embodiment has a flicker correction on / off control circuit 31 added to the configuration of the first embodiment shown in Figure 2.

[0141] The flicker correction on / off control circuit 31 receives the electronic shutter control value ES from the electronic shutter control value calculation circuit 25, generates a control signal, and transmits it to the flicker correction circuit 22.

[0142] Figure 18 is a flowchart showing the flicker correction process in step S3 of Figure 5.

[0143] When the endoscope control processing device 3 enters the flicker correction process in step S3 of Figure 5, the process shown in Figure 18 is executed.

[0144] Then, the exposure time ratio calculation circuit 26 waits to receive the exposure time target value Tgt and the electronic shutter control value ES from the electronic shutter control value calculation circuit 25, and the flicker correction on / off control circuit 31 waits to receive the electronic shutter control value ES from the electronic shutter control value calculation circuit 25 (step S21').

[0145] When the flicker correction on / off control circuit 31 receives the electronic shutter control value ES, it determines whether the electronic shutter control value ES is greater than or equal to the threshold value ESth (step S25).

[0146] Here, the threshold ESth is an electronic shutter control value ES that serves as a guideline for determining whether the variation in brightness from frame to frame, caused by errors in dimming control resulting from the discrete control of exposure time by the electronic shutter, is noticeable as image flicker. The threshold ESth is preset according to the specifications of the image sensor 11, etc.

[0147] The threshold value ESth is specifically set to a value that is less than or equal to the value obtained by subtracting "100 / target dimming accuracy" lines from the total number of lines N, as shown as the electronic shutter control value ESa in Figure 15.

[0148] The flicker correction on / off control circuit 31 generates a flicker correction on control signal and transmits it to the flicker correction circuit 22 when the electronic shutter control value ES is greater than or equal to the threshold ESth. When the flicker correction circuit 22 receives the flicker correction on control signal, it performs flicker correction in step S24 and returns to the process shown in Figure 5.

[0149] Furthermore, if the flicker correction on / off control circuit 31 is less than the threshold ESth, it generates a flicker correction off control signal and sends it to the flicker correction circuit 22. When the flicker correction circuit 22 receives the flicker correction off control signal, it returns to the process shown in Figure 5 without performing the flicker correction in step S24.

[0150] Therefore, when the electronic shutter control value ES is less than the threshold ESth, dimming is performed by controlling the electronic shutter using the electronic shutter control value ES calculated from dimming detection, without the need for gain adjustment related to flicker correction.

[0151] If the electronic shutter control value ES is less than the threshold ESth, the processes in steps S22 and S23 may be performed, and only the process in step S24 may be skipped. However, in the process shown in Figure 18, not only steps S24 but also steps S22 and S23 are skipped. In this case, the flicker correction on / off control circuit 31 should send a processing stop control signal to the exposure time ratio calculation circuit 26 and the flicker correction gain calculation circuit 27. This makes it possible to more effectively reduce power consumption when flicker correction is not performed.

[0152] As explained with numerical examples in the first embodiment, when the number of lines exposed is 5, the absolute value of the change in brightness when the electronic shutter control value ES is changed by 1 line is 20%. In contrast, when the number of lines exposed is 1000, the absolute value of the change in brightness is 0.1%.

[0153] Therefore, when the electronic shutter control value ES is below the threshold ESth and is far from the aperture's maximum setting, the brightness fluctuation from frame to frame is minimal and hardly noticeable as image flicker.

[0154] The second embodiment provides almost the same effects as the first embodiment described above. Furthermore, in the second embodiment, signal amplification is not performed in scenes where image flickering is not noticeable, thus avoiding a decrease in image quality due to the amplification of noise components. [Related Technology 1]

[0155] Incidentally, the endoscope control processing device 3 may also be configured to function as a light source device, for example. Figure 19 shows an example configuration of the endoscope control processing device 3 that also functions as a light source device, as described in related technology 1.

[0156] In addition to the image sensor 11 described above, the endoscope 2 includes a light guide 12, an illumination lens 13, an objective lens 14, and a drive circuit 15.

[0157] The light guide 12 transmits illumination light supplied from the endoscope control processing device 3 to the tip of the endoscope 2. The light guide 12 is composed of, for example, a fiber bundle made of bundled optical fibers.

[0158] The illumination lens 13 is located at the tip of the endoscope 2. The illumination lens 13 irradiates the subject with illumination light transmitted by the light guide 12.

[0159] The objective lens 14 forms an optical image of the subject illuminated by the illumination light onto the image sensor 11.

[0160] The drive circuit 15 is electrically connected to the image sensor 11, transmits a drive signal to the image sensor 11, receives the image signal acquired by the image sensor 11, and transmits it to the endoscope control processing device 3.

[0161] The endoscope control processing device 3 comprises a first image processing circuit 21, a second image processing circuit 23, a control circuit 34, a light source drive circuit 35, a light source 36, and a multiplexer 37.

[0162] For example, the first image processing circuit 21, the second image processing circuit 23, and the control circuit 34 are composed of a processor 30a and a memory 30b as shown in Figure 3. However, as mentioned above, some or all of these may be composed of electronic circuits.

[0163] The first image processing circuit 21 performs basic image processing on the image signal received from the endoscope 2.

[0164] The second image processing circuit 23 generates an image signal for observation on the monitor 4 from the image signal processed by the first image processing circuit 21 and outputs it to the monitor 4.

[0165] The control circuit 34 controls various parts of the endoscope control processing device 3, including the first image processing circuit 21 and the light source drive circuit 35, and also controls the endoscope 2.

[0166] The control circuit 34 includes a brightness detection circuit 38 and a dimming control circuit 39. The brightness detection circuit 38 receives an image signal from the first image processing circuit 21, performs dimming detection, and detects the brightness of the image. The dimming control circuit 39 performs dimming control based on the brightness of the image detected by the brightness detection circuit 38. The dimming control by the dimming control circuit 39 includes controlling the electronic shutter of the image sensor 11 via the drive circuit 15 and controlling the light source 36 via the light source drive circuit 35.

[0167] The light source drive circuit 35 controls the light source 36 based on commands from the dimming control circuit 39, causing the light source 36 to emit light.

[0168] The light source 36 includes a white light source that emits white illumination light, and, if necessary, a special light source that emits special light. Examples of special light sources include a laser light source for irradiating kidney stones, an excitation light source for causing fluorescence to be emitted from the subject, and a light source for performing NBI (Narrow Band Imaging).

[0169] Specifically, the light source 36 includes multiple light-emitting elements, such as LEDs (Light Emitting Diodes), that emit light at different wavelengths. For example, the LEDs constituting a white light source include white LEDs. Alternatively, the LEDs constituting a white light source may include LEDs that emit green (G) light, LEDs that emit red (R) light, and LEDs that emit blue (B) light. Special light sources include light-emitting elements such as LEDs and LDs (Laser Diodes) that emit light at other wavelengths.

[0170] The light source 36 emits light from one or more light-emitting elements under the control of the light source driving circuit 35.

[0171] The multiplexer 37 combines multiple lights of different wavelengths emitted from the light source 36 and supplies them to the endoscope 2 as illumination light.

[0172] The light source device may be provided separately from the endoscope control processing device 3.

[0173] Figure 20 is a block diagram showing an example configuration of the control circuit 34, light source driving circuit 35, and light source 36 in related technology 1.

[0174] The control circuit 34 is configured, for example, with an FPGA 34a acting as a processor 30a.

[0175] The light source driving circuit 35 comprises a light-emitting element driver 35a, an on / off and feedback circuit 35b, and a current detection circuit 35c. A more specific configuration of the light source driving circuit 35 will be described later with reference to Figure 22.

[0176] The light source 36 includes a light-emitting element 36a.

[0177] Figure 21 is a diagram showing an example of the light emission mode of the endoscope control processing device 3, which also serves as a light source device, in related technology 1.

[0178] The endoscope control processing device 3 emits illumination light in multiple emission modes. In the example shown in Figure 21, the endoscope control processing device 3 can be set to emit illumination light in strobe mode, WLI (White Light Imaging) mode, and NBI mode.

[0179] As described above, the light-emitting element 36a comprises multiple light-emitting elements (first light-emitting element, second light-emitting element, etc.). For example, focusing on the first light-emitting element, as shown in Figure 21, the first light-emitting element is driven with different current drive ranges and different time-division periods for each of the three light-emitting modes.

[0180] Furthermore, different light-emitting elements, such as the first and second light-emitting elements, have different current-driven ranges and time-division periods for light emission in the three light-emitting modes.

[0181] Figure 22 shows a more detailed example of the configuration of the light source driving circuit 35 and the light source 36 in related technology 1.

[0182] The light-emitting element driver 35a is configured, for example, as an integrated circuit (IC) and includes multiple output terminals and multiple input terminals.

[0183] The drive output terminal DRIVE_OUT of the light-emitting element driver 35a is connected to the anode of the light-emitting element 36a, which is configured as an LED, for example. The cathode of the light-emitting element 36a is connected to the on / off and feedback circuit 35b.

[0184] The on / off and feedback circuit 35b comprises three transistors Q1 to Q3 that function as switches, and six resistors R1 to R6.

[0185] Transistors Q1 to Q3 are each configured as, for example, enhancement-type N-channel power MOSFETs (Metal-Oxide-Semiconductor Field-Effect Transistors). Enhancement-type MOSFETs do not have drain current flow when the gate-source voltage is zero.

[0186] Transistors Q1 to Q3 are arranged in parallel, and their drains are connected to the cathodes of the light-emitting element 36a. The gate of transistor Q1 is connected to the control output terminal PWM_OUT1 of the light-emitting element driver 35a, the gate of transistor Q2 is connected to the control output terminal PWM_OUT2 of the light-emitting element driver 35a, and the gate of transistor Q3 is connected to the control output terminal PWM_OUT3 of the light-emitting element driver 35a.

[0187] One end of resistors R1 and R2, which are arranged in parallel, is connected to the source of transistor Q1. The other ends of resistors R1 and R2 are connected to ground.

[0188] One end of resistors R3 and R4, which are arranged in parallel, is connected to the source of transistor Q2. The other ends of resistors R3 and R4 are connected to the source of transistor Q1 and one end of resistors R1 and R2.

[0189] One end of resistors R5 and R6, which are arranged in parallel, is connected to the source of transistor Q3. The other ends of resistors R5 and R6 are connected to the source of transistor Q2 and one end of resistors R3 and R4.

[0190] Furthermore, arranging multiple resistors in parallel has the advantage of lowering the temperature of each resistor, thus providing a thermal management solution.

[0191] The positive input terminal LED_ISP of the light-emitting element driver 35a is connected between the source of transistor Q3 and one end of resistors R5 and R6.

[0192] A current sensing circuit 35c is connected to both ends of resistors R1 and R2, which are arranged in parallel. The other ends of resistors R1 and R2 are connected to the negative input terminal LED_ISN of the light-emitting element driver 35a. The light-emitting element driver 35a provides feedback control to each output based on the inputs from the positive input terminal LED_ISP and the negative input terminal LED_ISN.

[0193] In the configuration shown in Figure 22, when the light-emitting element 36a emits light, the three transistors Q1 to Q3 are controlled selectively, and only one of them is turned on.

[0194] In strobe mode, transistor Q1 is controlled to be turned on, and transistors Q2 and Q3 are turned off. The current path in strobe mode is shown by a solid line indicated by P1. The light-emitting element driver 35a controls the amount of light emitted in strobe mode by controlling the current value of the current path P1 and the time that transistor Q1 is turned on (the time that current flows).

[0195] In addition, in WLI emission mode, transistor Q2 is controlled to be turned on, and transistors Q1 and Q3 are controlled to be turned off. The current path in WLI emission mode is shown by a dashed line indicated by P2. At this time, the light-emitting element driver 35a controls the amount of light emitted in WLI emission mode by controlling the current value of the current path P2 and controlling transistor Q2 with PWM (Pulse-Width Modulation).

[0196] In NBI emission mode, transistor Q3 is controlled to be turned on, and transistors Q1 and Q2 are controlled to be turned off. The current path in NBI emission mode is shown by a dashed line with P3. At this time, the light-emitting element driver 35a controls the amount of light emitted in NBI emission mode by controlling the current value of the current path P3 and by PWM control of transistor Q3.

[0197] Let I1 be the current in current path P1, and V1 be the input voltage of the current detection circuit 35c in strobe mode. Let I2 be the current in current path P2, and V2 be the input voltage of the current detection circuit 35c in WLI mode. Let I3 be the current in current path P3, and V3 be the input voltage of the current detection circuit 35c in NBI mode. Here, the minimum value of each current in its driving range will be denoted as min, and the maximum value as max. The resistance values ​​of each resistor R1 to R6 will be represented using the same symbols R1 to R6. Furthermore, the equivalent resistance value when two resistors are connected in parallel will be represented using the symbol " / / ".

[0198] Specifically, the combined resistance values ​​(R1 / / R2), (R3 / / R4), and (R5 / / R6) are the values ​​shown in equations 6 to 8 below. (R1 / / R2)=R1×R2 / (R1+R2)…(Formula 6) (R3 / / R4)=R3×R4 / (R3+R4)…(Formula 7) (R5 / / R6)=R5×R6 / (R5+R6)…(Formula 8)

[0199] The current sensing circuit 35c detects the current value from the voltage value across the combined resistance (R1 / / R2). The voltage ranges V1, V2, and V3 that the current sensing circuit 35c detects as being within the normal range are as shown in equations 9 to 11 below. (Formula 9) TIFF2026110706000002.tif20128 (Formula 10) TIFF2026110706000003.tif17146 (Formula 11) TIFF2026110706000004.tif48166

[0200] In this way, different current paths P1 to P3 are provided for each light emission mode, and the current paths P1 to P3 are merged to perform current detection. At this time, by appropriately designing the combined resistance values ​​(R1 / / R2), (R3 / / R4), and (R5 / / R6) to correspond to the current drive range of each light emission mode, it becomes possible to detect the current in each light emission mode with a single current detection circuit 35c.

[0201] A commonly considered configuration involves providing multiple current sensing circuits for each of the multiple current paths. In this case, the current sensing circuits can be designed to match the current drive range for each light emission mode, but one current sensing circuit is required for each current path.

[0202] Compared to a typical configuration where a separate current detection circuit is provided for each light emission mode, the configuration shown in Figure 22 requires only one current detection circuit 35c, thus simplifying the circuit configuration.

[0203] Figure 23 is a graph showing examples of normal and abnormal ranges for the drive current detected by the current sensing circuit 35c in each light emission mode in related technology 1.

[0204] In Figure 23, the current values ​​corresponding to the normal voltage range shown in the table and equations 9 to 11 of Figure 21 are indicated by arrows, and the current values ​​corresponding to the abnormal range are shown by hatched bar graphs. In the illustrated example, for example, the median value of the normal range of current values ​​decreases in the order of strobe emission mode (current path P1), WLI emission mode (current path P2), and NBI emission mode (current path P3).

[0205] The FPGA 34a of the control circuit 34 receives the current value detected by the current sensing circuit 35c and determines whether the received current value is within the normal range or the abnormal range for the current light emission mode.

[0206] For example, if a component malfunctions while operating in WLI emission mode and the light-emitting element 36a is driven in strobe emission mode, a strong light will be shone on the subject. FPGA 34a compares the current value detected by the current sensing circuit 35c with the normal drive current range to detect abnormal conditions such as operation in an unintended emission mode. When FPGA 34a detects an abnormal condition, it controls the light-emitting element driver 35a to stop the emission by, for example, turning off all transistors Q1 to Q3.

[0207] Furthermore, the FPGA34a may also perform control to switch to another safe driving state (or safe driving path) when it detects an abnormal condition. For example, when an abnormal condition is detected, it may switch to the NBI lighting mode, which has the lowest median value of the normal range of current values ​​among the three lighting modes.

[0208] Figure 24 is a timing chart showing an example of the timing at which the current sensing circuit 35c performs current sensing in related technology 1.

[0209] The current detection circuit 35c receives, for example, a current detection timing signal transmitted from the control circuit 34 and performs current detection at the timing of the reception. The current detection timing signal is, for example, a pulse signal that becomes high level at a constant period t2. The current detection circuit 35c performs current detection when the pulse signal becomes high level.

[0210] The drive instruction signals in Figure 24 are signals output from the control output terminals PWM_OUT1 to PWM_OUT3 of the light-emitting element driver 35a. Let t1 be the period during which the drive instruction signals become high level and transistors Q1 to Q3 are turned on. Then, the period t2 of the current detection timing signal is set to be shorter than the period t1 (t1 > t2). As a result, current detection by the current detection circuit 35c is reliably performed at least once within the period t1 during which transistors Q1 to Q3 are turned on.

[0211] By detecting the current within period t1, it is possible to confirm whether the light-emitting element 36a is being driven with the intended current. For example, in WLI flash mode, it is possible to confirm whether the light-emitting element 36a is operating within the current drive range of strobe flash mode.

[0212] Furthermore, by performing current detection outside of period t1, it is possible to check whether current is flowing to the light-emitting element 36a at times when current drive is not instructed, i.e., at unintended times. For example, in strobe flash mode, it is possible to check whether the light-emitting element 36a is not emitting light at the timing when the strobe flash is turned off.

[0213] Since the duration t1 differs depending on the emission mode, the period t2 is also set according to the emission mode. Alternatively, the period t2 may be set to be shorter than the duration t1 in any given emission mode.

[0214] Figure 25 shows an example in which one current sensing circuit 35c is applied to a configuration in related technology 1 in which multiple light-emitting elements 36a1, 36a2, 36a3, ... are each driven by multiple light-emitting element drivers 35a1, 35a2, 35a3, ....

[0215] The light-emitting element driver 35a comprises multiple light-emitting element drivers 35a1, 35a2, 35a3, ... In addition, resistor elements Ra1, Ra2, Ra3, ... are arranged on the current paths P1 to P3 as shown below.

[0216] Note that the number of light-emitting elements, light-emitting element drivers, and resistors may be two or more than three, but the following explanation will describe an example with three elements. Figures 26 and 27 below will also be explained in accordance with the example with three elements.

[0217] In the first light emission mode (for example, strobe light emission mode), the light-emitting element 36a1 is driven by the light-emitting element driver 35a1, and the current path P1 passes through the light-emitting element 36a1 and the resistor Ra1.

[0218] In the second light emission mode (for example, the WLI light emission mode), the light-emitting element 36a2 is driven by the light-emitting element driver 35a2, and the current path P2 passes through the light-emitting element 36a2 and the resistors Ra2 and Ra1.

[0219] In the third light emission mode (for example, the NBI light emission mode), the light-emitting element 36a3 is driven by the light-emitting element driver 35a3, and the current path P3 passes through the light-emitting element 36a3 and the resistors Ra3, Ra2, and Ra1. As described above, the current paths P1 to P3 are selected alternately.

[0220] Current paths P1 to P3 merge at the resistor Ra1. The current sensing circuit 35c is configured to measure the current value based on the voltage across the resistor Ra1.

[0221] Therefore, even in the configuration shown in Figure 25, by appropriately setting the resistance values ​​of the resistor elements Ra1, Ra2, and Ra3, it becomes possible to detect the current values ​​flowing through multiple current paths P1 to P3 with a single current sensing circuit 35c.

[0222] Figure 26 shows an example in which one current sensing circuit 35c is applied to a configuration in related technology 1 in which multiple light-emitting elements 36a1, 36a2, and 36a3 are driven by one light-emitting element driver 35a.

[0223] The configuration in Figure 26 differs from the configuration in Figure 25 in that multiple light-emitting elements 36a1, 36a2, and 36a3 are driven by different channels ch1, ch2, ch3, ... of a single light-emitting element driver 35a. Channel ch1 is connected to light-emitting element 36a1, channel ch2 to light-emitting element 36a2, and channel ch3 to light-emitting element 36a3.

[0224] The configuration of the light-emitting elements 36a1, 36a2, and 36a3, the resistor elements Ra1, Ra2, and Ra3, and the current sensing circuit 35c is basically the same as in Figure 25. Also, the current paths P1 to P3 are selected alternately, as described above.

[0225] For example, channel ch1 controls the first flash mode (e.g., strobe flash mode), channel ch2 controls the second flash mode (e.g., WLI flash mode), and channel ch3 controls the third flash mode (e.g., NBI flash mode).

[0226] Current paths P1 to P3 merge at the resistor Ra1. The current sensing circuit 35c is configured to measure the current value based on the voltage across the resistor Ra1.

[0227] Therefore, in the configuration shown in Figure 26, just like in the configuration shown in Figure 25, it is possible to detect the current values ​​flowing through multiple current paths P1 to P3 with a single current sensing circuit 35c.

[0228] Figure 27 shows an example in which one current sensing circuit 35c is applied to a configuration in related technology 1 in which one light-emitting element 36a is driven by multiple light-emitting element drivers 35a1, 35a2, and 35a3.

[0229] Multiple light-emitting element drivers 35a1, 35a2, and 35a3 each drive the light-emitting element 36a in a different light-emitting mode, for example.

[0230] In the first light emission mode (for example, strobe light emission mode), the light-emitting element 36a is driven by the light-emitting element driver 35a1, and the current path P1 passes through the light-emitting element 36a and the resistor Ra1.

[0231] In the second light emission mode (for example, the WLI light emission mode), the light-emitting element 36a is driven by the light-emitting element driver 35a2, and the current path P2 passes through the light-emitting element 36a and the resistors Ra2 and Ra1.

[0232] In the third light emission mode (for example, the NBI light emission mode), the light-emitting element 36a is driven by the light-emitting element driver 35a3, and the current path P3 passes through the light-emitting element 36a and the resistors Ra3, Ra2, and Ra1. As described above, the current paths P1 to P3 are selected alternately.

[0233] Current paths P1 to P3 merge at the resistor Ra1. The current sensing circuit 35c is configured to measure the current value based on the voltage across the resistor Ra1.

[0234] Therefore, in the configuration of Figure 27, as in the configurations of Figures 25 and 26, it is possible to detect the current values ​​flowing through multiple current paths P1 to P3 with a single current sensing circuit 35c.

[0235] According to related technology 1 shown in Figures 19 to 27, there are multiple current paths passing through the light-emitting element 36a, and the current driving range of each current path is different in the light source driving circuit 35. A circuit element or path is provided that allows current to flow in common regardless of which current path drives the light-emitting element 36a. Then, current detection is performed on that circuit element or path by a single current detection circuit 35c.

[0236] Therefore, regardless of which current path is used to drive the light-emitting element 36a, the current can be detected by a single current detection circuit 35c. Consequently, the abnormal current in the event of a malfunction can be detected collectively by a single current detection circuit 35c, preventing malfunction of the light source device.

[0237] When an abnormal condition is detected, FPGA34a switches the light source drive circuit 35 and light source 36 to a safe drive state (or a safe drive path). This reliably prevents the subject from being continuously exposed to illumination light in an abnormal state. Therefore, the subject will not be continuously exposed to, for example, strong light. [Related Technology 2]

[0238] Figure 28 shows an example configuration of the endoscope control processing device 3, which also serves as a light source device, in related technology 2.

[0239] In endoscopy, observation is performed using a strobe mode in which a strobe flashes in sync with the periodic vibrations of the subject (strobe observation mode). One example of an object observed using strobe observation mode is the vocal cords, which vibrate periodically. Other examples of objects observed using strobe observation mode include the periodically beating heart and the periodically pulsating arteries.

[0240] In this type of strobe observation, a correction process is performed to remove uneven exposure between frames or lines.

[0241] For example, Japanese Patent Publication No. 2020-151402 describes how, when controlling the LED current to adjust brightness in strobe observation mode, uniformity correction is performed based on the peak value of the LED current. In other words, this publication uses the peak value (current value) of the LED current as brightness information for each pulse of light.

[0242] However, LEDs generally do not have a perfectly linear (directly proportional) relationship between current and brightness. Therefore, if the peak value of the LED current is used directly as brightness information for each light pulse, an error will occur between the brightness based on the peak value and the actual brightness, reducing the accuracy of the uniformity correction process.

[0243] Therefore, Figure 28 shows an example configuration that improves the accuracy of the unevenness removal process.

[0244] The endoscope system 1 comprises an endoscope 2, an endoscope control processing device 3, and a monitor 4, as described above.

[0245] Endoscope 2 is configured as an electronic endoscope, as described above, and is equipped with an image sensor 11.

[0246] The endoscope control processing device 3 comprises the first image processing circuit 21, the second image processing circuit 23, the dimming detection calculation circuit 24, the user interface 29, the light source driving circuit 35, and the light source 36 as described above. The endoscope control processing device 3 further comprises a light source control circuit 41, a memory 42, a pulse brightness calculation circuit 43, and an unevenness correction circuit 44.

[0247] As described above, the light source 36 includes a light-emitting element 36a (see Figure 20, etc.) configured as, for example, an LED.

[0248] The light source control circuit 41 obtains a dimming detection value from the dimming detection calculation circuit 24. Based on the dimming detection value, the light source control circuit 41 generates a signal to control the light source 36 via the light source drive circuit 35.

[0249] The signals generated by the light source control circuit 41 include, for example, a PWM control signal and a current value control signal. The PWM control signal in strobe emission mode is a signal synchronized with the periodic vibration of the subject. The light source drive circuit 35 supplies a rectangular pulse-shaped current to the light source 36 at the timing indicated by the PWM control signal. The light source 36 emits pulsed light when current is supplied.

[0250] Memory 42 functions as a conversion parameter storage circuit. Memory 42 non-volatilely stores conversion parameters for converting the current control value I, indicated by the current value control signal, into pulse brightness L. Memory 42 may be part of memory 30b shown in Figure 3, or it may be provided separately from memory 30b.

[0251] The current value control signal generated by the light source control circuit 41 is also transmitted to the pulse brightness calculation circuit 43. The pulse brightness calculation circuit 43 reads conversion parameters from the memory 42 and converts the current control value I indicated by the current value control signal into pulse brightness L information.

[0252] Figure 29 is a graph showing a first example of conversion parameters for converting the current control value I to pulse brightness L in related technology 2.

[0253] As shown by the curve L(I) in Figure 29, the current control value I and the pulse brightness L are generally not in a linear relationship. Let Imax be the maximum value of the current control value I, and Lmax = L(Imax) be the maximum pulse brightness when I = Imax. If we assume linearity by drawing a straight line between (I,L) = (0,0) and (I,L) = (Imax,Lmax) and calculate the pulse brightness L from the current control value I, the calculated pulse brightness L will have a large error compared to the actual pulse brightness.

[0254] Therefore, as shown in the straight line L1(I) in Figure 29, a straight line is set that is closer to the curve L(I). The minimum value of the current control value I is Imin, and the minimum value of the pulse brightness when I=Imin is Lmin=L(Imin). The straight line L1(I) is set as the straight line connecting (I,L)=(Imin,Lmin) and (I,L)=(Imax,Lmax). The straight line L1(I) is represented by the linear function shown in Equation 12 below, where k1 is the slope and m1 is the intercept. L1(I)=k1×I+m1…(Formula 12)

[0255] As can be seen in Figure 29, extending the straight line L1(I) generally does not pass through (I,L)=(0,0). Furthermore, it can be seen that the error between the straight line L1(I) and the curve L(I) representing the actual pulse brightness is smaller than that of the straight line passing through (I,L)=(0,0).

[0256] Memory 42 stores the slope k1 and intercept m1, or (Imin, Lmin) and (Imax, Lmax), as conversion parameters.

[0257] The pulse brightness calculation circuit 43 reads, for example, the slope k1 and intercept m1 from the memory 42 and converts the current control value I, indicated by the current value control signal, into pulse brightness L information based on equation 12. However, instead, the pulse brightness calculation circuit 43 may read (Imin, Lmin) and (Imax, Lmax) from the memory 42 and calculate the pulse brightness L corresponding to the current control value I by interpolation or the like.

[0258] Figure 30 is a graph showing a second example of conversion parameters for converting the current control value I to pulse brightness L in related technology 2.

[0259] In Figure 29, we found the straight line L1(I) connecting the point of minimum value (Imin, Lmin) and the point of maximum value (Imax, Lmax). In Figure 30, however, we added an intermediate point (Imid, Lmid) and approximated the curve L(I) with a polyline.

[0260] The intermediate point (Imid, Lmid) can be set as the point where the sum of the area enclosed by curve L(I) and line L1a(I) and the area enclosed by curve L(I) and line L1b(I) takes its minimum value.

[0261] Alternatively, you can set Imid = (Imin + Imax) / 2 and Lmid = L(Imid). Furthermore, you can set Lmid = (Lmin + Lmax) / 2 and Imid = L^(-1)(Lmid). Here, L^(-1)(L) represents the inverse function of the curve L(I).

[0262] The line L1a(I) is defined as the line connecting (I,L)=(Imin,Lmin) and (I,L)=(Imid,Lmid). The line L1a(I) is expressed by the linear function shown in Equation 13 below, where k1a is the slope and m1a is the intercept. L1a(I)=k1a×I+m1a…(Formula 13)

[0263] The straight line L1b(I) is set as the straight line connecting (I, L) = (Imid, Lmid) and (I, L) = (Imax, Lmax). The straight line L1b(I) is represented by the following linear function formula shown in Equation 14 with a slope k1b and an intercept m1b. L1b(I) = k1b × I + m1b …(Equation 14)

[0264] As can be seen by comparing FIG. 30 with FIG. 29, it can be understood that the broken line composed of the straight lines L1a(I) and L1b(I) has a smaller error from the curve L(I) indicating the actual pulse brightness than the straight line L1(I).

[0265] In the memory 42, the slope k1a and the intercept m1a, or (Imin, Lmin) and (Imid, Lmid) are stored as the first conversion parameters. Also, in the memory 42, the slope k1b and the intercept m1b, or (Imid, Lmid) and (Imax, Lmax) are stored as the second conversion parameters.

[0266] When the current control value I < Imid, the pulse brightness calculation circuit 43 reads, for example, the slope k1a and the intercept m1a from the memory 42, and converts the current control value I into information on the pulse brightness L based on Equation 13. However, instead of this, the pulse brightness calculation circuit 43 may read (Imin, Lmin) and (Imid, Lmid) from the memory 42 and calculate the pulse brightness L corresponding to the current control value I by interpolation calculation or the like.

[0267] On the other hand, when the current control value I ≥ Imid, the pulse brightness calculation circuit 43 reads, for example, the slope k1b and the intercept m1b from the memory 42, and converts the current control value I into information on the pulse brightness L based on Equation 14. However, instead of this, the pulse brightness calculation circuit 43 may read (Imid, Lmid) and (Imax, Lmax) from the memory 42 and calculate the pulse brightness L corresponding to the current control value I by interpolation calculation or the like.

[0268] Note that while Figure 30 uses three points to construct a line graph, it is also acceptable to use four or more points. In this case, the resulting linear function will have "number of acquired data points - 1" types.

[0269] The conversion parameters shown in Figure 29 or Figure 30 are determined during process inspection. Specifically, during process inspection, the relationship between the current control value I and the pulse brightness L is obtained for two or more points, including the point of minimum value (Imin, Lmin) and the point of maximum value (Imax, Lmax). Then, the conversion parameters are calculated based on a straight line connecting two points or a polyline connecting three or more points.

[0270] Furthermore, process inspection may be performed on each individual unit of the endoscope control processing device 3. Alternatively, the results of process inspection performed on a representative unit of the endoscope control processing device 3 may be applied to each individual unit of the same product.

[0271] Furthermore, while a linear function was used above, the method is not limited to this; a function of degree two or higher may also be used.

[0272] Furthermore, although the pulse brightness calculation circuit 43 reads and uses the conversion parameters stored in memory 42 as described above, it is not limited to this. For example, if a pulse brightness calculation circuit 43 with a pre-configured arithmetic circuit corresponding to the above-described function formula is used, memory 42 may not be necessary. For example, if conversion parameters are used for each individual unit, it is advisable to use memory 42. Also, if conversion parameters common to each unit of the same product are used, either a configuration with memory 42 or a configuration without memory 42 may be adopted.

[0273] Figure 31 is a timing chart used to explain the uneven exposure between frames when the image sensor 11 uses a global shutter system, as described in Related Technology 2.

[0274] As described above, the pulsed emission of light source 36 is synchronized with the vibration of the subject (for example, the vibration of the vocal cords as described above). Therefore, the pulsed emission of light source 36 is generally not synchronized with the frame rate.

[0275] When the imaging device 11 is a global shutter type such as a CCD imager, for example, the number of pulse emissions within the exposure period of one frame may not be the same between different frames.

[0276] FIG. 31 shows an example in which the emission cycle Ti is shorter than the frame cycle Tf, i.e., Ti < Tf. Pulse emission is performed twice within the exposure period of frame n, but only once within the exposure period of frame (n + 1), resulting in uneven exposure between frames. This uneven exposure between frames is also called flicker.

[0277] FIG. 32 is a timing chart for explaining the uneven exposure between lines when the imaging device 11 is of a rolling shutter type in Related Art 2.

[0278] When the imaging device 11 is of a rolling shutter type such as a CMOS imager, for example, the number of pulse emissions within the exposure period may not be the same for each line within one frame.

[0279] FIG. 32 shows an example in which the emission cycle Ti is shorter than the frame cycle Tf, i.e., Ti < Tf. Focusing on frame n, pulse emission is performed twice within the exposure period of line 1, but only once within the exposure period of, for example, the last line, resulting in uneven exposure between lines.

[0280] The uniformity correction circuit 44 acquires information on the pulse brightness L from the pulse brightness calculation circuit 43 and corrects the uneven exposure between frames as shown in FIG. 31 or between lines as shown in FIG. 32, for example, by applying a gain based on the ratio of brightness.

[0281] According to Related Art 2 shown in FIGS. 28 to 32, by obtaining the pulse brightness L from the current control value I using a function formula with a small error from the actual pulse brightness, the uneven exposure between frames or between lines can be corrected more accurately.

[0282] In addition, by using a linear function expression, high-speed calculation can be performed with a small calculation load. Further, by using a plurality of linear function expressions configured as broken lines, the correction accuracy of exposure unevenness can be further improved. [Related Art 3]

[0283] FIG. 33 is a diagram showing a configuration example of an endoscope control processing apparatus 3 that also serves as a light source device in Related Art 3.

[0284] For example, Japanese Unexamined Patent Application Publication No. 2018-498 describes an endoscope system including a correction value setting unit and a recording unit. The correction value setting unit sets a white balance correction value. The recording unit records a coefficient used in a function indicating the relationship between the white balance correction value of reference light and the white balance correction value of other light. The endoscope system sets a new white balance correction value used for white balance correction of a reference light source. When switching from a reference light source to another light source, the endoscope system calculates the white balance correction value of the other light source using the new white balance correction value and the coefficient. The endoscope system performs white balance correction of the other light source using the calculated white balance correction value. Note that the publication describes a xenon lamp, a halogen lamp, a metal halide lamp, an LED lamp, etc. as the light source.

[0285] By the way, even if the light source 36 is the same, the white balance (emission color) may change depending on the emission luminance of the light source 36.

[0286] Therefore, FIG. 33 shows a configuration example for correcting the color change when the dimming control value changes.

[0287] The endoscope system 1 includes an endoscope 2, an endoscope control processing apparatus 3, and a monitor 4, as described above.

[0288] The endoscope 2 is configured as an electronic endoscope as described above and includes an imaging element 11.

[0289] The endoscope control processing device 3 comprises the first image processing circuit 21, the second image processing circuit 23, the dimming detection calculation circuit 24, the user interface 29, the light source drive circuit 35, the light source 36, and the light source control circuit 41 as described above. The endoscope control processing device 3 further comprises a memory 46, a color correction value calculation circuit 47, and a color correction circuit 48.

[0290] As described above, the light source 36 includes a light-emitting element 36a (see Figure 20, etc.) configured as, for example, an LED.

[0291] Memory 46 functions as a parameter storage circuit for calculating color correction values, and stores the parameters for calculating color correction values ​​in a non-volatile manner. Here, the parameters for calculating color correction values ​​are parameters for color correcting the image signal according to the dimming control value. Memory 46 may be part of memory 30b shown in Figure 3, or it may be provided separately from memory 30b.

[0292] The dimming detection calculation circuit 24 receives an image signal from the first image processing circuit 21 and averages the brightness signals within the pixel area of ​​the image sensor 11, where the optical image of the endoscope 2 is formed, to create a dimming detection value. Furthermore, the dimming detection calculation circuit 24 compares this dimming detection value with a dimming detection target value to create a dimming control value that determines the next emission brightness of the light source 36, and outputs it to the light source control circuit 41. The light source control circuit 41 linearly controls the emission brightness of the light source 36 in relation to the dimming control value. At this time, the dimming control value generated by the dimming detection calculation circuit 24 is also transmitted to the color correction value calculation circuit 47. The color correction value calculation circuit 47 reads the parameters for calculating the color correction value from the memory 46 and calculates the color correction value from the dimming control value.

[0293] For example, suppose an image signal consists of a red (R) signal, a green (G) signal, and a blue (B) signal. Furthermore, suppose that no color correction gain is applied to the green (G) signal. In this case, the color correction values ​​would be the color correction gain for the red (R) signal and the color correction gain for the blue (B) signal.

[0294] Figure 34 is a graph showing an example of a function formula for calculating color correction values ​​from dimming control values ​​in related technology 3.

[0295] The color correction value calculation circuit 47 calculates the color correction value from the dimming control value, as shown in Figure 29. The function equation shown in Figure 29 is a linear function, and if the dimming control value is x, the color correction value is y, the slope is u, and the intercept is v, it is expressed by the following equation 15. y=u×x+v…(Formula 15)

[0296] More specifically, if the slope for the red (R) signal is ur and the intercept is vr, the color correction value yr (gain for R) for the red (R) signal can be obtained from the dimming control value x by the following equation 16. yr = ur × x + vr …(Equation 16)

[0297] Furthermore, if the slope for the blue (B) signal is ub and the intercept is vb, the color correction value yb (gain for B) for the blue (B) signal can be obtained from the dimming control value x by the following equation 17. yb=ub×x+vb…(Equation 17)

[0298] The slope ur and intercept vr for the red (R) signal, and the slope ub and intercept vb for the blue (B) signal are calculated from spectral data obtained when the light source 36 emits light at different luminous intensities during process inspection. For example, the B / G ratio, R / G ratio, and peak wavelength may be obtained as spectral data.

[0299] Here, the luminescence of the light source 36 differs depending on the emission mode of the illumination light. Specifically, in strobe emission mode, the light source 36 emits light at a high luminescence for a short time using a large current. In WLI emission mode, the light source 36 emits light at a moderate luminescence (i.e., lower than in strobe emission mode) using a medium current. In NBI emission mode, the light source 36 emits light at a low luminescence (i.e., lower than in WLI emission mode) using a small current.

[0300] Therefore, in process inspection, based on spectral data obtained by emitting light from the light source 36 in each emission mode (especially the WLI emission mode and strobe emission mode for observing the subject in normal color), the slope ur and intercept vr for the red (R) signal and the slope ub and intercept vb for the blue (B) signal are calculated as parameters for calculating color correction values.

[0301] Specifically, in WLI flash mode, the dimming control value x1, the R / G ratio RG1, and the B / G ratio BG1 are obtained. Furthermore, in strobe flash mode, the dimming control value x2, the R / G ratio RG2, and the B / G ratio BG2 are obtained.

[0302] Furthermore, the R / G ratio of the standard white light is set to RG0, and the B / G ratio to BG0. At this time, the following ratios r1, r2, b1, and b2 are calculated. r1 = RG1 / RG0 r² = RG² / RG₀ b1 = BG1 / BG0 b2 = BG2 / BG0

[0303] Then, the slope ur and intercept vr of Equation 16 are calculated such that the curve passes through (x,yr)=(x1,r1) and (x,yr)=(x2,r2). Similarly, the slope ub and intercept vb of Equation 17 are calculated such that the curve passes through (x,yb)=(x1,b1) and (x,yb)=(x2,b2). The calculated color correction value parameters are stored non-volatile in memory 46 as described above.

[0304] Furthermore, process inspection may be performed on each individual unit of the endoscope control processing device 3. Alternatively, the results of process inspection performed on a representative unit of the endoscope control processing device 3 may be applied to each individual unit of the same product.

[0305] The color correction circuit 48 is configured as, for example, a white balance circuit or a paint circuit. The color correction circuit 48 obtains color correction values ​​yr and yb from the color correction value calculation circuit 47 based on the dimming control value x. The color correction circuit 48 applies the color correction value yr (gain for R) to the red (R) signal and the color correction value yb (gain for B) to the blue (B) signal to perform color correction.

[0306] Although a linear function was used above, the method is not limited to this; a function of degree two or higher may also be used.

[0307] Furthermore, although the color correction value calculation circuit 47 reads and uses the color correction value calculation parameters stored in memory 46 as described above, this is not the only way. For example, if a color correction value calculation circuit 47 with a pre-configured arithmetic circuit corresponding to the function expression described above is used, memory 46 may not be necessary. For example, if individual color correction value calculation parameters are used, it is advisable to use memory 46. Also, if common color correction value calculation parameters are used for each individual unit of the same product, either a configuration with memory 46 or a configuration without memory 46 may be adopted.

[0308] Furthermore, although the above description explained an example of calculating the color correction value from the dimming control value output by the dimming detection calculation circuit 24, the color correction value may also be calculated from the current control value I indicated by the current value control signal generated by the light source control circuit 41.

[0309] While the configurations shown in Equations 15 to 17, which determine color correction values ​​from dimming control values, are applicable to all emission modes, relatively high effectiveness can also be obtained by adopting the following simplified configuration.

[0310] In other words, since the NBI emission mode is not a mode for observing the subject with normal color balance, color correction is omitted. Also, in the WLI emission mode, the color shift of the image in the current drive range shown in Figure 21 is considered to be small. Therefore, even if the drive current changes in the WLI emission mode, color correction is omitted. Furthermore, the light emitted in the WLI emission mode is considered to be the reference white light (i.e., the color correction value is considered to be yr=yb=1 in the WLI emission mode), and color correction is performed only in the strobe emission mode. At this time, in the strobe emission mode, depending on the magnitude of the drive current (i.e., depending on the dimming control value), the color correction values ​​yr and yb are calculated from the dimming control value, etc., as shown in Equations 16 and 17, and color correction is performed by the color correction circuit 48.

[0311] According to related technology 3 shown in Figures 33 to 34, for example, the white balance distortion of an image caused by the difference in spectral characteristics of the light source 36 between the WLI flash mode and the strobe flash mode can be effectively corrected.

[0312] Furthermore, in the same emission mode (especially the strobe emission mode), it is possible to effectively correct the color changes in the image caused by changes in the spectral characteristics of the light source 36 due to the magnitude of the drive current corresponding to the dimming control. [Related Technology 4]

[0313] Furthermore, the present invention may be applied to an endoscope processor including the following configuration. Figure 35 shows an example configuration of an endoscope processor in related technology 4, in which power for the audio input circuit is supplied by an isolator element 53.

[0314] As illustrated in Figure 35, the endoscope processor has a configuration for supplying power to the endoscope from a primary power supply 51, and a configuration for supplying power to the audio input circuit from a secondary power supply 52 using an isolator element 53 with a built-in isolation transformer. In the configuration of Figure 35, the audio signal is also transmitted to the secondary circuit using the isolator element 53.

[0315] An endoscope processor having the configuration shown in Figure 35 can be connected to an endoscope. Furthermore, the endoscope processor can be connected to a microphone that acquires the patient's voice. The endoscope processor can perform laryngeal stroboscopy. In laryngeal stroboscopy, frequency components are extracted from the sound acquired using the microphone. Then, by controlling the illumination of the strobe light in synchronization with the frequency of vocal cord vibration, observation with suppressed fluctuations in vocal cord vibration can be performed during laryngeal stroboscopy. There are types of microphones that acquire sound by being in contact with the patient. Even when using such types of microphones, the configuration including the isolator element 53 as shown in Figure 35 has the effect of protecting the patient from electric shock in the event of equipment failure and making the voice signal less susceptible to noise.

[0316] Furthermore, an endoscope processor capable of performing laryngeal stroboscopy may have the following configuration. Figure 36 is a graph illustrating how, in related technology 4, an amplification line with high gain is used when the input sound pressure to the microphone is below a threshold, and an amplification line with low gain is used when it is above the threshold. Figure 37 is a diagram showing an example configuration of an endoscope processor equipped with multiple amplification mechanisms with different amplification levels, in related technology 4.

[0317] An endoscope processor comprising: an audio input unit 61 that receives sounds from the patient and acquires an audio signal; multiple amplification mechanisms (amplifiers) 63 as illustrated in Figure 37 that amplify the acquired audio signal; a buffer circuit 62 that suppresses the effects of impedance differences due to different amplification levels; an audio processing unit 64 that converts the audio signal from an analog signal to a digital signal; and a control unit 65 that controls the audio processing unit 64.

[0318] The amplification mechanism 63 includes a first amplification mechanism 63a and a second amplification mechanism 63b. The first amplification mechanism 63a is for when the audio input signal is small and has a higher gain than the second amplification mechanism 63b. The second amplification mechanism 63b is for when the audio input signal is large and has a lower gain than the first amplification mechanism 63a. The first amplification mechanism 63a has an ALC (Automatic Level Control) function. An amplification mechanism with an ALC function reduces the amplification when an input above a specified level is received, preventing clipping of the input audio. When measuring the loudness of the patient's voice (dB detection), the amplification line is switched according to the patient's voice, i.e., the sound pressure input to the microphone, as shown in Figure 36. Specifically, when the input sound pressure is below a threshold, the amplification line passing through the first amplification mechanism 63a with a high gain is used, and when the input sound pressure exceeds the threshold, it switches to the amplification line passing through the second amplification mechanism 63b with a low gain. The amplification line passing through the first amplification mechanism 63a, which has a high gain, has an ALC function as described above, which automatically reduces the gain when the input sound pressure is high. Therefore, it is not possible to use only the amplification line with a high gain for dB detection.

[0319] The volume of a patient's vocalizations varies greatly depending on the severity of their illness and the efficiency of their individual vocalizations, which in turn widens the range of volume captured by the endoscopic device. Furthermore, the sensitivity of the microphone used to capture the sound also affects the volume of the audio signal captured by the endoscopic device. Therefore, if the captured audio is amplified with a single gain, the acquired audio may become too quiet, or conversely, it may be amplified too much, causing the audio to clip. However, by incorporating multiple amplification mechanisms as described above, the amount of audio input can be optimized.

[0320] Furthermore, an endoscope processor capable of performing laryngeal stroboscopy may have the following configuration:

[0321] An endoscope processor comprising an audio input unit 61 for acquiring sounds produced by the patient, a microphone detection unit for detecting the connection of a microphone, and a circuit for blocking the audio signal output from the audio input unit 61 when the microphone is not connected. The circuit for blocking the audio signal output from the audio input unit 61 prevents noise from being recorded when the microphone is not connected and when recording endoscope images. Alternatively, the audio processing unit 64 may determine the level of the input audio and block the audio signal output if the level of the input audio is below a certain value.

[0322] When a microphone is not connected, background noise is amplified, resulting in loud noise being recorded. In particular, if the ALC function is equipped, the gain is maximized when a microphone is not connected, which increases the noise level. To address this, a circuit is provided that blocks the audio signal output from the audio input section 61, preventing the recording of noise.

[0323] Furthermore, an endoscope processor capable of performing laryngeal stroboscopy may have the following configuration:

[0324] An endoscope processor comprising an audio input unit 61 for acquiring sounds produced by the patient, a setting unit for selecting connected device information, and an audio output unit for outputting signal-processed audio.

[0325] Audio line levels (voltage levels of audio signals) are not standardized and are designed with various voltage levels for each device. Therefore, the voltage level of the audio signal output by an endoscope device may not be appropriate for audio recording equipment. In contrast, by adopting the above configuration, the output audio signal level can be optimally adjusted according to the set (selected) external device (connected device).

[0326] Although the above description primarily focuses on the case where the present invention is an endoscope control processing device and an endoscope system including the endoscope control processing device, it is not limited to this. For example, the present invention may be a method for operating an endoscope control processing device. Furthermore, the present invention may be a computer program for causing a computer to perform the same processing as the endoscope control processing device, etc. Moreover, the present invention may be a non-temporary recording medium readable by a computer that stores the computer program, etc.

[0327] Some examples of recording media for storing computer program products include portable recording media such as flexible disks, CD-ROMs (Compact Disc Read-only memory), DVDs (Digital Versatile Discs), and hard disks. The recording media may store not only the entire computer program, but also only a portion of it. Furthermore, the entire or a portion of the computer program may be distributed or provided via a communication network. Users can install the computer program from the recording media onto their computer, or download and install it via a communication network, allowing the computer to read the program, execute all or part of its operations, and perform the operations of the endoscope control processing device described above.

[0328] Furthermore, the present invention is not limited to the embodiments described above. The present invention can be implemented by modifying its components during the implementation stage, without departing from the spirit of the invention. In addition, various forms of the invention can be formed by appropriately combining the multiple components disclosed in the above embodiments. For example, some components may be deleted from all the components disclosed in the embodiments. Furthermore, components from different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

Claims

1. It is an endoscope device, It is equipped with a processor that processes the image signal output from the image sensor, The aforementioned processor, Exposure correction information is obtained based on a target value for exposure time and the amount of exposure control actually applied to the image sensor. Based on the exposure compensation information, the gain of the image signal acquired from the image sensor is adjusted. An endoscope device characterized by being configured in such a way.

2. In the endoscopic device according to claim 1, The endoscope apparatus is characterized in that the target value for the exposure time is calculated based on the result of dimming detection performed on the image signal acquired by the image sensor.

3. In the endoscopic device according to claim 2, The endoscope device is characterized in that the target value for the exposure time is calculated as the exposure time to be applied to the subsequent frame based on the image signal acquired in the preceding frame.

4. In the endoscopic device according to claim 3, The endoscope device is characterized in that the target value for the exposure time is calculated based on the ratio of the dimmed detection value to the dimmed detection target value.

5. In the endoscopic device according to any one of claims 1 to 4, The endoscope device is characterized in that the exposure control amount actually applied to the image sensor is an electronic shutter control value determined based on a target value for the exposure time.

6. In the endoscopic device according to claim 5, The endoscope device is characterized in that the electronic shutter control value is a value representing the number of lines that are blocked from light during exposure in the image sensor.

7. In the endoscopic device according to claim 6, An endoscope device characterized in that the actual exposure time is calculated based on the exposure control amount actually applied to the image sensor.

8. The endoscope apparatus according to claim 1, characterized in that the exposure correction information is obtained based on the relationship between a target value for the exposure time and the actual exposure time.

9. In the endoscopic device according to claim 8, The endoscope device is characterized in that the exposure compensation information includes the ratio of the actual exposure time to a target value related to the exposure time.

10. In the endoscopic device according to claim 9, The endoscopic device is characterized in that the processor determines the gain applied to the image signal using the reciprocal of the ratio.

11. In the endoscopic device according to claim 1, The exposure compensation information is, Based on the first image signal obtained by the image sensor reading the charge corresponding to the first frame, a target value for the exposure time in the second frame, which is a frame following the first frame, is determined. The exposure control amount actually applied to the image sensor in the second frame, Obtained based on An endoscope device characterized by the following features.

12. The endoscope device according to claim 1, The exposure compensation information includes an exposure time ratio, which is the ratio of a target value for the exposure time to the actual exposure time determined by the exposure control amount actually applied to the image sensor. The processor adjusts the gain of the image signal based on the exposure time ratio. An endoscope device characterized by the following features.

13. In the endoscopic device according to claim 12, The aforementioned processor, If the exposure control amount actually applied to the image sensor is less than a predetermined threshold, The process of acquiring the exposure time ratio and the process of adjusting the gain of the image signal based on the exposure time ratio are stopped. An endoscope device characterized by the following features.

14. In the endoscope device according to claim 13, The aforementioned processor, If the exposure control amount actually applied to the image sensor is less than the threshold, By not adjusting the gain of the aforementioned image signal, Suppresses the increase in noise components caused by signal amplification. An endoscope device characterized by the following features.

15. In the endoscopic device according to claim 12, The aforementioned image sensor has multiple pixels arranged in line units, The exposure control amount actually applied to the image sensor represents the number of lines that are blocked from light during exposure in the image sensor. The aforementioned processor, The gain adjustment of the image signal based on the exposure time ratio is performed as follows: This is performed under exposure conditions in which the number of light-shielded lines is greater than or equal to a predetermined number. An endoscope device characterized by the following features.

16. Endoscope and, A processor that processes the image signal output from the image sensor provided in the endoscope, An endoscope system comprising, The aforementioned processor, Exposure correction information is obtained based on a target value for exposure time and the amount of exposure control actually applied to the image sensor. Based on the exposure compensation information, the gain of the image signal acquired from the image sensor is adjusted. An endoscope system characterized by being configured in such a way.

17. An image processing method for processing image signals output from an image sensor provided in an endoscope, A step of acquiring exposure correction information based on a target value for exposure time and the exposure control amount actually applied to the image sensor, A step of adjusting the gain of the image signal acquired from the image sensor based on the exposure compensation information, An image processing method characterized by including

Images