Method, device, electronic device and storage medium for automatically adjusting endoscope video brightness

Abstract

The present disclosure provides an automatic adjustment method, device, electronic device and storage medium for endoscopic video brightness. The automatic adjustment method for endoscopic video brightness disclosed in the present disclosure includes: determining the average brightness value of the current frame image of the endoscopic video; determining the brightness increment parameter of the current frame image of the endoscopic video according to the average brightness value of the current frame image of the endoscopic video and the predetermined expected brightness average value; determining the light source brightness value of the illumination light source in the endoscopic camera system using a fuzzy control algorithm according to the brightness increment parameter of the current frame image of the endoscopic video; transmitting the light source brightness value to the light source device so that the light source device adjusts the brightness of the illumination light source to the light source brightness value; determining the gain value and exposure value of the image sensor and transmitting them to the image sensor so that the image sensor adjusts its own gain and exposure. The present disclosure can realize real-time dynamic adjustment of the brightness of the light source in the endoscopic camera system, and at the same time can ensure image brightness and low image noise, good image quality, good image fluency, no stroboscopic phenomenon, etc. and reduce the power consumption of the light source of the endoscopic camera system.

Description

Technical Field

[0001] The present invention relates to a method, a device, an electronic device and a storage medium for automatically adjusting the brightness of an endoscope video. Background Art

[0002] Taking a medical endoscope as an example, the endoscope camera system mainly includes: a handle, an imaging lens, an image sensor, an image display, an image processing unit, an illumination light source, and a light source device for controlling the brightness of the illumination light source, etc.

[0003] At present, the video brightness adjustment of endoscope camera systems mostly relies on automatic exposure algorithms. Traditional automatic exposure algorithms need to configure the exposure value and gain parameters of the sensor under the condition of fixed light source brightness, and then write the parameter information to be configured into the image sensor through the IIC bus. When configuring the gain value, noise will be generated, thus affecting the image quality. If the gain is not adjusted, the image brightness cannot meet the requirements. Although the expected video brightness value can be reduced to ensure that the gain value adjustment range is not too large and the noise intensity is minimized, it is impossible to effectively achieve a higher range of image brightness and avoid the generation of noise. Therefore, this automatic exposure algorithm is likely to cause increased video image noise and deterioration of video image quality.

[0004] Since the image processing unit of the endoscope camera system is not strongly linked to the light source device, the light source device must output at a specific brightness. To ensure that the automatic exposure algorithm can run, most light source devices operate at a high brightness, resulting in high power consumption. If the light source device operates at high brightness for a long time, it is easy to cause the handle end of the endoscope camera system to heat up, and it is also easy to cause thermal damage to biological tissues, affecting the doctor's experience. Summary of the invention

[0005] In order to solve at least one of the above technical problems, the present disclosure provides a method, device, electronic device and storage medium for automatically adjusting the brightness of an endoscope video.

[0006] According to a first aspect of the present disclosure, there is provided a method for automatically adjusting the brightness of an endoscope video, comprising:

[0007] Determine the average brightness of the current frame image of the endoscope video;

[0008] Determining a brightness increment parameter of a current frame image of the endoscopic video according to a brightness average value of a current frame image of the endoscopic video and a predetermined expected brightness average value;

[0009] According to the brightness increment parameter of the current frame image of the endoscope video, a fuzzy control algorithm is used to determine the light source brightness value of the illumination light source in the endoscope camera system;

[0010] Transmitting the light source brightness value to the light source device so that the light source device adjusts the brightness of the illumination light source to the light source brightness value;

[0011] A gain value and an exposure value of the image sensor are determined and transmitted to the image sensor so that the image sensor adjusts its own gain and exposure.

[0012] According to the method for automatically adjusting the brightness of an endoscopic video of at least one embodiment of the present disclosure, determining the average brightness value of a current frame image of the endoscopic video includes: extracting brightness data of the current frame image of the endoscopic video; performing histogram statistics on the number of pixels of the brightness data at each grayscale value; and calculating the average brightness value of the current frame image based on the result of the histogram statistics.

[0013] According to the method for automatically adjusting the brightness of an endoscopic video of at least one embodiment of the present disclosure, the brightness increment parameter of the current frame image of the endoscopic video is determined according to the difference between the brightness average value of the current frame image of the endoscopic video and the expected brightness average value.

[0014] According to at least one embodiment of the present disclosure, a method for automatically adjusting the brightness of an endoscopic video is provided, and a fuzzy control algorithm is used to determine the light source brightness value of an illumination light source in an endoscopic camera system according to the brightness increment parameter of a current frame image of the endoscopic video, including: using the brightness increment parameter as an observation quantity of fuzzy control, using the light source brightness value as a control quantity of fuzzy control, constructing a fuzzy set of the brightness increment parameter and a fuzzy set of the light source brightness value, and determining a fuzzy control rule; performing a fuzzy decision according to the brightness increment parameter of the current frame image of the endoscopic video, the fuzzy set of the brightness increment parameter, the fuzzy set of the light source brightness value, and the fuzzy control rule, and obtaining the light source brightness value of the illumination light source by performing a defuzzification operation on the result of the fuzzy decision.

[0015] According to the method for automatically adjusting the brightness of endoscopic video of at least one embodiment of the present disclosure, the transmission of the light source brightness value to the light source device includes: encoding and integrating the light source brightness value to obtain light source control transmission information; and transmitting the light source control transmission information to the light source device via a bus in the frame blanking area of ​​the endoscopic video.

[0016] According to at least one embodiment of the present disclosure, the method for automatically adjusting the brightness of an endoscopic video also includes: performing the following brightness adjustment operations in the frame blanking area of ​​the endoscopic video through a light source device: turning on the illumination light source, receiving and parsing the light source control transmission information, adjusting the brightness of the illumination light source to the light source brightness value, and turning off the illumination light source after the brightness adjustment of the illumination light source is completed.

[0017] According to at least one embodiment of the present disclosure, the method for automatically adjusting the brightness of endoscope videos further includes: performing a timing alignment operation to align the timing of sending instructions for the brightness value of the light source with the timing of outputting frame images by the image sensor.

[0018] According to at least one embodiment of the present disclosure, a method for automatically adjusting the brightness of an endoscope video is disclosed, in which the gain value and exposure value of an image sensor are determined and transmitted to the image sensor, including: setting the current gain value of the image sensor to a predetermined first fixed value; setting the current exposure value of the image sensor to a predetermined second fixed value.

[0019] According to the method for automatically adjusting the brightness of an endoscope video of at least one embodiment of the present disclosure, the second fixed value is the maximum exposure value of the image sensor.

[0020] According to a second aspect of the present disclosure, there is provided an automatic endoscope video brightness adjustment device, comprising:

[0021] An image brightness determination unit, used to determine the average brightness of a current frame image of the endoscope video;

[0022] a brightness increment parameter determination unit, configured to determine a brightness increment parameter of a current frame image of the endoscope video according to a brightness average value of the current frame image of the endoscope video and a predetermined expected brightness average value;

[0023] A light source brightness value determination unit, used to determine the light source brightness value of the illumination light source in the endoscope camera system using a fuzzy control algorithm according to a brightness increment parameter of a current frame image of the endoscope video;

[0024] A light source brightness transmission unit, used to transmit the light source brightness value to the light source device, so that the light source device adjusts the brightness of the illumination light source to the light source brightness value;

[0025] The image parameter determination unit is used to determine the gain value and exposure value of the image sensor and transmit the values ​​to the image sensor so that the image sensor can adjust its own gain and exposure.

[0026] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0027] A memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the automatic adjustment method of endoscope video brightness as described above.

[0028] According to a fourth aspect of the present disclosure, there is provided an endoscope camera system, comprising:

[0029] Lighting source;

[0030] A camera device, including an image sensor, for collecting image data of biological tissue under illumination conditions of an illumination light source to generate an endoscopic video;

[0031] A light source device, used to adjust the brightness of the illumination light source according to the light source brightness value provided by the endoscope video brightness automatic adjustment device; and

[0032] The above-mentioned endoscope video brightness automatic adjustment device.

[0033] According to a fifth aspect of the present disclosure, a readable storage medium is provided, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above-mentioned method for automatically adjusting the brightness of endoscope videos.

[0034] In the disclosed embodiments, the exposure value and gain value of the image sensor are fixed, and the brightness of the light source is automatically determined and adjusted dynamically and in real time in the blanking area of ​​the video stream. During the adjustment of the brightness of the light source, the image brightness can be guaranteed and the image noise is small, the image quality is good, the image transition is smooth, the image fluency is good, there is no stroboscopic phenomenon, no flickering and brightness jump, the brightness of the light source is determined and adjusted quickly, and the power consumption of the light source can be reduced at the same time, effectively avoiding heating of the handle end of the endoscope and burns of biological tissue due to high power consumption of the light source, thereby improving the doctor's user experience and reducing the overall power consumption of the endoscope camera system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0036] Figure 1 It is a flowchart of a method for automatically adjusting the brightness of an endoscope video according to an embodiment of the present disclosure.

[0037] Figure 2 It is a timing diagram of turning on, adjusting the brightness and turning off the lighting source under the VGA standard timing according to one embodiment of the present disclosure.

[0038] Figure 3 This is an example diagram of an automatic brightness adjustment device for endoscope video using a hardware implementation of a processing system according to an embodiment of the present disclosure.

[0039] Figure 4 It is a schematic structural diagram of an endoscope camera system according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0041] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0043] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0044] When a component is referred to as being "on," "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0045] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0046] Figure 1 FIG. 1 is a flow chart of a method for automatically adjusting the brightness of an endoscope video according to an embodiment of the present disclosure. Figure 1 As shown, the method for automatically adjusting the brightness of an endoscope video according to an embodiment of the present disclosure may include:

[0047] Step S102, determining the average brightness of the current frame image of the endoscope video;

[0048] In some implementations, step S102 may include: acquiring a current frame image of the endoscopic video, performing histogram statistics on the current frame image of the endoscopic video, and calculating a brightness average value of the current frame image based on the result of the histogram statistics.

[0049] Specifically, acquiring the current frame image of the endoscopic video may include: acquiring RAW data of the current frame image of the endoscopic video from an image sensor.

[0050] Specifically, the histogram statistics may include the following steps:

[0051] Step a1: Convert the RAW data of the current frame image of the endoscope video from the image sensor into a YUV format to extract the Y component.

[0052] The Y component of the current frame image can be extracted by the following formula (1).

[0053] Y = 0.257*R + 0.645*G + 0.098*B (1)

[0054] Among them, R represents the R component of the RAW data of the current frame image, G represents the G component of the RAW data of the current frame image, B represents the B component of the RAW data of the current frame image, and Y represents the Y component of the YUV data of the current frame image.

[0055] Since the original image of the current frame image is in RAW format, that is, each pixel has only one color component, the remaining color components can be set to zero for calculation to obtain the Y component, and the value range of the Y component can be between 0-255.

[0056] Among them, YUV is a color encoding mode, in which Y represents brightness (Luminance), that is, grayscale value, and UV represents chrominance (Chrominance) and concentration (Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0057] Step a2: Count and record the number of pixels N(i) of the Y component at each gray value P(i) to complete histogram statistics.

[0058] In some implementations, the average brightness value Lc of the current frame image may be calculated using the following formula (2).

[0059]

[0060] Where Lc represents the average brightness of the current frame image, P(i) represents the image grayscale value of the i-th pixel, and the value range of i is between 0 and 255. N(i) is the number of pixels in the histogram corresponding to the grayscale value P(i). Na is the total number of pixels in the current frame image.

[0061] Step S104, determining a brightness increment parameter of the current frame image of the endoscope video according to the brightness average value of the current frame image of the endoscope video and a predetermined expected brightness average value;

[0062] In some implementations, the brightness increment parameter of the current frame image of the endoscopic video may be determined based on the difference between the brightness average value of the current frame image of the endoscopic video and the expected brightness average value.

[0063] Specifically, in step S104, the brightness increment parameter of the current frame image can be calculated by the following formula (3).

[0064] Pa = Lc–Lp (3)

[0065] Wherein, Pa represents a brightness increment parameter, Lc represents a brightness average value of the current frame image, and Lp represents a predetermined expected brightness average value. Here, the specific value of the expected brightness average value can be flexibly set as needed and can be a fixed value.

[0066] Step S106, determining the brightness value of the light source of the endoscope using a fuzzy control algorithm according to the brightness increment parameter of the current frame image of the endoscope video;

[0067] Step S108, transmitting the light source brightness value to the light source device so that the light source device adjusts the brightness of the illumination light source to the light source brightness value;

[0068] Step S110 , determining a gain value and an exposure value of the image sensor and transmitting the values ​​to the image sensor so that the image sensor can adjust its own gain and exposure.

[0069] In some implementations, step S110 may include: setting the current gain value (Gain) of the image sensor to a preset first fixed value, and setting the current exposure value (Exp) of the image sensor to a preset second fixed value. Thus, by fixing the gain and exposure values ​​of the image sensor and controlling the brightness of the light source, the noise of the ultra-high-definition video image can be reduced and the image quality can be improved.

[0070] In specific applications, the specific values ​​of the first fixed value and the second fixed value can be flexibly set according to actual needs, and the embodiments of the present disclosure are not limited to this. For example, the second fixed value can be the maximum exposure value of the image sensor. In addition to being set to the maximum exposure value of the image sensor, the current exposure value can also be set to other fixed values.

[0071] In some implementations, in step S110 , the current gain value and the current exposure value of the image sensor may be sent to the image sensor through a control interface of the image sensor.

[0072] Since there is no need to dynamically adjust the gain value Gain and the exposure value Exp of the image sensor in the embodiment of the present disclosure, in a specific application, step S110 may be selectively performed once or repeatedly.

[0073] In some implementations, step S106 may include the following steps b1 to b5:

[0074] Step b1, using the brightness increment parameter Pa as the observation quantity of fuzzy control, and selecting the light source brightness value Wdat as the control quantity.

[0075] Step b2, constructing a fuzzy set of brightness increment parameter Pa;

[0076] If the image brightness increment parameter Pa is set to take a value range of [-5,5], the fuzzy matrix table corresponding to the image brightness increment parameter Pa is shown in Table 1. In the example of Table 1, the image brightness increment parameter Pa is divided into five fuzzy sets, namely, negative large (NB), negative small (NS), zero (ZO), positive small (PS), and positive large (PB). Pa is negative, indicating that the current brightness average is lower than the expected image brightness average, and Pa is positive, indicating that the current image brightness average is higher than the expected image brightness average.

[0077] Pa -5 -4 -3 -2 -1 0 1 2 3 4 5 PB 0 0 0 0 0 0 0 0 0.5 0.7 1 PS 0 0 0 0 0 0 0.5 1 0.5 0.3 0 ZO 0 0 0 0 0.5 1 0.5 0 0 0 0 NS 0 0.3 0.5 1 0.5 0 0 0 0 0 0 NB 1 0.7 0.5 0 0 0 0 0 0 0 0

[0078] Table 1

[0079] Step b3, constructing a fuzzy set of light source brightness value Wdat;

[0080] If the value range of the light source brightness value Wdat is set to [-5,5], the fuzzy matrix table corresponding to the light source brightness value Wdat is shown in Table 2. In the example of Table 2, the light source brightness value Wdat is divided into 5 fuzzy sets, namely, negative large (NB1), negative small (NS1), zero (ZO1), positive small (PS1), and positive large (PB1). Wdat is negative, which means increasing the light source brightness value, and Wdat is positive, which means decreasing the light source brightness value.

[0081] Wdat -5 -4 -3 -2 -1 0 1 2 3 4 5 PB1 0 0 0 0 0 0 0 0 0.5 0.7 1 PS1 0 0 0 0 0 0 0.5 1 0.5 0.3 0 ZO1 0 0 0 0 0.5 1 0.5 0 0 0 0 NS1 0 0.3 0.5 1 0.5 0 0 0 0 0 0 NB1 1 0.7 0.5 0 0 0 0 0 0 0 0

[0082] Table 2

[0083] Step b4, determining the fuzzy control rule, which represents the relationship between the fuzzy set of the observed quantity and the fuzzy set of the controlled quantity.

[0084] The fuzzy control rule may include multiple statements. In some implementations, the fuzzy control rule of the embodiment of the present disclosure may be expressed as the following statement:

[0085] If Pa is negative and large, then Wdat is negative and large;

[0086] If Pa is negative and small, then Wdat is negative and small;

[0087] If Pa is zero, then Wdat is zero;

[0088] If Pa is positively small, then Wdat is positively small;

[0089] If Pa is positive, then Wdat is positive.

[0090] Exemplarily, taking the fuzzy sets in Table 1 and Table 2 as examples, the fuzzy control rules can be expressed as a fuzzy relationship matrix R, and the fuzzy relationship matrix R can be expressed as the following formula (4).

[0091] R=(NB∩NB1)∪(NS∩NS1)∪(ZO∩ZO1)∪(PS∩PS1)∪(PB∩PB1) (4)

[0092] Among them, R represents a fuzzy relationship, “∩” represents intersection, and “∪” represents parallel.

[0093] Step b5, making a fuzzy decision based on the brightness increment parameter of the current frame image of the endoscope video, the fuzzy set of the brightness increment parameter Pa, the fuzzy set of the light source brightness value Wdat and the fuzzy control rule, and obtaining the light source brightness value Wdat of the illumination light source by performing a defuzzification operation on the result of the fuzzy decision.

[0094] In some implementations, the defuzzification operation may use, but is not limited to, the centroid method.

[0095] It should be noted that the implementation of the above steps b1 to b5 is only an example of an implementation of step S106, and is not intended to limit the specific implementation of step S106. Those skilled in the art should understand that the specific implementation of step S106 is not limited to the above.

[0096] In some embodiments, step S108 may include: encoding and integrating the light source brightness value to obtain light source control transmission information; and transmitting the light source control transmission information to the light source device through the bus in the frame blanking area of ​​the endoscope video. In this way, the transmission timing of the light source control transmission information Wctrl is the frame blanking area of ​​the endoscope video, and the brightness of the illumination light source can be adjusted to the light source brightness value determined in step S106 through the light source device under the premise of ensuring that the output endoscope video does not flicker.

[0097] In some implementations, in step S108, the light source brightness value Wdat can be encoded in a specific transmission protocol format. For example, the specific transmission protocol format can be "data header + instruction type + Wdat + checksum + data tail". Among them, the data header can be 0x55 in hexadecimal, the instruction type is used to distinguish whether the current instruction is a switch light source module or a brightness adjustment instruction, the checksum verification method can use CRC verification, the data tail can be 0xFA in hexadecimal, and the value range of the light source brightness value Wdat can be set to 0x00~0x63.

[0098] Exemplarily, the instruction type and instruction content when encoding the light source brightness value Wdat can be configured according to the following Table 3.

[0099] Instruction Type Instruction content Turn on the lighting source 0x55 0x01 0x00 0xf1 0xFA Turn off lighting 0x55 0x00 0x00 0xe4 0xFA Adjust the brightness of the lighting source 0x55 0x02 Wdat 0xce 0xFA

[0100] Table 3

[0101] In step S108, the transmission timing of the light source control transmission information carrying the light source brightness value is selected as the frame blanking area of ​​the video. For occasions where the video frame rate and response speed are required to be high, the brightness of each frame can be adjusted, that is, in step S108, the light source control transmission information WCtrl can be sent in the frame blanking area of ​​each frame. For occasions where the video frame rate and response speed are required to be low, in step S108, the light source control transmission information WCtrl can be transmitted in the frame blanking area every few frames or even more than ten frames. In this way, not only the purpose of adjusting the brightness can be achieved, but also the power consumption of the light source can be effectively reduced.

[0102] To ensure that the transmission rate can meet the requirements and the light source device can respond in real time. In some implementations, in step S108, the bus interface for transmitting the light source control transmission information WCtrl may preferably be, but not limited to, UART, SPI, IIC, etc.

[0103] In some implementations, in step S108 , the light source control transmission information may be transmitted to the light source device using an encrypted communication method to improve the stability and security of the endoscope camera system.

[0104] In some embodiments, after step S108, the process may further include: performing the following brightness adjustment operation in the frame blanking area of ​​the endoscope video through the light source device: turning on the illumination light source, receiving and parsing the light source control transmission information WCtrl, adjusting the brightness of the illumination light source to the light source brightness value carried in the light source control transmission information WCtrl, and turning off the illumination light source after the brightness adjustment of the illumination light source is completed. In this way, the brightness of the illumination light source can be adjusted only when the illumination light source is turned on, and the illumination light source can be turned off after the brightness adjustment of the illumination light source is completed, thereby minimizing the power consumption of the light source, effectively avoiding heating of the handle end of the endoscope camera system or thermal damage to biological tissue caused by the long-term high-brightness operation of the light source, and then effectively improving the doctor's user experience.

[0105] Figure 2 FIG. 1 shows a schematic diagram of the timing of turning on, adjusting the brightness and turning off the illumination source in the VGA standard timing sequence. Figure 2 As shown, in the VGA standard timing, the VGA timing consists of two parts, namely, the horizontal synchronization timing (HSync) and the field synchronization timing (VSync). Figure 2 In the video, Video represents the transmitted image information, and HSync represents the horizontal synchronization signal.

[0106] The line synchronization timing is a line scan cycle from the rising edge to the next rising edge. A complete line scan cycle contains 6 stages: Sync, Back Porch, Left Border, Addressable Video, Right Border, Front Porch. In a complete line scan cycle, Video completes the scanning and display of a line of image under the synchronization of the line synchronization signal. The image information of Video is valid only in the "Addressable" Video (valid image) stage, and the image information is invalid in other stages. The line synchronization signal maintains a high level in the Sync stage and a low level in other stages.

[0107] The field synchronization timing is a field scanning cycle from the rising edge to the next rising edge. A complete field scanning cycle also includes 6 stages: Sync, Back Porch, Top Border, Addressable Video, Bottom Border, Front Porch. In a complete field scanning cycle, the display of a frame of image is completed under the joint action of the line synchronization signal and the field synchronization signal. The image information is valid only in the "Addressable" Video (valid image) stage, and the image information is invalid in other stages. The line synchronization signal maintains a high level in the Sync stage and a low level in other stages. After completing a field scanning cycle, the scanning of the next frame of image begins.

[0108] In the disclosed embodiment, the timing of turning on, adjusting the brightness and turning off the lighting light source and the timing of sending the light source control transmission information Wctrl can be executed in the synchronization phase and the trailing edge phase of the image information (Video). Specifically, the light source device can turn on the lighting light source at the start of the synchronization phase, the light source device can receive and parse the light source control transmission information Wctrl in the synchronization phase and the trailing edge phase, and adjust the brightness of the lighting light source to the light source brightness value carried in the light source control transmission information Wctrl, and the light source device can turn off the lighting light source at the end of the trailing edge phase.

[0109] In a specific application, by looping through steps S102 to S108 and executing step S110 once or repeatedly, automatic brightness adjustment can be performed for each frame image or selected frame image in the endoscopic video, thereby obtaining a better automatic brightness adjustment effect for the endoscopic video.

[0110] Step S110 may be executed synchronously with steps S102 to S108, or may be executed between these steps, or may be executed before these steps. The embodiment of the present disclosure does not limit the execution order.

[0111] Since the parameter setting of the image sensor and the brightness adjustment of the endoscope video image can be separated, the video data can be transmitted to the endoscope video brightness automatic adjustment device by wire after encoding, and the endoscope video brightness automatic adjustment device then outputs the video stream and the line field synchronization signal after decoding. Therefore, the video image actually received by the endoscope video brightness automatic adjustment device will have a certain delay relative to the output of the image sensor. Therefore, in order to ensure that the adjustment process of the illumination light source is in the frame blanking area of ​​the image sensor, it is necessary to strictly align the adjustment process of the light source device and the timing of the image sensor's own output. Therefore, in some embodiments, the endoscope video brightness automatic adjustment method disclosed in the present invention may also include: performing a timing alignment operation to align the instruction sending timing of the light source brightness value with the timing of the image sensor output frame image. The timing alignment operation can be performed before step S102 or before step S108. The specific execution timing of the timing alignment operation is not limited by the embodiments of the present disclosure.

[0112] In some implementations, the timing alignment operation may be performed according to the following steps d1 to d2:

[0113] Step d1, configuring the image sensor to be in a slave output mode, that is, the image sensor outputs image data according to an external video timing.

[0114] In step d2, the host to which the endoscope video brightness automatic adjustment device 300 belongs generates ultra-high-definition video timing. Considering that the transmission delay between the image sensor and the host is controllable, a start signal can be sent to the image sensor at a fixed time difference before the field synchronization of each frame of the ultra-high-definition timing is generated to start the image data output. At this time, the image sensor outputs according to the video timing generated by the host, and can ensure that the image sensor timing and the host timing where the endoscope video brightness automatic adjustment device 300 is located are strictly aligned.

[0115] Figure 3 An example diagram of an automatic endoscope video brightness adjustment device using a hardware implementation of a processing system is shown.

[0116] The device may include a corresponding module for performing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be performed by a corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0117] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnected buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 400 connects various circuits including one or more processors 500, memory 600, and / or hardware modules together. The bus 400 can also connect various other circuits 700 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0118] The bus 400 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one connecting line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0119] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0120] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or used in conjunction with such instruction execution systems, apparatuses, or devices.

[0121] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0122] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0123] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0124] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0125] like Figure 3 As shown, in some embodiments of the present disclosure, the endoscope video brightness automatic adjustment device 300 may include:

[0126] An image brightness determination unit 302 is used to determine the average brightness of the current frame image of the endoscope video;

[0127] A brightness increment parameter determination unit 304, configured to determine a brightness increment parameter of a current frame image of the endoscope video according to a brightness average value of the current frame image of the endoscope video and a predetermined expected brightness average value;

[0128] A light source brightness value determination unit 306, configured to determine the light source brightness value of the illumination light source in the endoscope camera system using a fuzzy control algorithm according to a brightness increment parameter of a current frame image of the endoscope video;

[0129] The light source brightness transmission unit 308 is used to transmit the light source brightness value to the light source device so that the light source device adjusts the brightness of the illumination light source to the light source brightness value;

[0130] The image parameter determination unit 310 is used to determine the gain value and exposure value of the image sensor and transmit the values ​​to the image sensor so that the image sensor can adjust its own gain and exposure.

[0131] In some embodiments, the image brightness determination unit 302 is used to: extract brightness data of the current frame image of the endoscopic video; perform histogram statistics on the number of pixels of the brightness data at each grayscale value; and, based on the results of the histogram statistics, calculate the average brightness of the current frame image.

[0132] In some implementations, the brightness increment parameter determination unit 304 is used to determine the brightness increment parameter of the current frame image of the endoscopic video according to the difference between the brightness average value of the current frame image of the endoscopic video and the expected brightness average value.

[0133] In some embodiments, the light source brightness value determination unit 306 is used to: use the brightness increment parameter as the observation quantity of fuzzy control, use the light source brightness value as the control quantity of fuzzy control, construct a fuzzy set of brightness increment parameters and a fuzzy set of light source brightness values, and determine the fuzzy control rules; and make a fuzzy decision based on the brightness increment parameter of the current frame image of the endoscopic video, the fuzzy set of brightness increment parameters, the fuzzy set of light source brightness values, and the fuzzy control rules, and obtain the light source brightness value of the illumination light source by performing a defuzzification operation on the result of the fuzzy decision.

[0134] In some implementations, the light source brightness transmission unit 308 is used to: encode and integrate the light source brightness values ​​to obtain light source control transmission information; and transmit the light source control transmission information to the light source device via a bus in the frame blanking area of ​​the endoscope video.

[0135] In some implementations, the endoscope video brightness automatic adjustment device 300 may further include: a timing alignment unit 312, configured to perform a timing alignment operation so as to align the instruction sending timing of the light source brightness value with the timing of the image sensor outputting the frame image.

[0136] In some implementations, the image parameter determination unit 310 is used to: set the current gain value of the image sensor to a preset first fixed value; and set the current exposure value of the image sensor to a preset second fixed value. For example, the second fixed value may be the maximum exposure value of the image sensor.

[0137] Figure 4 FIG. 1 is a schematic diagram of the structure of an endoscope camera system according to an embodiment of the present disclosure. Figure 4 As shown, the endoscope camera system 800 may include: an illumination light source 802, a camera device 804, a light source device 806 and an endoscope video brightness automatic adjustment device 300 for executing the above-mentioned endoscope video brightness automatic adjustment device.

[0138] The camera device 804 includes an image sensor, and the camera device 804 can be used to collect image data of biological tissue under the lighting condition of the lighting light source to generate an endoscopic video.

[0139] Among them, the light source device 806 can be used to adjust the brightness of the illumination light source according to the light source brightness value provided by the endoscope video brightness automatic adjustment device;

[0140] In some implementations, the endoscope camera system 800 may further include: a display device 808 for displaying a current frame image processed by the endoscope video automatic brightness adjustment device 300 .

[0141] In some embodiments, the light source device 806 can be connected to the endoscope video automatic brightness adjustment device 300 via a communication bus, and the camera device 804 can be connected to the endoscope video automatic brightness adjustment device 300 via an image sensor control interface. The display device 808 can be connected to the endoscope video automatic brightness adjustment device 300 via a communication bus or a specific interface.

[0142] In a specific application, the endoscope camera system 400 may include a host, the endoscope video brightness automatic adjustment device 300 may be implemented by the image processing software and / or image processing related hardware in the host, and the display device 808 may be implemented by the display software and / or display hardware in the host.

[0143] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above-mentioned endoscope video brightness automatic adjustment method.

[0144] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above-mentioned method for automatically adjusting the brightness of endoscope videos.

[0145] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0146] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0147] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for automatically adjusting the brightness of an endoscope video, characterized in that: include: Determine the average brightness of the current frame image of the endoscope video; Determining a brightness increment parameter of a current frame image of the endoscopic video according to a difference between a brightness average value of a current frame image of the endoscopic video and a predetermined expected brightness average value; According to the brightness increment parameter of the current frame image of the endoscope video, a fuzzy control algorithm is used to determine the light source brightness value of the illumination light source in the endoscope camera system; Transmitting the light source brightness value to the light source device so that the light source device adjusts the brightness of the illumination light source to the light source brightness value; Determining a gain value and an exposure value of an image sensor and transmitting the values ​​to the image sensor so that the image sensor adjusts its own gain and exposure; Wherein, determining the gain value and exposure value of the image sensor and transmitting them to the image sensor includes: setting the current gain value of the image sensor to a predetermined first fixed value; setting the current exposure value of the image sensor to a predetermined second fixed value; wherein the second fixed value is the maximum exposure value of the image sensor.

2. The method for automatically adjusting the brightness of endoscope video according to claim 1, characterized in that: The step of determining the average brightness of the current frame image of the endoscope video includes: Extracting brightness data of a current frame image of an endoscope video; Perform histogram statistics on the number of pixels at each grayscale value of the brightness data; According to the statistical result of the histogram, the average brightness value of the current frame image is calculated.

3. The method for automatically adjusting the brightness of endoscope video according to claim 1, characterized in that: According to the brightness increment parameter of the current frame image of the endoscope video, the fuzzy control algorithm is used to determine the light source brightness value of the illumination light source in the endoscope camera system, including: Taking the brightness increment parameter as the observation quantity of fuzzy control and the light source brightness value as the control quantity of fuzzy control, the fuzzy set of brightness increment parameter and the fuzzy set of light source brightness value are constructed, and the fuzzy control rules are determined; Fuzzy decision is made according to the brightness increment parameter of the current frame image of the endoscope video, the fuzzy set of the brightness increment parameter, the fuzzy set of the light source brightness value and the fuzzy control rule, and the light source brightness value of the illumination light source is obtained by defuzzifying the result of the fuzzy decision.

4. The method for automatically adjusting the brightness of endoscope video according to claim 1, characterized in that: The step of transmitting the light source brightness value to the light source device comprises: Encoding and integrating the brightness values ​​of the light sources to obtain light source control transmission information; In the frame blanking area of ​​the endoscope video, the light source control transmission information is transmitted to the light source device through the bus.

5. The method for automatically adjusting the brightness of endoscope video according to claim 4, characterized in that: Also includes: The following brightness adjustment operation is performed in the frame blanking area of ​​the endoscopic video through the light source device: turn on the illumination light source, receive and parse the light source control transmission information, adjust the brightness of the illumination light source to the light source brightness value, and turn off the illumination light source after the brightness adjustment of the illumination light source is completed.

6. The method for automatically adjusting the brightness of endoscope video according to claim 1, characterized in that: Also includes: A timing alignment operation is performed to align the timing of sending the instruction of the light source brightness value with the timing of outputting the frame image by the image sensor.

7. An automatic endoscope video brightness adjustment device, characterized in that: include: An image brightness determination unit, used to determine the average brightness of a current frame image of the endoscope video; a brightness increment parameter determination unit, configured to determine a brightness increment parameter of a current frame image of the endoscope video according to a difference between a brightness average value of the current frame image of the endoscope video and a predetermined expected brightness average value; A light source brightness value determination unit, used to determine the light source brightness value of the illumination light source in the endoscope camera system using a fuzzy control algorithm according to a brightness increment parameter of a current frame image of the endoscope video; A light source brightness transmission unit, used to transmit the light source brightness value to the light source device, so that the light source device adjusts the brightness of the illumination light source to the light source brightness value; An image parameter determination unit, used for determining a gain value and an exposure value of the image sensor and transmitting the values ​​to the image sensor so that the image sensor can adjust its own gain and exposure; Wherein, determining the gain value and exposure value of the image sensor and transmitting them to the image sensor includes: setting the current gain value of the image sensor to a predetermined first fixed value; setting the current exposure value of the image sensor to a predetermined second fixed value; wherein the second fixed value is the maximum exposure value of the image sensor.

8. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the endoscope video brightness automatic adjustment method according to any one of claims 1 to 6.

9. An endoscope camera system, characterized in that: include: Lighting source; A camera device, including an image sensor, for collecting image data of biological tissue under illumination conditions of an illumination light source to generate an endoscopic video; A light source device, used to adjust the brightness of the illumination light source according to the light source brightness value provided by the endoscope video brightness automatic adjustment device; as well as An automatic endoscope video brightness adjustment device as described in claim 7.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the method for automatically adjusting the brightness of an endoscope video according to any one of claims 1 to 6.

Images