Method, device and system for automatically adjusting endoscope video brightness

Abstract

The invention discloses an endoscope imaging system and a method for automatically adjusting image exposure parameters, and belongs to the technical field of image processing.The method comprises the steps that a first endoscope image is collected and subjected to resolution reduction processing, and a second endoscope image is obtained; identifying an effective operation area of the surgery and calculating the area overexposure degree; and adaptively adjusting the endoscope video brightness based on the overexposure degree. Effective operation area identification and overexposure degree calculation can be realized by adopting a deep learning model or a traditional image processing method. Brightness adjustment judges whether overexposure exists or not according to the overexposure degree; if not, automatic exposure is kept; and if overexposure exists, sequentially switching to an exposure mode of intelligent photometry, reducing light source brightness, adjusting a Gamma curve and reducing exposure brightness so as to eliminate overexposure and restore image details. The system comprises a light source device, a camera device, a brightness adjusting device, a storage device and a display device, can automatically detect overexposure and prompt a user, and realizes intelligent adjustment of endoscope image brightness.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method, apparatus, and system for automatically adjusting the brightness of endoscopic video. Background Technology

[0002] In endoscopic surgery, high-definition, stable video imaging is a crucial prerequisite for surgeons to perform precise operations. Endoscopic systems provide surgeons with a surgical field of view by transmitting images of internal tissues to a monitor. However, due to the complexity of internal anatomy and the dynamic changes during surgical procedures, the imaging environment often undergoes drastic variations, posing a significant challenge to the stability of image brightness. When surgical instruments or tissue areas (such as fat, serous membranes, or certain white tissues) enter the field of view and occupy a large proportion of the image, their high surface reflectivity can cause the automatic exposure control module of the endoscopic camera system to overreact, drastically increasing the overall image brightness to balance metering. This global increase in brightness can cause other important anatomical structures, especially areas with already low contrast such as vascular textures, to lose detail due to overexposure, making their outlines, directions, and boundaries with surrounding tissues blurred or even completely invisible. In such cases, surgeons cannot clearly identify the location of blood vessels, increasing the risk of misoperation (such as accidental injury to blood vessels or perforation of natural cavities) and potentially prolonging surgical time due to the inability to accurately determine anatomical layers, directly impacting the safety and efficiency of the surgery.

[0003] Currently, the traditional automatic exposure control algorithm commonly used in endoscopic systems can be considered the closest existing technology to this invention. This technology typically relies on feedback control based on brightness statistics (such as average brightness or brightness histogram) of the entire image frame or the central region. However, this approach suffers from blind global adjustments: it only focuses on the overall brightness level of the image and cannot distinguish the semantic content of different areas within the frame. When small, reflective tissue appears in the instrument's operating area, the exposure control system often weakens or ignores the influence of this area to ensure uniform brightness across the entire image, resulting in severe overexposure and loss of texture detail in that area.

[0004] Therefore, there is an urgent need in the field for an endoscopic image processing method that can overcome the above-mentioned shortcomings. This method can effectively identify local highlights in the instrument operation area and restore the vascular texture details obscured by overexposed areas to a certain extent, thereby providing surgeons with a continuous, stable, and detailed surgical field of view. Summary of the Invention

[0005] In view of the above, the purpose of this invention is to propose a method, device, and system for automatically adjusting the brightness of endoscopic videos. This method can effectively overcome the blind global adjustment defect in traditional automatic exposure control algorithms. By intelligently identifying and distinguishing the instrument operation area and key anatomical structures in the image, it can selectively adjust the brightness of locally bright areas, effectively suppressing local overexposure caused by reflective tissues. On this basis, it can restore important tissue information such as vascular textures that were obscured in overexposed areas, providing surgeons with a surgical field of view with stable brightness and clear details, thereby ensuring the safety and efficiency of the operation.

[0006] According to a first aspect of this disclosure, a method for automatically adjusting the brightness of an endoscopic video is provided, comprising:

[0007] The first endoscope image was acquired based on the automatic exposure mode;

[0008] The second endoscopic image is obtained by down-resolution processing based on the first endoscopic image;

[0009] Based on the second endoscopic image, the effective operating area of ​​the surgery is identified, and the overexposure value of the area is calculated.

[0010] Feedback adjustment is performed based on the comparison between the overexposure value of the effective operating area and the preset overexposure threshold, gradually adjusting the overexposure suppression until the overexposure suppression function is turned off.

[0011] A method for automatically adjusting the brightness of an endoscopic video according to at least one embodiment of the present disclosure, wherein the step of identifying the effective surgical area based on the second endoscopic image, calculating the overexposure value of the area, and implementing this method using a deep learning model includes:

[0012] The effective surgical operation area is identified using a U-Net++ network architecture, which includes a feature extraction module, a multi-scale feature fusion module, and a region segmentation module.

[0013] A detection model is built based on a densely connected convolutional neural network to calculate the degree of overexposure in a region. By capturing overexposed regions of different sizes through multi-scale receptive fields, the model outputs two detection results: a binary mask of the overexposed region and a numerical value of the degree of overexposure.

[0014] A method for automatically adjusting the brightness of an endoscopic video according to at least one embodiment of the present disclosure, wherein the step of identifying the effective surgical area and calculating the degree of overexposure of the area based on the second endoscopic image is implemented using conventional image processing methods, includes:

[0015] A pre-defined effective operating area is generated with the centroid of the second endoscope image as the center.

[0016] Obtain the image brightness value information of the effective operation area;

[0017] Based on the image brightness value information, a brightness distribution map is obtained;

[0018] Based on the brightness distribution map, the degree of overexposure in the region is obtained.

[0019] A method for automatically adjusting the brightness of an endoscopic video according to at least one embodiment of the present disclosure, comprising feedback adjustment based on a comparison of the overexposure value of the effective operating area with a preset overexposure threshold, gradually adjusting overexposure suppression until the overexposure suppression function is turned off, including:

[0020] If the overexposure value is greater than or equal to the preset overexposure threshold, then the current scene is overexposed. In this case, the overexposure suppression function will be enabled to adjust the brightness of the endoscopic image.

[0021] A method for automatically adjusting the brightness of an endoscopic video according to at least one embodiment of the present disclosure, wherein the current scene is overexposed, an overexposure suppression function is activated, and the brightness of the endoscopic image is adjusted, including:

[0022] Change the exposure mode from automatic exposure to intelligent metering;

[0023] Acquire third-endoscope images after changing the exposure mode and obtain the overexposure value;

[0024] If the current scene is determined to be overexposed based on the overexposure value, the brightness of the lighting source will be reduced to the brightness of the first light source according to the preset light source brightness step.

[0025] The fourth endoscope image was acquired after the brightness of the illumination source was changed, and the overexposure value was obtained;

[0026] If the overexposure value indicates that the current scene is overexposed, adjust the original Gamma1 curve to Gamma2 curve. Change the Gamma curve to better compress the brightness of the overexposed areas and improve the details and textures in the dark areas.

[0027] Acquire the fifth endoscope image after changing the Gamma curve, and obtain the overexposure value;

[0028] If the current scene is determined to be overexposed based on the overexposure value, the exposure brightness is reduced according to the preset exposure brightness increment.

[0029] Repeat the above steps until the overexposure suppression function is turned off, then change the exposure mode from smart metering to automatic exposure to acquire the endoscope video.

[0030] According to at least one embodiment of the present disclosure, the method for automatically adjusting the brightness of an endoscope video, after performing an overexposure suppression adjustment step, determines that there is no overexposure in the current scene based on the newly acquired endoscope image, and then continues to use the current exposure mode, illumination source brightness, Gamma curve, and exposure brightness combination to acquire the endoscope video.

[0031] A method for automatically adjusting the brightness of an endoscopic video according to at least one embodiment of the present disclosure, comprising feedback adjustment based on a comparison of the overexposure value of the effective operating area with a preset overexposure threshold, gradually adjusting overexposure suppression until the overexposure suppression function is turned off, including:

[0032] The overexposure value is less than the preset overexposure threshold, and there is no overexposure in the current scene. Continue to use the automatic exposure mode to acquire the endoscopic video.

[0033] According to a second aspect of this disclosure, an endoscope video brightness adjustment device is provided, and the method for automatically adjusting the brightness of the endoscope video includes:

[0034] The image acquisition unit is used to acquire endoscopic video images;

[0035] Image downsampling unit, used for subsampling processing of endoscopic images;

[0036] The effective operating area identification unit is used to identify the key anatomical areas that the surgeon is focused on and directly manipulates with instruments during the current surgical phase.

[0037] The region overexposure quantification unit is used to quantify the overexposure of the image output by the effective operating region identification unit.

[0038] The overexposure suppression enable unit is used to turn the overexposure suppression function on and off. When it is on, the brightness adjustment unit can adjust the brightness according to the overexposure level.

[0039] The brightness adjustment unit is used to adaptively adjust the exposure metering area, illumination source brightness, Gamma curve processing effect, and exposure brightness of the system based on the quantized overexposure value and the overexposure suppression switch, so as to remove overexposure in the effective operating area and restore image texture details.

[0040] The OSD prompt unit is used to display the current image brightness adjustment mode on the display.

[0041] According to at least one embodiment of the present disclosure, an endoscopic video brightness adjustment device includes an overexposure suppression enabling unit that enables and disables the overexposure suppression function, comprising:

[0042] Automatically activated when the overexposure level is determined to be overexposed in the current scene based on the results calculated by the regional overexposure quantization unit; or manually activated by the operator when overexposure is determined to be present.

[0043] The overexposure suppression function can be turned off manually by the surgeon based on their assessment of the surgical situation.

[0044] According to a third aspect of this disclosure, an endoscopic imaging system for automatically adjusting the brightness of an endoscopic video is provided, and the method for applying the method for automatically adjusting the brightness of the endoscopic video includes:

[0045] A light source device for adjusting the brightness of the illumination source based on the brightness value provided by the endoscope video brightness adjustment device;

[0046] A camera device, including an image sensor, is used to acquire image data of biological tissue under illumination conditions of a light source to generate endoscopic video;

[0047] An endoscope brightness adjustment device is used to detect the effective operating area based on the endoscope video captured by the camera system, and to adjust the image brightness according to the overexposure degree of the effective operating area.

[0048] Storage device for storing execution instructions, which, when executed by a processor, are used to implement the above-described method for automatically adjusting the brightness of the endoscope video;

[0049] A display device is used to display the endoscopic video and OSD prompts.

[0050] This invention provides a method, apparatus, and system for automatically adjusting the brightness of endoscopic video, and provides a method and apparatus for adaptively adjusting video brightness by judging the overexposure degree of the effective operating area of ​​the image. The beneficial effects of this invention are: (1) By specifically identifying the effective operating area of ​​the surgery and calculating the overexposure degree only for this area, the target of exposure adjustment directly serves the clinical surgical field of view, avoiding ineffective or erroneous brightness adjustments, and significantly improving the pertinence and practicality of the judgment; (2) This invention provides a step-by-step, progressive adjustment mechanism that can suppress overexposure more smoothly and effectively, while preserving the tissue texture and details in the dark areas of the image to the maximum extent; (3) The system of this invention actively provides visual prompts to the surgeon through the OSD function, and combines automatic suppression with manual switch control, which not only realizes intelligent automatic processing, but also respects the surgeon's subjective judgment and operating habits at different stages of surgery, realizes the optimization of human-machine collaboration, and improves the safety and efficiency of surgery. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a method for automatically adjusting the brightness of an endoscopic video according to an embodiment of the present invention;

[0053] Figure 2 This is a flowchart of a method for adjusting the brightness of an endoscopic image to suppress overexposure, according to an embodiment of the present invention.

[0054] Figure 3 is a schematic diagram of the brightness of an overexposed area in a Gamma curve compression image according to an embodiment of the present invention.

[0055] Figure 4 is a schematic diagram of a display image brightness adjustment mode according to an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0057] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0058] To keep the following description of the embodiments of the present invention clear and concise, detailed descriptions of some known functions and known components have been omitted.

[0059] Figure 1 This is a flowchart illustrating a method for automatically adjusting the brightness of an endoscopic video according to one embodiment of this disclosure. Figure 1 As shown, the method for automatically adjusting the brightness of an endoscope video according to embodiments of this disclosure may include:

[0060] Step S100: Acquire a first endoscope image based on the automatic exposure mode;

[0061] Specifically, acquiring the first endoscopic image may include: obtaining RAW data of the current frame image of the endoscopic video from the image sensor.

[0062] Step S200: Perform resolution reduction processing on the first endoscope image to obtain a second endoscope image;

[0063] Optionally, in the embodiments of this application, various image downsampling methods can be used to reduce resolution, including but not limited to: nearest neighbor interpolation, bilinear interpolation, or pooling. Through the above-mentioned resolution reduction processing, the computational complexity of subsequent image processing operations can be significantly reduced and processing efficiency improved while effectively preserving the image texture details.

[0064] In practice, appropriate downsampling methods and downsampling ratios can be selected based on the image quality requirements and computational resource constraints of the actual application scenario to achieve the optimal balance between preserving image details and computational efficiency.

[0065] Step S300: Based on the second endoscopic image, identify the effective operating area of ​​the surgery and calculate the degree of overexposure in the area;

[0066] In some embodiments, a deep learning model may be used to identify the effective operating area of ​​the surgery and calculate the degree of overexposure in the area.

[0067] In specific implementation, the effective operating area refers to the key anatomical area that the surgeon focuses on and directly manipulates instruments in during the current surgical phase. For example, in laparoscopic cholecystectomy, this area may be the Calot's triangle; in gastrointestinal surgery, this area may be the severed end of the intestine to be anastomosed. Optionally, this step can be implemented through the following technical solutions:

[0068] Step S310: Identify the effective surgical operation area using a trained convolutional neural network model;

[0069] Optionally, in this embodiment, the identification of the effective surgical operation area is implemented using a U-Net++ network architecture, which includes a feature extraction module, a multi-scale feature fusion module, and a region segmentation module.

[0070] The feature extraction module uses a pre-trained ResNet-34 as the encoder and extracts multi-level feature representations of the image through five stages of downsampling operations. Each stage contains multiple residual blocks. The number of input channels in the first stage is adapted to the color characteristics of the second endoscope image. It uses four channels, R, Gr, Gb, and B, as input and processes them using a specific preprocessing convolutional layer.

[0071] The multi-scale feature fusion module achieves deep fusion of encoder and decoder feature maps through dense skip connections. The scale feature fusion module employs an attention gating mechanism to adaptively learn the weight coefficients of features at different scales, highlighting semantic features relevant to the surgical region.

[0072] The region segmentation module adopts a progressive upsampling strategy, which gradually restores the spatial resolution of the feature map by combining bilinear interpolation and transposed convolution. The output layer uses the Sigmoid activation function to generate a pixel-level region probability distribution map.

[0073] The pre-trained U-Net++ network model performs semantic segmentation on the second endoscopic image, classifying each pixel in the image into either a "valid operating region" or a "background / non-operating region." The model extracts multi-scale features from the image through its deep network structure, learns and understands anatomical prior knowledge in complex surgical scenarios, accurately identifies and segments the valid operating region in the image, and finally outputs a probability distribution map of each pixel belonging to the valid operating region. Based on this probability distribution map, a corresponding binary mask is generated by setting an appropriate confidence threshold, thereby accurately defining the boundary range of the valid operating region.

[0074] Step S320: Based on the effective surgical operation area, an overexposed area detection is performed using a deep learning model;

[0075] Specifically, to avoid interference from high brightness in non-critical areas of the surgical scene (such as reflective instrument tips, saline mirror reflections, etc.), the analysis scope of overexposure detection is strictly limited to the effective operating area identified in step S310. Based on the identification of the effective surgical operating area, overexposure region detection is performed on this area to detect the presence of overexposure. The overexposure region detection model is constructed based on a densely connected convolutional neural network, capturing overexposure regions of different sizes through multi-scale receptive fields. The model outputs two detection results: a binary mask of the overexposure region and a numerical value indicating the degree of overexposure.

[0076] Specifically, the training convolutional neural network model is used to identify the effective surgical operation area and quantify the degree of overexposure. The training data construction adopts a multi-expert annotation mechanism, in which more than three clinical experts independently annotate the boundary of the effective surgical operation area, and the final annotation result is determined by majority voting. The annotation of overexposed areas is based on objective brightness measurement standards and verified by combining expert subjective evaluation.

[0077] Optionally, the deep learning model is trained end-to-end using a training dataset containing annotations of the effective surgical operation region and overexposed regions, employing a joint loss function to simultaneously optimize region segmentation accuracy and overexposed detection accuracy. The joint loss function consists of a weighted average of segmentation loss and detection loss, where the segmentation loss uses a set similarity loss function and the detection loss uses a cross-entropy loss function, calculated using equations 1-3 below:

[0078]

[0079]

[0080]

[0081] in, This represents the value of the i-th pixel of the real label, belonging to class C. This represents the probability of predicting the class C of the i-th pixel in the segmentation. This represents the segmentation loss during training. This represents the true label value of the i-th class. This indicates the predicted overexposure value. This indicates the detection loss during training. , They represent loss function and The weights of the loss function, where L represents the joint loss function used in model training.

[0082] In another embodiment, a conventional image processing method can be used to generate an exposure detection window based on the second endoscopic image, identify the effective operating area of ​​the surgery, and calculate the degree of overexposure in the area. Specifically, this includes the following steps:

[0083] Step S330: Using the centroid of the second endoscope image as the center, generate a pre-set effective operating area. In this embodiment, the effective operating area is a portion of the second endoscope image, and can be determined in the following ways:

[0084] Method 1: A circular window is generated with the centroid of the second endoscope image as the center and a preset radius parameter is used. The pixel data at the location of the second endoscope image covered by the circular window is the effective operating area. Optionally, in this embodiment, the effective operating area of ​​the second endoscope image can be obtained by means of Hough circle transform, adaptive thresholding, etc.

[0085] It should be noted that the centroid refers to the average value of the positions of all pixels within a region, representing the center of that region. Optionally, in this embodiment, the centroid position of the effective region is determined by Formula 4:

[0086]

[0087] in, Indicates the coordinates of the centroid in the second endoscopic image. Let x represent the pixel value corresponding to each pixel in the second endoscopic image, y represent the horizontal coordinate of the pixel, M represent the horizontal coordinate of the second endoscopic image, and N represent the vertical coordinate of the second endoscopic image.

[0088] Specifically, the effective operating area covered by the circular window Defined by the following equation 5:

[0089]

[0090] in, Here, R represents the pixel coordinates in the second endoscopic image, and R represents the preset radius parameter; the effective operating area is... That is, the set of pixel data extracted from the second endoscope image.

[0091] Method 2: Using the centroid of the second endoscopic image as the center, a rectangular window is generated according to pre-set image length and width parameters. The pixel data of the second endoscopic image location covered by the rectangular window constitutes the effective operating area. Specifically, the effective operating area covered by the rectangular window... Defined by the following formula 6:

[0092]

[0093] Where L represents a preset length parameter, W represents a preset width parameter, and the effective operating area is... That is, the set of pixel data extracted from the second endoscope image.

[0094] Step S340: Obtain the image brightness value information of the effective operation area. Specifically, locate a minimum repeating unit of 2x2 pixels in the effective operation area. This unit contains one red R pixel, one blue B pixel, and two green G pixels. The image brightness value in the minimum repeating unit is defined by the following formula 7:

[0095]

[0096] Where R represents the R color component in the smallest repeating unit. , These represent the two G color components in the smallest repeating unit, B represents the B color component in the smallest repeating unit, and Y represents the luminance component in the smallest repeating unit.

[0097] By traversing each 2x2 minimum repeating unit within the entire effective operating area and repeating step 7 of the above formula, the image brightness value information of the effective operating area is generated.

[0098] Step S350: Obtain a brightness distribution map based on the image brightness value information. Specifically, the method for obtaining the brightness distribution map includes the following steps:

[0099] Step S351: Perform threshold segmentation processing on the image brightness value information. Optionally, in this embodiment of the application, a fixed threshold method can be used to binarize each pixel in the image brightness value information to obtain a threshold segmented image.

[0100] Step S352: Apply the opening operation method to the thresholded segmentation image, and process it using the following formula 8 to eliminate small-area noise in the thresholded segmentation image, while smoothing the contour of the larger foreground region:

[0101]

[0102] in, This represents the first structural image after the opening operation. This represents the first structure element used for the opening operation. This refers to the erosion operation in morphology. This represents the dilation operation in morphology.

[0103] Step S353: Apply a closing operation to the threshold segmentation image using Equation 9 to fill in the small holes inside the region and obtain the overexposed connected regions in the brightness distribution map.

[0104]

[0105] in, This represents the second structure image after the closing operation. This represents the second structure element used for the closing operation. It should be noted that the first structure element... With the second structural element It is a rectangular structural element of size kxk. The two structural elements can be the same or different, and the specific size k is preset according to the resolution of the endoscope video.

[0106] Step S360: Based on the brightness distribution map, obtain the degree of overexposure in the region. Specifically, the degree of overexposure in the region is calculated using the following formula 10:

[0107]

[0108] Where E represents the overexposure value of the acquired region, m represents the horizontal coordinate of the second structural image, and n represents the vertical coordinate of the second structural image.

[0109] Step S400: Based on the regional overexposure value obtained in step S300, adaptively adjust the brightness of the endoscopic video;

[0110] Specifically, the adaptive adjustment of the endoscope video brightness includes the following steps:

[0111] Step S410: Determine the overexposure situation by comparing the overexposure value with the preset overexposure threshold. If the overexposure value is greater than or equal to the preset overexposure threshold, the current scene is overexposed; if the overexposure value is less than the preset overexposure threshold, the current scene is not overexposed.

[0112] Step S420: If there is no overexposure in the current scene, continue to use the automatic exposure mode to acquire the endoscope video;

[0113] Step S430: If the current scene is overexposed, perform overexposure suppression and adjust the brightness of the endoscopic image. Specifically, for example... Figure 2 As shown, it includes the following steps:

[0114] Step S431: Detect whether the overexposure suppression function is enabled. If the overexposure function is not enabled, continue to use the automatic exposure mode to acquire the endoscopic video. If the overexposure function is enabled, the exposure mode is changed to intelligent metering. The effective surgical instrument operation area is identified using a U-Net++ network architecture, and the size of the effective metering area is automatically divided. The network architecture used is the same as the network described in step S310 of Example 1, and will not be described in detail here.

[0115] Step S432: Acquire the third endoscopic image after changing the exposure mode, and use steps S200 and S300 to obtain the overexposure value to determine whether the current scene still has an overexposure phenomenon;

[0116] Step S433: If there is no overexposure in the current scene, continue to use the intelligent metering exposure mode to acquire the endoscope video; if there is still overexposure in the current scene, reduce the brightness of the illumination source to the brightness of the first light source according to the preset light source brightness step.

[0117] Step S434: Acquire the fourth endoscope image after changing the brightness of the illumination source, and use steps S200 and S300 to obtain the overexposure value to determine whether the current scene still has an overexposure phenomenon;

[0118] Step S435: If there is no overexposure in the current scene, continue to use the intelligent metering exposure mode and the brightness of the first light source to obtain the endoscope video; if there is still overexposure in the current scene, adjust the original Gamma1 curve to the Gamma2 curve, as shown in Figure 3. The left figure (a) is the original Gamma1 curve, and the right figure (b) is the Gamma2 curve. Changing the Gamma curve can better compress the brightness of the image in the bright overexposed areas and improve the details and textures in the dark areas.

[0119] Step S436: Acquire the fifth endoscopic image after changing the Gamma curve, and use steps S200 and S300 to obtain the overexposure value to determine whether the current scene still has an overexposure phenomenon;

[0120] Step S437: If there is no overexposure in the current scene, continue to use the intelligent metering exposure mode, the brightness of the first light source, and the Gamma2 curve to acquire the endoscope video; if there is still overexposure in the current scene, reduce the exposure brightness according to the preset exposure brightness step.

[0121] Step S438: Determine whether the overexposure suppression function is turned off. If the overexposure suppression function is turned off, change the exposure mode from intelligent metering to automatic exposure to acquire the endoscope video. If the overexposure suppression function is not turned off, repeat steps S432 to S437 for processing.

[0122] Based on the same inventive concept, the video brightness adjustment method described in this application can be implemented by an endoscopic video brightness adjustment device. The endoscopic video brightness adjustment device of this disclosure embodiment may include:

[0123] The image acquisition unit is used to acquire endoscopic video images;

[0124] Image downsampling unit, used for subsampling processing of endoscopic images;

[0125] The effective operating area identification unit is used to identify the key anatomical areas that the surgeon is focused on and directly manipulates with instruments during the current surgical phase.

[0126] The region overexposure quantification unit is used to quantify the overexposure of the image output by the effective operating region identification unit.

[0127] The overexposure suppression enable unit is used to turn the overexposure suppression function on and off. When enabled, the brightness adjustment unit can adjust the brightness according to the overexposure level. In this embodiment, it can be enabled in two ways: automatically when the current scene is determined to be overexposed based on the result calculated by the regional overexposure quantification unit; or manually by the surgeon when overexposure is detected. The overexposure suppression function is disabled by the surgeon manually by pressing a button based on the surgical situation.

[0128] The brightness adjustment unit is used to adaptively adjust the exposure metering area, illumination source brightness, Gamma curve processing effect, and exposure brightness of the system based on the quantized overexposure value and the overexposure suppression switch, so as to remove overexposure in the effective operating area and restore image texture details.

[0129] The OSD prompt unit is used to display the current image brightness adjustment mode on the display. Specifically, when the endoscope video brightness adjustment unit detects overexposure, as shown in Figure 4(a), the display flashes the message "It is recommended to enable overexposure suppression function"; as shown in Figure 4(b), when the overexposure suppression switch is detected to be enabled, the display flashes the message "Enable overexposure suppression".

[0130] Based on the same inventive concept as the above-described method embodiments, this application also provides an endoscopic imaging system for automatically adjusting the brightness of endoscopic video. The endoscopic imaging system for automatically adjusting the brightness of endoscopic video according to this disclosure embodiment may include:

[0131] A light source device for adjusting the brightness of the illumination source based on the brightness value provided by the endoscope video brightness adjustment device;

[0132] A camera device, including an image sensor, is used to acquire image data of biological tissue under illumination conditions of a light source to generate endoscopic video;

[0133] An endoscope brightness adjustment device is used to detect the effective operating area based on the endoscope video captured by the camera system, and to adjust the image brightness according to the overexposure degree of the effective operating area.

[0134] Storage device for storing execution instructions, which, when executed by a processor, are used to implement the above-described method for automatically adjusting the brightness of the endoscope video;

[0135] A display device is used to display the endoscopic video and OSD prompts.

[0136] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This application is described with reference to flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for automatically adjusting the brightness of an endoscopic video, characterized in that, Include: The first endoscope image was acquired based on the automatic exposure mode; The second endoscopic image is obtained by down-resolution processing based on the first endoscopic image; Based on the second endoscopic image, the effective operating area of ​​the surgery is identified, and the overexposure value of the area is calculated. Feedback adjustment is performed based on the comparison between the overexposure value of the effective operating area and the preset overexposure threshold, gradually adjusting the overexposure suppression until the overexposure suppression function is turned off.

2. The method for automatically adjusting the brightness of an endoscope video according to claim 1, characterized in that, The step of identifying the effective surgical area based on the second endoscopic image, calculating the overexposure level of the area, and implementing this using a deep learning model includes: The effective surgical operation area is identified using a U-Net++ network architecture, which includes a feature extraction module, a multi-scale feature fusion module, and a region segmentation module. A detection model is built based on a densely connected convolutional neural network to calculate the degree of overexposure in a region. By capturing overexposed regions of different sizes through multi-scale receptive fields, the model outputs two detection results: a binary mask of the overexposed region and a numerical value of the degree of overexposure.

3. The method for automatically adjusting the brightness of an endoscopic video according to claim 1, characterized in that, The step of identifying the effective surgical area and calculating the overexposure level of the area based on the second endoscopic image, using traditional image processing methods, includes: A pre-defined effective operating area is generated with the centroid of the second endoscope image as the center. Obtain the image brightness value information of the effective operation area; Based on the image brightness value information, a brightness distribution map is obtained; Based on the brightness distribution map, the degree of overexposure in the region is obtained.

4. The method for automatically adjusting the brightness of an endoscope video according to claim 1, characterized in that, Feedback adjustment is performed based on a comparison between the overexposure value of the effective operating area and a preset overexposure threshold, gradually adjusting overexposure suppression until the overexposure suppression function is turned off, including: If the overexposure value is greater than or equal to the preset overexposure threshold, then the current scene is overexposed. In this case, the overexposure suppression function will be enabled to adjust the brightness of the endoscopic image.

5. The method for automatically adjusting the brightness of an endoscope video according to claim 4, characterized in that, The current scene is overexposed. The overexposure suppression function is activated to adjust the brightness of the endoscopic image, including: Change the exposure mode from automatic exposure to intelligent metering; Acquire third-endoscope images after changing the exposure mode and obtain the overexposure value; If the current scene is determined to be overexposed based on the overexposure value, the brightness of the lighting source will be reduced to the brightness of the first light source according to the preset light source brightness step. The fourth endoscope image was acquired after the brightness of the illumination source was changed, and the overexposure value was obtained; If the overexposure value indicates that the current scene is overexposed, adjust the original Gamma1 curve to Gamma2 curve. Change the Gamma curve to better compress the brightness of the overexposed areas and improve the details and textures in the dark areas. Acquire the fifth endoscope image after changing the Gamma curve, and obtain the overexposure value; If the current scene is determined to be overexposed based on the overexposure value, the exposure brightness is reduced according to the preset exposure brightness increment. Repeat the above steps until the overexposure suppression function is turned off, then change the exposure mode from smart metering to automatic exposure to acquire the endoscope video.

6. The method for automatically adjusting the brightness of an endoscopic video according to claim 5, characterized in that, After performing the overexposure suppression adjustment step, if it is determined that there is no overexposure in the current scene based on the newly acquired endoscopic image, the current exposure mode, illumination source brightness, Gamma curve, and exposure brightness combination will continue to be used to acquire endoscopic video.

7. The method for automatically adjusting the brightness of an endoscopic video according to claim 1, characterized in that, Feedback adjustment is performed based on a comparison between the overexposure value of the effective operating area and a preset overexposure threshold, gradually adjusting overexposure suppression until the overexposure suppression function is turned off, including: The overexposure value is less than the preset overexposure threshold, and there is no overexposure in the current scene. Continue to use the automatic exposure mode to acquire the endoscopic video.

8. An endoscopic video brightness adjustment device, employing the method for automatically adjusting endoscopic video brightness as described in claims 1-7, characterized in that, include: The image acquisition unit is used to acquire endoscopic video images; Image downsampling unit, used for subsampling processing of endoscopic images; The effective operating area identification unit is used to identify the key anatomical areas that the surgeon is focused on and directly manipulates with instruments during the current surgical phase. The region overexposure quantification unit is used to quantify the overexposure of the image output by the effective operating region identification unit. The overexposure suppression enable unit is used to turn the overexposure suppression function on and off. When it is on, the brightness adjustment unit can adjust the brightness according to the overexposure level. The brightness adjustment unit is used to adaptively adjust the exposure metering area, illumination source brightness, Gamma curve processing effect, and exposure brightness of the system based on the quantized overexposure value and the overexposure suppression switch, so as to remove overexposure in the effective operating area and restore image texture details. The OSD prompt unit is used to display the current image brightness adjustment mode on the display.

9. An endoscopic video brightness adjustment device according to claim 8, characterized in that, The overexposure suppression enabling unit enables and disables the overexposure suppression function, including: Automatically activated when the overexposure level is determined to be overexposed in the current scene based on the results calculated by the regional overexposure quantization unit; or manually activated by the operator when overexposure is determined to be present. The overexposure suppression function can be turned off manually by the surgeon based on their assessment of the surgical situation.

10. An endoscopic imaging system for automatically adjusting the brightness of an endoscopic video, employing the method for automatically adjusting the brightness of an endoscopic video according to claims 1-7, characterized in that, include: A light source device for adjusting the brightness of the illumination source based on the brightness value provided by the endoscope video brightness adjustment device; A camera device, including an image sensor, is used to acquire image data of biological tissue under illumination conditions of a light source to generate endoscopic video; An endoscope brightness adjustment device is used to detect the effective operating area based on the endoscope video captured by the camera system, and to adjust the image brightness according to the overexposure degree of the effective operating area. Storage device for storing execution instructions, which, when executed by a processor, are used to implement the above-described method for automatically adjusting the brightness of the endoscope video; A display device is used to display the endoscopic video and OSD prompts.

Images