Dynamic luminance calibration (DLC) method

Abstract

The present invention relates to the technical field of endoscope photography, and specifically relates to a dynamic luminance calibration (DLC) method, comprising the following steps: performing automatic photometry on a dynamic video; performing automatic dimming on the dynamic video which has been subjected to photometry, wherein the automatic photometry and the automatic dimming fall within the scope of automatic exposure; and performing dynamic luminance adjustment on the dimmed video, such that the picture luminance of the video remains in a rational state. In the present invention, the DLC method is designed, such that the exposure and picture luminance can be quickly and effectively adjusted; during exposure adjustment, the image luminance quickly converges to the vicinity of a target luminance, a convergence process is smooth, the visual experience is comfortable, the luminance is stable after convergence is completed, and there is no breathing effect; when the picture is relatively dark, the luminance can be dynamically improved; and during dynamic luminance adjustment, the picture should be kept at an enough contrast, such that the picture luminance remains in a rational state.

Description

[Corrected according to Rule 91 23.04.2025] A dynamic luminance calibration method (D.L.C.) TECHNICAL FIELD

[0001] [Corrected according to Rule 91 23.04.2025] The present invention relates to the field of endoscopic imaging technology, in particular to a dynamic luminance calibration method (D.L.C.). BACKGROUND

[0002] In the field of endoscopic imaging, the luminance stability of the picture is highly required. On the one hand, stable pictures can reduce the visual fatigue of doctors, and on the other hand, stable pictures can make doctors more calm in the face of surgical scenes.

[0003] For an image sensor, the image luminance (LUMA) is determined by the exposure time (ET) and the analog gain amplification factor (GAIN). The working principle of the image sensor determines that the exposure time (ET) and the analog gain amplification factor (GAIN) are substantially linearly related to the image luminance, and the image luminance can be represented as LUMA=k*GAIN*ET

[0004] Therefore, the automatic exposure algorithm needs to obtain the current frame image luminance and calculate the exposure time and analog gain amplification factor required for the next frame.

[0005] Due to the existence of a long and narrow channel in the medical endoscope application scene, after the automatic exposure algorithm is adjusted, there may still be dark areas in the picture, which leads to poor performance of the dark areas of the picture, thereby leading to the following common problems:

[0006] (1) unstable dimming, leading to non-linear luminance change between frames during dimming process, picture flickering.

[0007] (2) slow dimming speed, leading to long-time overexposure or underexposure of the picture.

[0008] (3) Since the actual exposure register value is discrete, it may lead to inaccurate dimming amplitude, unstable picture luminance during dimming process, and automatic exposure cannot be adjusted to the appropriate luminance, picture bright and dark.

[0009] (4) The picture at the proximal end of the long and narrow cavity is too bright, and the picture at the distal end is relatively dark. SUMMARY

[0010] [According to the rules 91 correction 23.04.2025] In view of this, the purpose of the present application is to propose a dynamic luminance calibration method (D.L.C., Dynamic Luminance Calibration) to solve the problem that due to the existence of long and narrow channels in the application scene of medical endoscopes, after the automatic exposure algorithm is adjusted, there may still be dark areas in the picture, resulting in poor performance of the dark areas of the picture.

[0011] [According to the rules 91 correction 23.04.2025] Based on the above purpose, the present application provides a dynamic luminance calibration method (D.L.C., Dynamic Luminance Calibration), comprising the following steps:

[0012] S1, automatic light measurement of dynamic video;

[0013] S2, automatic light adjustment of the dynamic video after light measurement, wherein automatic light measurement and automatic light adjustment belong to the category of automatic exposure;

[0014] S3, dynamic luminance adjustment of the light-adjusted video, so that the video picture brightness is always in a reasonable state;

[0015] Step S1 includes image valid area detection and automatic light measurement, and the image valid area detection specifically includes the following sub-steps:

[0016] S1.1, selecting valid pixel points in the image through a threshold parameter TH_valid;

[0017] S1.2, for each pixel point (x, y) in the image I, if I c (x, y) > TH_valid, the pixel point (x, y) is considered as a valid pixel point, wherein c represents the RGB three channels, and the ratio of the number of valid pixel points to the total number of pixel points is recorded as ratio, wherein the value of TH_valid is set to 8 by default;

[0018] Automatic light measurement specifically includes the following sub-steps:

[0019] S1.3, divide the image into several blocks, record the total number of blocks as total_block, for each block i with a size of s*s, calculate its average brightness L i , the formula of the average brightness L i is as follows:

[0020] Wherein, L i (x, y) represents the brightness of the pixel point in the block, R i (x, y), G i(x, y), B i (x, y) respectively represent the value of point (x, y) in RGB three channels, represents rounding down;

[0021] S1.4, calculate the number of selected blocks; block = ratio * total_block;

[0022] S1.5, sort the average brightness of all blocks from high to low, and the light measurement result is;

[0023] wherein, L j represents the average brightness of the jth selected block, S is a tolerance parameter, which can be set to any value within 0-1

[0024] Preferably, for the Nth image in the video stream, the automatic light adjustment module input conditions are: current image brightness Y N , last frame image brightness Y N-1 , target image brightness TY, exposure time ET N , ET N-1 , analog gain amplification GAIN N , GAIN N-1 , in S2, whether the light adjustment process is needed is judged as follows:

[0025] S2.1, when abs(TY-Y N )<range, that is, the difference between the current image brightness and the target brightness is small, the exposure parameter remains unchanged, and the automatic light adjustment is exited, wherein abs() is an absolute value function, and range is a threshold constant;

[0026] S2.2 when abs(TY-Y N )>range, then enter the light adjustment process.

[0027] Preferably, the light adjustment process in step S2.2 includes correcting the exposure model and calculating the exposure parameter, and the correcting the exposure model specifically includes the following sub-steps:

[0028] S2.21, according to the working principle of the image sensor, under the condition that the scene does not change, there is the following relationship: GAIN N+1 -GAIN N =K1*(Y N+1 -Y N ) ET N+1 -ET N =K2*(Y N+1 -Y N );

[0029] S2.22, set K as initial value, K value needs to be corrected in time for changing scene, in dimming process, when one of GAIN or ET changes, recalculate K value of current frame according to brightness change, when both change, K value is shown as follows:

[0030] S2.23, in order to improve robustness of automatic exposure in endoscope movement, K value can take weighted value of current frame and previous several frames, after estimated K value is obtained, according to difference between current frame brightness and target brightness, change of exposure parameter can be estimated.

[0031] Preferably, calculating exposure parameter specifically includes following sub-steps:

[0032] S2.24, introduce n damping coefficients p to further adjust dimming intermediate process, damping coefficient p can also be used to adjust dimming speed, damping coefficient p is shown as follows:

[0033] S2.25, since increasing exposure time is helpful to improve signal-to-noise ratio, exposure time should be used preferentially in dimming process, when exposure time is insufficient to meet brightness requirement, then adjust analog gain amplification multiple, when GAIN=1, GAIN is not used, adjust ET: ET N+1 = ET N + ΔET

[0034] Calculate and issue actual register value Reg ET , end current frame dimming.

[0035] Preferably, when GAIN≠1, adjust GAIN: GAIN N+1 = GAIN N + ΔGAIN

[0036] At this time, calculate actual register value Reg GAIN , when ΔGAIN is less than threshold th_g, further correct through D gain ;

[0037] Let difference between actual register value corresponding amplification multiple and actual amplification multiple of previous frame be A, then A*abs(D gain -1)=p*K1*(TY-Y N )

[0038] Wherein D gain >0, according to this relationship, adjusted D gain can be calculated, finally calculate Reg ET and Reg GAINEnd dimming and multiply D on N+1 frame image gain to get N+1 frame image.

[0039] Preferably, in step S3, the following sub-steps are included:

[0040] S3.1, dynamically determine whether to adjust the picture;

[0041] S3.2, adaptively brighten the dark area;

[0042] S3.3, dynamically enhance the contrast.

[0043] Preferably, step S3.1 determines whether to adjust the picture, and specifically includes the following sub-steps:

[0044] S3.11, first convert the image from the RGB domain to the YUV domain, and the formula is as follows: Y = 0.1826 * R + 0.6142 * G + 0.062 * B + 16 U = -0.1006 * R - 0.3386 * G + 0.4392 * B + 128 V = 0.4392 * B - 0.3989 * G - 0.0403 * B + 128;

[0045] S3.12, perform histogram statistics on the Y channel, and continue to use the idea of "image valid area detection in step S1" when performing histogram statistics, that is, only pixels with a brightness greater than TH_valid are counted, and after obtaining the brightness histogram Hist_L, sum the number of pixels of each gray scale of Hist_L to obtain Hist_L_sum, and then divide the number of pixels of each gray scale of Hist_L by Hist_L_sum to obtain the weight of each gray scale, and the formula is as follows: Hist_L_w j = Hist_L j / Hist_L_sum;

[0046] S3.13, weight and sum the weights and gray scale values to obtain the weighted gray scale value of the image, and the formula is as follows,

[0047] In the histogram Hist_L, the number of pixels with a gray scale value less than TH_GT is left_num, and the number of pixels with a gray scale value greater than or equal to TH_GT is right_num, where TH_GT is 90 by default, and r_l_ratio is the ratio of the number of pixels to the right of TH_GT to the number of pixels to the left of TH_GT, and the formula is as follows: r_l_ratio = right_num / left_num;

[0048] S3.14, when gray_avg is greater than or equal to TH_GT, it is considered that the picture is normal and no change is needed;

[0049] When r_l_ratio is greater than or equal to 1, it is considered that the picture is normal and no change is needed. Except for the above two cases, it is considered that the picture is generally dark and dynamic brightening is needed.

[0050] Preferably, step S3.2, adaptive dark area brightening specifically includes the following sub-steps:

[0051] S3.21, when the value of gray_avg is low, it is considered that the overall brightness of the picture is low and more enhancement of the dark area is needed. However, at this time, the state of double peaks may appear, so r_l_ratio ratio needs to be introduced. When the ratio is closer to 1, it indicates that the overall distinction of the picture is good and the overall distribution of light and dark is appropriate, so no dark area brightening is needed.

[0052] S3.22, a curve dynamically adjusted according to the values of gray_avg and r_l_ratio is designed, and the formula is as follows: f(x) = (x b *c d ) / (x b *c d +(1-x) d *a b )

[0053] Where x is the gray scale value, the value range is [0, 255], f(x) represents that the original gray scale value will be mapped to a new gray scale value, where a is 0.3 and d is 1.

[0054] c is dynamically adjusted according to gray_avg, and the formula is as follows: c_i = min((TH_GT-gray_avg) / (TH_GT-40), 1) c = 0.3 + c_i*0.3.

[0055] b is dynamically adjusted according to r_l_ratio and gray_avg, where min represents the smaller value of the two values, and the formula is as follows: b_i = 1 - c_i*(1-new_r_l_ratio) b = min(0.35+0.65*b_i, 1);

[0056] S3.23, apply the new gray scale value of f(x) to the Y channel of the YUV image to obtain a new brightened YUV_new image.

[0057] Preferably, step S3.3, dynamic contrast enhancement includes the following sub-steps:

[0058] S3.31, first convert YUV_new to RGB_new, that is, convert the YUV domain image to the RGB domain;

[0059] S3.32, using adaptive linear contrast mode to enhance RGB_new, the specific enhancement formula is as follows:

[0060] Wherein, m represents a certain position of the pixel, n represents a certain color channel in RGB, c_f is a contrast enhancement factor, and the formula of c_f is as follows: c_f=1.4-0.4*b_i

[0061] Wherein, the clip operation means that when the calculation result is less than 0, it is truncated to 0, and when it is greater than 255, it is truncated to 255, wherein That is, the final result image.

[0062] Preferably, in step S3.31, the image in YUV domain is converted to RGB domain, and the formula is as follows: R=1.1644*(Y-16)+1.7928*(V-128) G=1.1644*(Y-16)-0.2133*(U-127)-0.533*(V-128) B=1.1644*(Y-16)+2.1124*(U-128).

[0063] The beneficial effects of the present application are as follows:

[0064] I. The effective area detection is increased, the adaptability of automatic exposure in different endoscope scenes is improved, and the automatic light measurement result is more stable.

[0065] II. The light adjustment process can correct the exposure theoretical model in real time, thereby ensuring the accuracy of the light adjustment model and improving the robustness of the automatic exposure in the endoscope movement process.

[0066] III. The light adjustment process introduces a damping coefficient to ensure the light adjustment stability and light adjustment speed. Since increasing the exposure time helps to improve the signal-to-noise ratio, the exposure time should be used preferentially in the light adjustment process. When the exposure time is insufficient to meet the brightness requirement, the analog gain amplification multiple is adjusted, thereby improving the accuracy of light adjustment.

[0067] IV. For the scene of the long and narrow cavity of the endoscope image, a dynamic brightness adjustment algorithm is designed. When the picture is dark, the picture dark area brightness can be adaptively adjusted. When the brightness is dynamically adjusted, the picture still maintains a high contrast ratio using the adaptive contrast method.

[0068] V. In summary, the method can quickly and effectively adjust the exposure and picture brightness. In the process of adjusting the exposure, the image brightness quickly converges to the target brightness, and the convergence process is smooth, comfortable to watch, and the brightness is stable after convergence without breathing effect. When the picture is dark, the brightness can be dynamically improved. When the brightness is dynamically adjusted, the picture should maintain sufficient contrast ratio, so that the picture brightness is always in a reasonable state. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only illustrate the embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0070] Fig. 1 is a general flow chart of the embodiments of the present application;

[0071] Fig. 2 is an automatic exposure flow chart of the embodiments of the present application;

[0072] Fig. 3 is an automatic light adjustment flow chart of the embodiments of the present application;

[0073] Fig. 4 is a dynamic brightness adjustment flow chart of the embodiments of the present application;

[0074] Fig. 5 is an original image of the embodiments of the present application without using dynamic brightness adjustment;

[0075] Fig. 6 is a result image of the embodiments of the present application using adaptive dark area brightening;

[0076] Fig. 7 is a result image of the embodiments of the present application using both adaptive dark area brightening and dynamic contrast enhancement. DETAILED DESCRIPTION

[0077] In order to make the objectives, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to specific embodiments and drawings.

[0078] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the common meanings understood by those skilled in the art to which the present application belongs. The terms "first", "second" and similar terms used in the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar terms mean that the components or objects before the terms cover the components or objects listed after the terms and their equivalents, and do not exclude other components or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like only represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0079] [Corrected according to Rule 91 23.04.2025] As shown in FIG. 1, FIG. 2, FIG. 3, FIG. 4, FIG. 5, FIG. 6, FIG. 7, a dynamic luminance calibration method (D.L.C., Dynamic Luminance Calibration) comprises the following steps:

[0080] S1, automatic photometry is performed on the dynamic video.

[0081] Step S1 includes image valid region detection and automatic photometry, and the image valid region detection specifically comprises the following sub-steps:

[0082] S1.1, selecting valid pixel points in the image by a threshold parameter TH_valid.

[0083] S1.2, for each pixel point (x, y) in the image I, if I c (x, y) > TH_valid, the pixel point (x, y) is considered as a valid pixel point, wherein TH_valid is set to 8 by default, and the ratio of the number of valid pixel points to the total number of pixel points is denoted as ratio.

[0084] Automatic photometry specifically comprises the following sub-steps:

[0085] S1.3, the image is divided into a plurality of blocks, each block can have a size of 8*8, 16*16 or 32*32, and the total number of blocks is denoted as total_block. For each block i with a size of s*s, the average luminance L i of the block is calculated. i The formula of the average luminance L

[0086] wherein L i (x, y) represents the luminance of the pixel point in the block, R i (x, y), G i (x, y) and B i (x, y) represent the values of the point (x, y) in the RGB three channels respectively, and represents the floor function. The calculation method has been verified to have strong robustness for bleeding scenes.

[0087] S1.4, the number of selected blocks is calculated; block = ratio*total_block.

[0088] S1.5, the average luminances of all blocks are sorted from high to low, and the photometry result is L

[0089] wherein L jS represents the average brightness of the selected jth block, and S is a tolerance parameter, which can be set to any value within 0-1.

[0090] S2, automatically adjusting the light of the dynamic video after the light measurement, wherein the automatic light measurement and the automatic light adjustment belong to the category of automatic exposure.

[0091] For the Nth image in the video stream, the input conditions of the automatic light adjustment module are: the current image brightness Y N , the last frame image brightness Y N-1 , the target image brightness TY, the exposure time ET N , ET N-1 , the analog gain amplification GAIN N , GAIN N-1 , and the judgment in S2 whether the light adjustment process is needed is as follows:

[0092] S2.1, when abs(TY-Y N )<range, that is, the difference between the current image brightness and the target brightness is small, the exposure parameter remains unchanged, and the automatic light adjustment is exited, wherein abs() is an absolute value function, and range is a threshold constant.

[0093] S2.2, when abs(TY-Y N )>range, the light adjustment process is entered.

[0094] The light adjustment process in step S2.2 includes correcting the exposure model and calculating the exposure parameter, and the correction of the exposure model specifically includes the following sub-steps:

[0095] S2.21, according to the working principle of the image sensor, under the condition that the scene does not change, there is the following relationship: GAIN N+1 -GAIN N =K1*(Y N+1 -Y N ) ET N+1 -ET N =K2*(Y N+1 -Y N ).

[0096] S2.22, set K to the initial value, for the constantly changing scene, the K value needs to be corrected in time, in the light adjustment process, when one of GAIN or ET changes, the K value of the current frame is recalculated according to the brightness change amount, and when both change, the K value is as follows:

[0097] S2.23, in order to improve the robustness of automatic exposure during endoscope movement, the K value can take the weighted value of the current frame and several previous frames, and after obtaining the estimated K value, the change amount of the exposure parameter can be estimated according to the difference between the current frame brightness and the target brightness.

[0098] The calculation of the exposure parameter specifically includes the following sub-steps:

[0099] Some threshold values should be set in the dimming process to limit the adjustment range of the exposure parameter, so as to avoid excessive or insufficient adjustment, for example, the maximum ΔET / ET N and the maximum ΔGAIN / GAIN N .

[0100] S2.24, n damping coefficients p are introduced to further adjust the dimming intermediate process, and the dimming speed can also be adjusted through the damping coefficients, and the damping coefficient p is represented as follows:

[0101] S2.25, since increasing the exposure time helps to improve the signal-to-noise ratio, the exposure time should be used first in the dimming process, and when the exposure time is insufficient to meet the brightness requirement, the analog gain amplification factor is adjusted, and when GAIN=1, GAIN is not enabled, and ET is adjusted: ET N+1 = ET N + ΔET

[0102] The actual register value Reg ET is calculated and issued, and the dimming of the current frame is ended.

[0103] When GAIN≠1, GAIN is adjusted: GAIN N+1 = GAIN N + ΔGAIN

[0104] At this time, the actual register value Reg GAIN is calculated, and when ΔGAIN is less than the threshold th_g, D gain is further corrected.

[0105] Let the difference between the actual amplification factor corresponding to the actual register value and the actual amplification factor of the previous frame be A, then A*abs(D gain -1)=p*K1*(TY-Y N )

[0106] where D gain >0, according to this relationship, the adjusted D gain can be calculated, and finally Reg ET and Reg GAIN are calculated to end the dimming, and D gain, to obtain N+1 frames of images.

[0107] S3, dynamically adjusting the brightness of the video after dimming, so that the brightness of the video picture is always in a reasonable state.

[0108] In step S3, the following sub-steps are included:

[0109] S3.1, dynamically determine whether the picture is adjusted.

[0110] Step S3.1 determines whether to adjust the picture, which includes the following sub-steps:

[0111] S3.11, first convert the image from RGB domain to YUV domain, the formula is as follows: Y = 0.1826 * R + 0.6142 * G + 0.062 * B + 16 U = -0.1006 * R - 0.3386 * G + 0.4392 * B + 128 V = 0.4392 * B - 0.3989 * G - 0.0403 * B + 128.

[0112] S3.12, histogram statistics are performed on the Y channel. In histogram statistics, the idea of "image valid area detection in step S1" is continued, that is, only the pixels with brightness greater than TH_valid are counted, and after obtaining the brightness histogram Hist_L, the sum of the pixel numbers of each gray scale of Hist_L is obtained to obtain the weight of each gray scale, and the formula is as follows: Hist_L_w j = Hist_L j / Hist_L_sum.

[0113] S3.13, the weight is weighted and summed with each gray scale value to obtain the weighted gray value of the image, and the formula is as follows:

[0114] In the histogram Hist_L, the number of pixels with gray scale value less than TH_GT is left_num, and the number of pixels with gray scale value greater than or equal to TH_GT is right_num, wherein TH_GT is 90 by default, and r_l_ratio is the ratio of the number of pixels on the right side of TH_GT to the number of pixels on the left side of TH_GT, and the formula is as follows: r_l_ratio = right_num / left_num.

[0115] S3.14, when gray_avg is greater than or equal to TH_GT, it is considered that the picture is normal and no change is needed;

[0116] When r_l_ratio is greater than or equal to 1, it is considered that the picture is normal and no change is needed. Except for the above two cases, it is considered that the picture is generally dark and dynamic brightening is needed.

[0117] S3.2, adaptive dark area brightening.

[0118] Step S3.2, adaptive dark area brightening specifically includes the following sub-steps:

[0119] S3.21, when the value of gray_avg is low, it is considered that the overall brightness of the picture is low and more enhancement of the dark area is needed. However, at this time, the state of double peaks may appear, so r_l_ratio ratio needs to be introduced. When the ratio is closer to 1, it indicates that the overall differentiation of the picture is good and the overall distribution of light and dark is appropriate, and dark area brightening is not needed.

[0120] S3.22, a curve dynamically adjusted according to gray_avg and r_l_ratio values is designed, and the formula is as follows: f(x) = (x b *c d ) / (x b *c d +(1-x) d *a b )

[0121] Where x is the gray scale value, the value range is [0, 255], f(x) represents that the original gray scale value will be mapped to a new gray scale value, where a is 0.3 and d is 1.

[0122] c is dynamically adjusted according to gray_avg, and the formula is as follows: c_i = min((TH_GT-gray_avg) / (TH_GT-40),1) c = 0.3 + c_i*0.3.

[0123] b is dynamically adjusted according to r_l_ratio and gray_avg, where min represents the smaller value of the two values, and the formula is as follows: b_i = 1 - c_i*(1-new_r_l_ratio) b = min(0.35+0.65*b_i,1).

[0124] S3.23, apply the new gray scale value of f(x) to the Y channel of the YUV image to obtain a new brightened YUV_new image.

[0125] S3.3, dynamic contrast enhancement.

[0126] After using the enhancement curve for certain enhancement, the contrast of the picture will generally be reduced, at which time the method of contrast enhancement is needed to restore part of the contrast to improve the viewing of the picture.

[0127] In step S3.3, dynamic contrast enhancement includes the following sub-steps:

[0128] S3.31, first convert YUV_new to RGB_new, that is, convert the image in YUV domain to RGB domain.

[0129] In step S3.31, the formula for converting the image in YUV domain to RGB domain is as follows: R=1.1644*(Y-16)+1.7928*(V-128) G=1.1644*(Y-16)-0.2133*(U-127)-0.533*(V-128) B=1.1644*(Y-16)+2.1124*(U-128).

[0130] S3.32, use adaptive linear contrast method to enhance RGB_new, and the specific enhancement formula is as follows:

[0131] Wherein, Wherein, m represents a pixel at a certain position, n represents a color channel in RGB, c_f is a contrast enhancement factor, and the formula of c_f is as follows: c_f=1.4-0.4*b_i

[0132] Wherein, the clip operation means that when the calculation result is less than 0, it is truncated to 0, and when it is greater than 255, it is truncated to 255, wherein That is, the final result image.

[0133] Those skilled in the art should understand that the above discussion of any embodiment is only exemplary and is not intended to imply that the scope of the present application is limited to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in details.

[0134] Embodiments of the present application are intended to cover all such alternatives, modifications and variations falling within the broad scope of the appended claims. Therefore, any omission, modification, equivalent replacement, improvement, etc. made in the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. [Corrected 23.04.2025 according to Rule 91] A method for dynamic luminance stabilization (DLC, Dynamic Luminance Calibration), characterized in that The following steps are involved: S1. Automatically measure the light of the dynamic video; S2. Automatically adjust the light of the measured dynamic video, where automatic light measurement and automatic light adjustment belong to the category of automatic exposure; S3. Dynamically adjust the brightness of the video after light adjustment to keep the brightness of the video image in a reasonable state at all times; Step S1 includes image valid area detection and automatic light measurement, and the image valid area detection specifically includes the following sub-steps: S1.

1. Select the valid pixel points in the image through the threshold parameter TH_valid; S1.

2. For each pixel point (x, y) in the image I, if I c (x, y)>TH_valid, then the pixel point (x, y) is considered a valid pixel point, where c represents the RGB three channels, and the ratio of the number of valid pixel points to the total number of pixel points is denoted as ratio. Among them, the value of TH_valid is default set to 8; The automatic light measurement specifically includes the following sub-steps: S1.

3. Divide the image into several blocks, and record the total number of blocks as total_block. For each block i with a size of s*s, calculate its average brightness L i , and the average brightness L i The formula is as follows: Among them, L i (x, y) represents the brightness of the pixel point in the block, and R i (x, y), G i (x, y), B i (x, y) respectively represent the values of the point (x, y) in the RGB three channels, represents rounding down; S1.

4. Calculate the selected number block; block = ratio * total_block; S1.

5. Sort the average brightness of all blocks from high to low, and the light measurement result is; Among them, L j represents the average brightness of the jth selected block, and S is the tolerance parameter, which can be set to any value within 0-1; For the Nth frame image in the video stream, the input conditions of the automatic light adjustment module are: the current image brightness Y N , the brightness Y of the previous frame image N-1 , the target image brightness TY, the exposure time ET N , ET N-1 , the analog gain amplification factor GAIN N , GAIN N-1 , and the judgment of whether light adjustment is required in S2 is as follows: S2.

1. When abs(TY - Y N ) < range, that is, the difference between the current picture brightness and the target brightness is small, and the exposure parameters remain unchanged, and the automatic light adjustment is exited. Among them, abs() is the absolute value function, and range is the threshold constant; S2.2 When abs(TY - Y N )>range, then enter the dimming process; in step S3, including the following sub-steps: S3.1, dynamically determine whether the picture is adjusted; S3.2, adaptively brighten the dark area; S3.3, dynamically enhance the contrast; Step S3.1 determines whether to adjust the picture, specifically including the following sub-steps; S3.11, first convert the image from the RGB domain to the YUV domain, the formula is as follows: Y = 0.1826*R + 0.6142*G + 0.062*B + 16U = -0.1006*R - 0.3386*G + 0.43 92*B+128V=0.4392*B-0.3989*G-0.0403*B+128; S3.12, perform histogram statistics on the Y channel. When performing histogram statistics, only pixels with brightness greater than TH_valid will be counted. After obtaining the brightness histogram Hist_L, sum the number of grayscale pixels in Hist_L to obtain Hist_L_sum. Then divide the number of grayscale pixels in Hist_L by Hist_L_sum to obtain the weight of each grayscale. The formula is as follows: Hist_L_w j =Hist_L j / Hist_L_sum; S3.13, the weighted sum of the weight and each grayscale value is obtained to obtain the weighted grayscale value of the image, the formula is as follows: In the histogram Hist_L, the number of pixels with grayscale values ​​less than TH_GT is counted as left_num, and the number of pixels greater than or equal to TH_GT is counted as right_num, where TH_GT defaults to 90, and r_l_ratio is the ratio of the number of pixels to the right of TH_GT to the number of pixels to the left of TH_GT, as shown in the following formula: r_l_ratio = right_num / left_num; S3.14, when gray_avg is greater than or equal to TH_GT, the image is considered normal and no changes are required; when r_l_ratio is greater than or equal to 1, the image is considered normal and no changes are required. , except for the above two cases, it is considered that the overall picture is dark and needs to be dynamically brightened; step S3.2, adaptive dark area brightening specifically includes the following sub-steps: S3.21, when the gray_avg value is small, it is considered that the overall brightness of the picture is low, and the dark area needs to be enhanced more, but at this time there may be a double peak state, and it is necessary to introduce the r_l_ratio ratio. When the ratio is closer to 1, it indicates that the overall picture has better discrimination and the overall distribution of light and dark is appropriate, and there is no need to brighten the dark area; S3.22, design a curve that is dynamically adjusted according to the gray_avg and r_l_ratio values. The formula is as follows: f(x) = (x b *c d ) / (x b *c d +(1-x) d *a b ) where x is the grayscale value, ranging from [0,255], f(x) indicates that the original grayscale value will be mapped to a new grayscale value, where a is 0.3 and d is 1; c is dynamically adjusted according to gray_avg, as follows: c_i = min((TH_GT - gray_avg) / (TH_GT - 40), 1) c = 0.3 + c_i * 0.3; b is dynamically adjusted according to r_l_ratio and gray_avg, where min indicates the smaller of the two values, as follows: b_i=1-c_i*(1-new_r_l_ratio)b=min(0.35+0.65*b_i,1); S3.23, apply the new grayscale value of f(x) to the Y channel of the YUV image to obtain a new brightened YUV_new image; step S3.3, dynamic contrast enhancement includes the following sub-steps: S3.31, first convert YUV_new into RGB_new, that is, convert the YUV domain image to the RGB domain; S3.32, enhance RGB_new using an adaptive linear contrast method, the specific enhancement formula is as follows: in, Where m represents a pixel at a certain position, n represents a color channel in RGB, c_f is the contrast enhancement factor, and the c_f formula is as follows: c_f = 1.4-0.4*b_i. The clip operation means that when the calculation result is less than 0, it is truncated to 0, and when it is greater than 255, it is truncated to 255. This is the final result graph.

2. [Corrected 23.04.2025 according to Rule 91] A method for dynamic luminance stabilization (DLC) according to claim 1, characterized in that: In step S2.2, the dimming process includes modifying the exposure model and calculating the exposure parameters, and modifying the exposure model specifically includes the following sub-steps: S2.

21. According to the working principle of the image sensor, under the condition that the scene remains unchanged, there is the following relationship: GAIN N+1 -GAIN N =K1*(Y N+1 -Y N )ET N+1 -ET N =K2*(Y N+1 -Y N ); S2.22, set K to the initial value. For constantly changing scenes, the K value needs to be corrected in time. In the dimming process, when either GAIN or ET changes, the K value of the current frame is recalculated based on the change in brightness. When both change at the same time, the K value is expressed as follows: S2.

23. In order to improve the robustness of automatic exposure during endoscope movement, the K value can be taken as the weighted value of the current frame and the previous frames. After obtaining the estimated K value, the change in exposure parameters can be estimated based on the difference between the current frame brightness and the target brightness.

3. [Corrected 23.04.2025 according to Rule 91] A method for dynamic luminance stabilization (DLC) according to claim 2, characterized in that: Calculating the exposure parameters specifically includes the following sub-steps: S2.24, introducing n damping coefficients p to further adjust the dimming process. The damping coefficients can also be used to adjust the dimming speed. The damping coefficients p are expressed as follows: S2.

25. Since increasing exposure time helps improve the signal-to-noise ratio, exposure time should be used first during dimming. When exposure time is insufficient to meet brightness requirements, adjust the analog gain. When GAIN = 1, GAIN is not enabled. Adjust ET: ET N+1 =ET N +ΔET calculates and sends the actual register value Reg ET , end the current frame dimming.

4. [Corrected 23.04.2025 according to Rule 91] A method for dynamic luminance stabilization (DLC) according to claim 3, characterized in that: When GAIN≠1, adjust GAIN: GAIN N+1 =GAIN N +ΔGAIN calculates the actual register value Reg GAIN , when ΔGAIN is less than the threshold th_g, through D gain Make further corrections; let the difference between the magnification of the actual register value and the actual magnification of the previous frame be A, then: A*abs(D gain -1)=p*K1*(TY-Y N ) where D gain >0, the adjusted D can be calculated based on this relationship gain , and finally calculate Reg ET and Reg GAIN End dimming and multiply the N+1 frame image by D gain , and get N+1 frames of image.

5. [Corrected 23.04.2025 according to Rule 91] A method for dynamic luminance stabilization (DLC) according to claim 1, characterized in that: In step S3.31, the formula for converting the image in the YUV domain to the RGB domain is as follows: R = 1.1644*(Y-16) + 1.7928*(V-128) G = 1.1644*(Y-16) - 0.2133*(U-127) - 0.533*(V-128) B = 1.1644*(Y-16) + 2.1124*(U-128).

Images