A method, apparatus, device and medium for fast adaptive exposure adjustment

Abstract

The present application relates to the technical field of image processing, and relates to a fast self-adaptive exposure adjusting method, device, equipment and medium, comprising: dividing a to-be-processed image into N*M sub-blocks, traversing pixels in each sub-block to obtain a pixel mean P of the to-be-processed image and a number X of overexposed pixels, when it is determined that the number X of overexposed pixels is greater than a preset number threshold of overexposed pixels, using a super large step to reduce exposure, when it is determined that the number X of overexposed pixels is less than or equal to the preset number threshold of overexposed pixels, judging whether there is a scene change, when the scene changes, entering a punishment coefficient calculation link, and using a historical image frame mean change direction to determine the punishment coefficient, adjusting an exposure time or a digital gain value based on the punishment coefficient until the pixel mean P is in a target exposure interval, and the above operations are all completed on an image sensor, through simplifying logic and parameters, memory occupation is small, and at the same time, the punishment coefficient is added to the exposure adjustment, helping the control process to converge quickly.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for rapid adaptive exposure adjustment. Background Technology

[0002] In the image generation process of CMOS sensors, exposure has a crucial impact on image quality. If the exposure is too low during image processing, the overall image will be dark, losing some texture details. Conversely, if the exposure is too high, localized overexposure can occur, resulting in completely white areas that lose all image information. Appropriate exposure contributes to image signal processing and color restoration algorithms, and also makes the image easier for the human eye to read. Therefore, automatic exposure technology is widely used in image signal processing; in fact, the quality of automatic exposure adjustment can significantly affect subsequent image signal processing operations. In CMOS sensors, exposure adjustment mainly involves two modules: adjusting gain to adjust image exposure, and adjusting exposure time to adjust exposure.

[0003] The paper "A Fast Automatic Exposure Control Method Applied to CMOS Image Sensors" by Ge Zhiwei et al. was published in the *Journal of Tianjin University*. This scheme rapidly reduces the number of frames required for exposure adjustment by using a lookup table. While this scheme adjusts both gain and exposure time to adjust exposure, it doesn't further balance the relationship between gain and exposure. Moreover, this method has a limited application scenario, only suitable for fixed shooting scenarios where the content captured by the camera must remain unchanged. Any change would cause variations in image pixel values, and currently, this algorithm cannot distinguish whether these pixel value changes are caused by lighting conditions or scene changes.

[0004] Existing methods have limited applicability, requiring their exposure adjustment modules to maintain a stable image content. However, in practice, automatic exposure algorithms often involve changing scenarios, such as camera movement or movement of the subject being photographed. Therefore, the design of some current methods is not suitable for the real-world applications prevalent in industry.

[0005] Current image sensor methods require significant amounts of memory to store data, such as for histogram analysis. Memory is a very limited resource in image sensors. Many current CMOS sensor chips store data on a row-by-row basis. Many current methods require computational resources equivalent to several frames of images. This is unacceptable in the practical development of CMOS sensors.

[0006] In actual exposure adjustment, it's easy to encounter problems such as one frame being too bright and another too dark, failing to reach the ideal exposure position even after repeated adjustments across multiple frames. Alternatively, the image may gradually darken but remain too bright overall, or gradually brighten but remain too dark overall. Or, after adjustment, the overall image brightness may meet the standard, but localized areas may be overexposed, resulting in information loss. Furthermore, unilateral adjustments to gain or exposure time can lead to noise issues and reduced frame rate. Summary of the Invention

[0007] To solve the above problems, the present invention provides a method, apparatus, device, and medium for rapid adaptive exposure adjustment.

[0008] In a first aspect, embodiments of the present invention provide a method for rapid adaptive exposure adjustment, applied to an image sensor, comprising:

[0009] Obtain the image to be processed, and divide the image to be processed into N*M sub-blocks, where N and M are positive integers;

[0010] By iterating through the pixels in each sub-block, the average pixel value P and the number of overexposed pixels X of the image to be processed are obtained.

[0011] When it is determined that the number of overexposed pixels X is greater than the preset threshold for the number of overexposed pixels, a preset ultra-large step size is used to reduce the exposure;

[0012] When it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels, it is determined whether there is a scene change.

[0013] When the scene changes, the penalty coefficient calculation stage is entered, and the penalty coefficient is determined by using the direction of change of the average value of historical image frames.

[0014] When the scenario remains unchanged, the preset penalty parameter will be used as the penalty coefficient.

[0015] The exposure time or digital gain value of the image to be processed is adjusted based on the penalty coefficient until the pixel mean P is within the target exposure range.

[0016] As an optional approach, the step of traversing the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed includes:

[0017] Pre-set the overexposed pixel threshold;

[0018] When traversing each pixel in turn, it is determined whether the number of overexposed pixels is greater than the overexposed pixel threshold. If it is greater, the number of overexposed pixels is incremented by 1. If it is less, the pixel sum is accumulated to obtain the number of M*N overexposed pixels.

[0019] For each sub-block, calculate the pixel mean:

[0020] Sub-block pixel average = Sum of accumulated pixels / (Total number of sub-block pixels - Number of overexposed pixels)

[0021] The total number of pixels in the sub-block is calculated in advance based on the numerical settings of M and N.

[0022] The average pixel value P of the image to be processed is obtained by weighting the average values ​​of each sub-block.

[0023] As an optional solution, the step of determining whether a scene change has occurred when the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels includes:

[0024] Determine the exposure brightness correlation coefficient K. The formula for calculating K is:

[0025] K = (P - Dark current level) / Current exposure time;

[0026] Among them, the dark current black level is the background value of the pixel;

[0027] There are three criteria for determining whether a scene has changed; meeting any one of them is sufficient. These include:

[0028] If the change in adjacent image frame K exceeds the change threshold, then a scene change is determined.

[0029] If the average pixel value P of adjacent image frames changes beyond the preset value without adjusting the exposure and gain, then a scene change is determined.

[0030] If the direction of exposure gain adjustment in adjacent image frames is opposite to the direction of change in pixel mean P, then a scene change is determined.

[0031] As an optional approach, before proceeding to the penalty coefficient calculation stage when the scene changes, and before determining the penalty coefficient using the direction of change of the historical image frame average, the method further includes:

[0032] Read the direction of change of the mean value of a historical image frame, and set the expected mean value BESTV. These two values ​​are determined by the results of the previous frame and the frame before that.

[0033] For a historical image frame whose average exposure value changes in the direction of HISE, the calculation is as follows:

[0034] If the average exposure of adjacent historical image frames crosses BESTV, it is denoted as -1;

[0035] If the average exposure value of adjacent historical image frames does not cross BESTV, it is recorded as 1;

[0036] If the exposure gain of adjacent historical image frames does not change, it is recorded as 0;

[0037] Set an intermediate variable HISV, which is specifically calculated as the absolute value of the full-map mean minus the expected mean;

[0038] After obtaining the change direction of two historical image frames, the penalty coefficient calculation stage begins. The penalty weight parameter is pre-configured as J, the lowest penalty weight parameter is WMIN, and the highest penalty weight parameter is WMAX. The penalty coefficient is iteratively updated and initialized to 1.0.

[0039] As an optional approach, the step of calculating the penalty coefficient when the scene changes, and determining the penalty coefficient using the direction of change of the average value of historical image frames, includes:

[0040] Input HISE and HISV, the penalty coefficient calculated in the previous frame, and determine whether the adjustment judgment coefficient HISE is greater than zero. If it is greater than zero, the penalty coefficient = penalty coefficient / (1 + J) * (BESTV - HISV) / ​​BESTV. If it is not greater than zero, continue to determine whether the adjustment judgment coefficient is equal to zero. If it is equal to zero, the penalty coefficient remains unchanged. If it is not equal to zero, continue to determine whether it is less than zero. If it is less than zero, the penalty coefficient = penalty coefficient / (1 + J) * BESTV / (BESTV - HISV).

[0041] If the penalty coefficient is greater than WMAX, then WMAX will be used as the penalty coefficient.

[0042] If the penalty coefficient is less than WMIN, then WMIN will be used as the penalty coefficient.

[0043] As an optional approach, adjusting the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range includes:

[0044] Set a maximum exposure threshold. If the image to be processed is too dark, increase the exposure time first to improve the exposure. If the image to be processed has reached the maximum exposure threshold, increase the gain by a preset step size. If the image to be processed is too bright, determine to reduce the exposure of the image to be processed. When reducing the exposure, only reduce the exposure time value.

[0045] Pre-configure the minimum exposure boundary BMIN and the maximum exposure boundary BMAX;

[0046] Input the average pixel value P of the image to be processed;

[0047] When the average pixel value P is greater than the minimum exposure boundary BMIN and less than the maximum exposure boundary BMAX, determine whether the average pixel value P is greater than BESTV.

[0048] If it is less than or equal to BESTV, then reduce the exposure time by (BESTV-P) / K*penalty factor;

[0049] If the exposure time is greater than BESTV, the exposure time is increased by (P-BESTV) / K*penalty coefficient. The system continues to determine whether the exposure time is greater than the maximum exposure threshold. If it is determined that the exposure time is greater than the maximum exposure threshold, the exposure time is equal to the maximum exposure threshold, and the digital gain is increased by a fixed step size until the exposure of the image to be processed is within the target exposure range.

[0050] As an optional scheme, both M and N are 6.

[0051] Secondly, embodiments of the present invention provide a fast adaptive exposure adjustment device, comprising:

[0052] An acquisition unit is used to acquire the image to be processed and divide the image to be processed into N*M sub-blocks, where N and M are positive integers;

[0053] The traversal unit is used to traverse the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed.

[0054] The judgment unit is used to reduce the exposure by a preset ultra-large step size when it is determined that the number of overexposed pixels X is greater than a preset threshold for the number of overexposed pixels.

[0055] The judgment unit is also used to determine whether there is a scene change when it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels;

[0056] The judgment unit is also used to enter the penalty coefficient calculation stage when the scene changes, and to determine the penalty coefficient by using the direction of change of the average value of historical image frames.

[0057] The judgment unit is also used to use a preset penalty parameter as a penalty coefficient when the scene remains unchanged.

[0058] The judgment unit is further configured to adjust the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range.

[0059] Thirdly, embodiments of the present invention provide a computer device, comprising:

[0060] At least one processor; and

[0061] A memory communicatively connected to the at least one processor; wherein,

[0062] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the above-described fast adaptive exposure adjustment method.

[0063] Fourthly, in this embodiment of the invention, a non-transient computer-readable storage medium storing computer instructions is provided, the computer instructions being used to cause the computer to execute the above-described rapid adaptive exposure adjustment method.

[0064] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0065] 1. By simplifying the logic, it can be seen that the entire automatic exposure system described above operates on simple logic, without involving complex mathematical formulas or large amounts of memory. Therefore, compared to other automatic exposure solutions, it requires less memory and has a higher probability of practical application.

[0066] 2. In the embodiments of the present invention, the influence of overexposed pixels is fully considered during the calculation, which can largely avoid errors caused by overexposed pixels.

[0067] 3. Compared to other algorithms, the application scenarios in this invention are more diverse. This invention considers the impact of scene changes on the control logic. If scene changes are not considered, adjusting exposure while changing the scene will inevitably lead to chaos in the entire algorithm. This invention determines whether the scene has changed before entering the exposure control logic. The logic of the entire system is more rigorous and more suitable for everyday application scenarios.

[0068] 4. In this embodiment of the invention, by increasing the penalty coefficient, the exposure adjustment can converge more quickly. If the adjustment is too large, the penalty coefficient can be used to make the adjustment more conservative in the next adjustment. If the adjustment is too small, the penalty coefficient can be used to make the adjustment range larger in the next adjustment. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating a rapid adaptive exposure adjustment method according to a specific embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of sub-block division in a fast adaptive exposure adjustment method provided according to a specific embodiment of the present invention;

[0071] Figure 3 This is a flowchart illustrating the penalty coefficient calculation step in a fast adaptive exposure adjustment method according to a specific embodiment of the present invention.

[0072] Figure 4This is a flowchart illustrating the control process in a rapid adaptive exposure adjustment method according to a specific embodiment of the present invention.

[0073] Figure 5 This is a structural block diagram of a fast adaptive exposure adjustment device provided in a specific embodiment of the present invention;

[0074] Figure 6 This is a structural block diagram of a computer device provided according to a specific embodiment of the present invention. Detailed Implementation

[0075] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0077] Combination Figure 1 As shown, this embodiment of the invention provides a fast adaptive exposure adjustment method applied to an image sensor, comprising:

[0078] S101. Obtain the image to be processed and divide the image to be processed into N*M sub-blocks, where N and M are positive integers.

[0079] like Figure 2 As shown, an image to be processed is pre-divided into N*M blocks. The actual chip operation is from left to right and from top to bottom. In this way, the row coordinates and column coordinates can be used to determine which sub-block each pixel belongs to. In this embodiment, M and N can be set to 6 respectively, which can also reduce the required memory and reduce memory usage.

[0080] S102. Traverse the pixels in each sub-block to obtain the average pixel value P and the number of overexposed pixels X of the image to be processed.

[0081] For each sub-block, two values ​​need to be calculated: the accumulated pixel sum and the number of overexposed pixels, X. To determine the overexposed pixels, an overexposed pixel threshold needs to be configured. This threshold is used to judge whether each pixel is overexposed. When iterating through each pixel, it is necessary to check if it is greater than the overexposed pixel threshold. If it is, the number of overexposed pixels is incremented by 1; if it is less, the accumulated pixel sum is added. Finally, we obtain M*N accumulated pixel sums and M*N overexposed pixel counts.

[0082] Calculate the mean for each sub-block:

[0083] Sub-block average = Sum of accumulated pixels / (Total number of sub-block pixels - Number of overexposed pixels);

[0084] The total number of pixels in a sub-block can be pre-calculated by setting the values ​​of M and N. M multiplied by N weight information is also pre-stored, meaning each pixel square is pre-configured with a weight parameter. The final pixel mean of the entire image is the weighted average of the averages of each sub-block, and the number of overexposed pixels in the entire image is the sum of the overexposed pixels in each sub-block.

[0085] S103. When it is determined that the number of overexposed pixels X is greater than the preset threshold for the number of overexposed pixels, a preset ultra-large step size is used to reduce the exposure.

[0086] The input image frame is summed sequentially, and the pixel mean P of the entire image and the number of overexposed pixels X are calculated. Then, it's necessary to determine if the number of overexposed pixels exceeds a threshold. This threshold is a pre-configured parameter. If the number of overexposed pixels X exceeds the threshold, a large step size is needed to reduce the exposure. The amount of exposure reduction via the large step size is an input parameter that can be set as needed and is not limited.

[0087] S104. When it is determined that the number of overexposed pixels X is less than or equal to the preset threshold for the number of overexposed pixels, it is determined whether there is a scene change.

[0088] Determine the exposure brightness correlation coefficient K. The formula for calculating K is:

[0089] K = (P - Dark current level) / Current exposure time;

[0090] Among them, the dark current black level is the background value of the pixel.

[0091] There are three criteria for determining whether a scene has changed; meeting any one of them is sufficient. These include:

[0092] If the change in adjacent image frame K exceeds the change threshold, it is determined that the scene has changed; otherwise, if the change threshold is not exceeded, it is determined that the scene has not changed.

[0093] If the average pixel value P of adjacent image frames changes beyond a preset value without adjusting exposure and gain, then a scene change is determined; otherwise, if it does not exceed the preset value, then no scene change has occurred.

[0094] If the direction of exposure gain adjustment in adjacent image frames is opposite to the direction of change in pixel mean P, then a scene change is determined; otherwise, if they are not opposite, then no scene change is determined.

[0095] If the scene is determined to have changed, the penalty coefficient can be set to 1.0. If the scene is determined not to have changed, the penalty coefficient calculation can continue to the step S105.

[0096] S105. When the scene changes, the penalty coefficient calculation step is entered, and the penalty coefficient is determined by using the direction of change of the average value of historical image frames.

[0097] Combination Figure 3 As shown, before proceeding to the penalty coefficient calculation stage, it is necessary to read the direction of change of the mean of a historical image frame and set the expected mean value BESTV. These two values ​​are determined by the results of the previous frame and the frame before that.

[0098] For a historical image frame whose average exposure value changes in the direction of HISE, the calculation is as follows:

[0099] If the average exposure of adjacent historical image frames crosses BESTV, it is denoted as -1;

[0100] If the average exposure value of adjacent historical image frames does not cross BESTV, it is recorded as 1;

[0101] If the exposure gain of adjacent historical image frames does not change, it is recorded as 0;

[0102] Set an intermediate variable HISV, which is specifically calculated as the absolute value of the full-map mean minus the expected mean;

[0103] After obtaining the change direction of two historical image frames, the penalty coefficient calculation stage begins. The penalty weight parameter is pre-configured as J, the lowest penalty weight parameter is WMIN, and the highest penalty weight parameter is WMAX. The penalty coefficient is iteratively updated and initialized to 1.0.

[0104] After obtaining the change directions of two historical image frames, the penalty coefficient calculation phase begins. The penalty weight is set to J, which is a configuration parameter. The lowest penalty weight is WMIN, and the highest is WMAX. The penalty coefficient is iteratively updated and initialized to 1.0. The specific penalty coefficient calculation is as follows:

[0105] Input HISE and HISV, the penalty coefficient calculated in the previous frame, and determine whether the adjustment judgment coefficient HISE is greater than zero. If it is greater than zero, the penalty coefficient = penalty coefficient / (1 + J) * (BESTV - HISV) / ​​BESTV. If it is not greater than zero, continue to determine whether the adjustment judgment coefficient is equal to zero. If it is equal to zero, the penalty coefficient remains unchanged. If it is not equal to zero, continue to determine whether it is less than zero. If it is less than zero, the penalty coefficient = penalty coefficient / (1 + J) * BESTV / (BESTV - HISV).

[0106] When the penalty coefficient is greater than WMAX, then WMAX can be used as the penalty coefficient;

[0107] If the penalty coefficient is less than WMIN, then WMIN can be used as the penalty coefficient.

[0108] S106. When the scenario remains unchanged, the preset penalty parameter is used as the penalty coefficient.

[0109] If a scene change is detected, the penalty coefficient can be set to 1.0. It should be noted that the penalty coefficient can be set as needed and there is no limitation on it.

[0110] S107. Adjust the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range.

[0111] After obtaining the penalty coefficient, the process proceeds to exposure and gain adjustment. Adjusting the exposure involves two parameters: exposure time and digital gain. To increase the image exposure, either the exposure time or the digital gain value can be increased. To decrease the image exposure, either the exposure time or the digital gain value can be decreased. The specific strategy of this invention for balancing the adjustment relationship between exposure time and digital gain is as follows:

[0112] If the image to be processed is too dark (exposure value is less than the set threshold), and the exposure needs to be increased, then the exposure time is increased first. If the maximum exposure threshold has already been reached, then the gain is increased step by step, with the step size being the parameter setting. If the image to be processed is too bright (exposure value is greater than the set threshold), and the exposure needs to be reduced, then only the exposure time value is reduced. The set threshold can be selected as needed and is not limited to this.

[0113] Combination Figure 4 As shown, during the adjustment process, two desired boundaries can be set: the minimum exposure boundary BMIN and the maximum exposure boundary BMAX. If the adjusted value falls between BMIN and BMAX, the adjustment can be terminated. The specific adjustment process is as follows:

[0114] Pre-configure the minimum exposure boundary BMIN and the maximum exposure boundary BMAX;

[0115] Input the average pixel value P of the image to be processed;

[0116] When the average pixel value P is greater than the minimum exposure boundary BMIN and less than the maximum exposure boundary BMAX, determine whether the average pixel value P is greater than BESTV.

[0117] If it is less than or equal to BESTV, then reduce the exposure time by (BESTV-P) / K*penalty factor;

[0118] If the exposure time is greater than BESTV, the exposure time is increased by (P-BESTV) / K*penalty coefficient. The exposure time is then checked again to see if it is greater than the maximum exposure threshold. If it is determined to be greater than the maximum exposure threshold, the exposure time is equal to the maximum exposure threshold. The digital gain is increased by a fixed step size until the exposure of the image to be processed is within the target exposure range. The target exposure range mentioned here is the ideal exposure range that we want to obtain.

[0119] Specifically, first, the average pixel value P of the current full-frame image to be processed is calculated. It is then determined whether P falls within the target exposure range. If it does, no adjustment is needed; otherwise, further adjustment is required. Exposure adjustment is performed by comparing the current average pixel value P with BESTV. If P is greater than BESTV, the exposure time is increased. If, after increasing the exposure time, the current total exposure time exceeds the maximum exposure threshold, then the current total exposure time equals the maximum exposure threshold. Furthermore, the digital gain needs to be increased by the set fixed step size. If the average pixel value P is less than BESTV, the exposure time is decreased according to the formula, ultimately keeping the exposure level within the target exposure range.

[0120] Adding a penalty factor to exposure adjustments helps them converge quickly. If an adjustment crosses the target exposure range, the adjustment level needs to be reduced appropriately; this penalty parameter helps to reduce the adjustment level. If an adjustment does not cross the target exposure range, the adjustment level needs to be increased appropriately; this penalty parameter helps to increase the adjustment level.

[0121] The penalty parameter added to the exposure settings is related to the adjustment range. If an adjustment crosses the target exposure range by a significant margin, the adjustment range is reduced, with the reduction decreasing as the margin increases. Conversely, if an adjustment does not cross the target exposure range and the margin is small, the adjustment level is increased, with the adjustment level gradually increasing as the margin decreases. In this embodiment of the invention, when adjusting exposure time and digital gain, the exposure time is adjusted first, and only after the exposure time has been adjusted to a certain level is the digital gain adjusted, clearly defining the order of priority.

[0122] The fast adaptive exposure adjustment method provided by this invention is applicable to image sensor chips and CMOS chips. The advantages of this method are twofold: first, it implements adaptive exposure adjustment functionality using simple logic; second, the simple logic can be implemented on image sensors.

[0123] The fast adaptive exposure adjustment method provided in this embodiment of the invention has the following advantages:

[0124] 1. By simplifying the logic, it can be seen that the entire automatic exposure system described above operates on simple logic, without involving complex mathematical formulas or large amounts of memory. Therefore, compared to other automatic exposure solutions, it requires less memory and has a higher probability of practical application.

[0125] 2. In the embodiments of the present invention, the influence of overexposed pixels is fully considered during the calculation, which can largely avoid errors caused by overexposed pixels.

[0126] 3. Compared to other algorithms, the application scenarios in this invention are more diverse. This invention considers the impact of scene changes on the control logic. If scene changes are not considered, adjusting exposure while changing the scene will inevitably lead to chaos in the entire algorithm. This invention determines whether the scene has changed before entering the exposure control logic. The logic of the entire system is more rigorous and more suitable for everyday application scenarios.

[0127] 4. In this embodiment of the invention, by increasing the penalty coefficient, the exposure adjustment can converge more quickly. If the adjustment is too large, the penalty coefficient can be used to make the adjustment more conservative in the next adjustment. If the adjustment is too small, the penalty coefficient can be used to make the adjustment range larger in the next adjustment.

[0128] Combination Figure 5 As shown, correspondingly, this embodiment of the invention provides a fast adaptive exposure adjustment device, comprising:

[0129] The acquisition unit 501 is used to acquire the image to be processed and divide the image to be processed into N*M sub-blocks, where N and M are positive integers;

[0130] Traversal unit 502 is used to traverse the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed.

[0131] The judgment unit 503 is used to reduce the exposure by a preset ultra-large step size when it is determined that the number of overexposed pixels X is greater than a preset threshold for the number of overexposed pixels.

[0132] The judgment unit is also used to determine whether there is a scene change when it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels;

[0133] The judgment unit is also used to enter the penalty coefficient calculation stage when the scene changes, and to determine the penalty coefficient by using the direction of change of the average value of historical image frames.

[0134] The judgment unit is also used to use a preset penalty parameter as a penalty coefficient when the scene remains unchanged.

[0135] The judgment unit is further configured to adjust the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range.

[0136] This invention provides a fast adaptive exposure adjustment method applied to an image sensor, comprising: acquiring an image to be processed; dividing the image to be processed into N*M sub-blocks; traversing the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed; when it is determined that the number of overexposed pixels X is greater than a preset overexposed pixel number threshold, reducing the exposure with a preset ultra-large step size; when it is determined that the number of overexposed pixels X is less than or equal to the preset overexposed pixel number threshold, determining whether there is a scene change; when the scene changes, entering the penalty coefficient calculation stage, and determining the penalty coefficient using the direction of change of the historical image frame mean; when the scene does not change, using a preset penalty parameter as the penalty coefficient, adjusting the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is in the target exposure range. All of this is completed on the image sensor. By simplifying the logic and parameters, the memory footprint is very small, and the actual usability is very strong. Adding a penalty coefficient to the exposure adjustment helps the control process converge quickly.

[0137] Accordingly, according to embodiments of the present invention, the present invention also provides a computer device, a readable storage medium, and a computer program product.

[0138] Figure 6 This is a schematic diagram of the structure of a computer device 12 provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0139] like Figure 6 As shown, computer device 12 is represented in the form of a general-purpose computing device. Computer device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0140] The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0141] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0142] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0143] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0144] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0145] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with computer device 12, and / or with any device that enables computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0146] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the fast adaptive exposure adjustment method provided in the embodiments of the present invention.

[0147] This invention also provides a non-transient computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein the program, when executed by a processor, is the fast adaptive exposure adjustment method provided in all embodiments of this application.

[0148] The computer storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0149] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0150] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. The computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described rapid adaptive exposure adjustment method.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A fast adaptive exposure adjustment method, applied to an image sensor, characterized in that, include: Obtain the image to be processed and divide the image to be processed into N. There are M sub-blocks, where N and M are positive integers; By iterating through the pixels in each sub-block, the average pixel value P and the number of overexposed pixels X of the image to be processed are obtained. When it is determined that the number of overexposed pixels X is greater than the preset threshold for the number of overexposed pixels, a preset ultra-large step size is used to reduce the exposure; When it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels, it is determined whether there is a scene change. When it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels, it is determined whether there is a scene change, including: Determine the exposure brightness correlation coefficient K. The formula for calculating K is: K = (P - Dark current level) / Current exposure time; Among them, the dark current black level is the background value of the pixel; There are three criteria for determining whether a scene has changed; meeting any one of them is sufficient. These include: If the change in adjacent image frame K exceeds the change threshold, then a scene change is determined. If the average pixel value P of adjacent image frames changes beyond the preset value without adjusting the exposure and gain, then a scene change is determined. If the direction of exposure gain adjustment in adjacent image frames is opposite to the direction of change in pixel mean P, then a scene change is determined. When the scene changes, the penalty coefficient calculation stage is entered, and the penalty coefficient is determined by using the direction of change of the average value of historical image frames. When the scenario remains unchanged, the preset penalty parameter will be used as the penalty coefficient. The exposure time or digital gain value of the image to be processed is adjusted based on the penalty coefficient until the pixel mean P is within the target exposure range.

2. The rapid adaptive exposure adjustment method according to claim 1, characterized in that, The step of traversing the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed includes: Pre-set the overexposed pixel threshold; When iterating through each pixel, check if it is greater than the overexposed pixel threshold. If it is, increment the number of overexposed pixels by 1; otherwise, sum the pixel counts to obtain M. N accumulated pixels and M The number of N overexposed pixels; For each sub-block, calculate the pixel mean: Sub-block pixel average = Sum of accumulated pixels / (Total number of sub-block pixels - Number of overexposed pixels) The total number of pixels in the sub-block is calculated in advance based on the numerical settings of M and N. The average pixel value P of the image to be processed is obtained by weighting the average values ​​of each sub-block.

3. The rapid adaptive exposure adjustment method according to claim 1, characterized in that, Before entering the penalty coefficient calculation stage when the scene changes, and determining the penalty coefficient using the direction of change of the historical image frame average, the process also includes: Read the direction of change of the mean value of a historical image frame, and set the expected mean value BESTV. These two values ​​are determined by the results of the previous frame and the frame before that. For a historical image frame whose average exposure value changes in the direction of HISE, the calculation is as follows: If the average exposure of adjacent historical image frames crosses BESTV, it is denoted as -1; If the average exposure value of adjacent historical image frames does not cross BESTV, it is recorded as 1; If the exposure gain of adjacent historical image frames does not change, it is recorded as 0; Set an intermediate variable HISV, which is specifically calculated as the absolute value of the full-map mean minus the expected mean; After obtaining the change direction of two historical image frames, the penalty coefficient calculation stage begins. The penalty weight parameter is pre-configured as J, the lowest penalty weight parameter is WMIN, and the highest penalty weight parameter is WMAX. The penalty coefficient is iteratively updated and initialized to 1.

0.

4. The rapid adaptive exposure adjustment method according to claim 1, characterized in that, When the scene changes, the penalty coefficient calculation step is initiated, and the penalty coefficient is determined using the direction of change of the average value of historical image frames, including: Input HISE and HISV, the penalty coefficient calculated in the previous frame, and determine whether the control judgment coefficient HISE is greater than zero. If it is greater than zero, the penalty coefficient = penalty coefficient / (1 + J). (BESTV-HISV) / BESTV, if not greater than zero, then continue to determine whether the control judgment coefficient is equal to zero. If it is equal to zero, the penalty coefficient remains unchanged. If it is not equal to zero, then continue to determine whether it is less than zero. If it is less than zero, then the penalty coefficient = penalty coefficient / (1+J). BESTV / (BESTV-HISV) If the penalty coefficient is greater than WMAX, then WMAX will be used as the penalty coefficient. If the penalty coefficient is less than WMIN, then WMIN will be used as the penalty coefficient.

5. The rapid adaptive exposure adjustment method according to claim 1, characterized in that, The step of adjusting the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range includes: Set a maximum exposure threshold. If the image to be processed is too dark, increase the exposure time first to improve the exposure. If the image to be processed has reached the maximum exposure threshold, increase the gain by a preset step size. If the image to be processed is too bright, determine to reduce the exposure of the image to be processed. When reducing the exposure, only reduce the exposure time value. Pre-configure the minimum exposure boundary BMIN and the maximum exposure boundary BMAX; Input the average pixel value P of the image to be processed; When the average pixel value P is greater than the minimum exposure boundary BMIN and less than the maximum exposure boundary BMAX, determine whether the average pixel value P is greater than BESTV. If it is less than or equal to BESTV, then reduce the exposure time by (BESTV - P) / K. Penalty coefficient; If the exposure time is greater than BESTV, increase the exposure time by (P - BESTV / K). The penalty coefficient is used to continue determining whether the exposure time is greater than the maximum exposure threshold. If it is determined to be greater than the maximum exposure threshold, the exposure time is equal to the maximum exposure threshold, and the digital gain is increased by a fixed step size until the exposure of the image to be processed is within the target exposure range.

6. The rapid adaptive exposure adjustment method according to claim 1, characterized in that, Both M and N are 6.

7. A fast adaptive exposure adjustment device for implementing the fast adaptive exposure adjustment method as described in any one of claims 1 to 6, characterized in that, include: The acquisition unit is used to acquire the image to be processed and divide the image to be processed into N. There are M sub-blocks, where N and M are positive integers; The traversal unit is used to traverse the pixels in each sub-block to obtain the pixel mean P and the number of overexposed pixels X of the image to be processed. The judgment unit is used to reduce the exposure by a preset ultra-large step size when it is determined that the number of overexposed pixels X is greater than a preset threshold for the number of overexposed pixels. The judgment unit is also used to determine whether there is a scene change when it is determined that the number of overexposed pixels X is less than or equal to a preset threshold for the number of overexposed pixels; The judgment unit is also used to enter the penalty coefficient calculation stage when the scene changes, and to determine the penalty coefficient by using the direction of change of the average value of historical image frames. The judgment unit is also used to use a preset penalty parameter as a penalty coefficient when the scene remains unchanged. The judgment unit is further configured to adjust the exposure time or digital gain value of the image to be processed based on the penalty coefficient until the pixel mean P is within the target exposure range.

8. A computer device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the rapid adaptive exposure adjustment method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the rapid adaptive exposure adjustment method according to any one of claims 1 to 6.

Images