A dynamic adaptive black level correction method and device

Abstract

This application discloses a dynamic adaptive black level correction method and apparatus, relating to the field of image processing technology. Based on the spatial distribution differences of dark current, the effective pixel regions in each frame of the original image are dynamically divided; based on the dark pixel noise level, the initial black level calibration coefficients for each spatial block are determined; the current temperature and temperature change rate of each spatial block are obtained, and based on the initial black level calibration coefficients corresponding to each spatial block, the temperature-adaptive calibration coefficients for each spatial block are obtained; the calibration information from the previous frame and the calibration coefficients of the current frame are fused to obtain a stable cross-frame calibration result; based on the stable cross-frame calibration result, the effective pixels in each spatial block are corrected to obtain corrected pixel values, and the corrected pixel values ​​are reconstructed into the output image. This solves the problems of poor spatial uniformity, insufficient adaptability to temperature changes, and unstable calibration coefficients in existing black level calibration techniques.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a dynamic adaptive black level correction method and apparatus. Background Technology

[0002] Black level, as the output level of an image sensor when there is no light input, directly affects the dark details, contrast, and overall color reproduction of the image.

[0003] Existing technologies set up light-blocking pixel areas at the edges of the effective image region, without considering the spatial distribution differences of dark current. For example, the dark current of edge pixels may be higher than that of center pixels, leading to over-correction in some areas resulting in excessive darkness or under-correction resulting in excessive brightness, resulting in poor spatial uniformity. Furthermore, existing technologies only consider the impact of current temperature on dark current, neglecting the effect of temperature change rate. For instance, during rapid temperature increases, the increase in dark current lags behind the temperature change, leading to a decrease in correction effectiveness with rapid temperature changes. Existing technologies use single-frame calibration, calculating calibration coefficients only using dark pixels in the current frame, making them easily susceptible to single-frame noise interference, resulting in large fluctuations in calibration coefficients and unstable brightness in dark areas of the corrected image.

[0004] Therefore, existing black level calibration techniques suffer from poor spatial uniformity, insufficient adaptability to temperature changes, and unstable calibration coefficients. Summary of the Invention

[0005] In this embodiment, a dynamic adaptive black level correction method is provided to solve the problems of poor spatial uniformity, insufficient adaptability to temperature changes, and unstable calibration coefficients in existing black level calibration techniques.

[0006] In a first aspect, embodiments of this application provide a dynamic adaptive black level correction method, which includes: acquiring multiple frames of original images output by an image sensor; dynamically dividing the effective pixel region in each frame of the original image according to the difference in dark current spatial distribution to obtain multiple spatial blocks; determining the initial black level calibration coefficient of each spatial block based on the dark pixel noise level; obtaining the current temperature and temperature change rate of each spatial block, and obtaining the temperature adaptive calibration coefficient of each spatial block based on the initial black level calibration coefficient corresponding to each spatial block; fusing the calibration information of the previous frame and the calibration coefficient of the current frame to obtain a cross-frame stable calibration result; correcting the effective pixels in each spatial block based on the cross-frame stable calibration result to obtain the corrected pixel values, and reconstructing the corrected pixel values ​​into an output image.

[0007] In one possible implementation, the step of dynamically dividing the effective pixel region in each frame of the original image according to the difference in dark current spatial distribution to obtain multiple spatial blocks includes: obtaining the average value of the dark current in the dark pixel region of the current frame, and obtaining the spatial distribution variance of the dark current based on the average value, which is used as the global dark current variance; setting the size of the minimum block and the maximum block, where the side length of the maximum block divided by the side length of the minimum block is an integer; performing a sliding window scan with the maximum block as a candidate block, and when the number of remaining pixels cannot constitute the size of the complete maximum block, forming a candidate block from the remaining pixels; obtaining the dark current variance of the dark pixels in each candidate block, which is used as the local dark current variance; and dividing the candidate block into spatial blocks based on the ratio of the local dark current variance to the global dark current variance.

[0008] In one possible implementation, dividing the candidate block into spatial blocks based on the ratio of local dark current variance to global dark current variance includes: when the ratio of local dark current variance to global dark current variance is greater than a first preset threshold, dividing the candidate block equally according to the size of the minimum block to obtain spatial blocks; when the number of remaining pixels is insufficient to form a complete minimum block size, forming a spatial block from the remaining pixels; when the ratio of local dark current variance to global dark current variance is less than a second preset threshold, directly using the candidate block as a spatial block; when the ratio of local dark current variance to global dark current variance is within a first preset threshold... When the threshold is between the preset threshold and the second preset threshold, the candidate blocks are divided equally based on the size of the dynamically different adaptation blocks to obtain spatial blocks; when the number of remaining pixels cannot form the size of the complete minimum block, an adjustment operation is performed; the adjustment operation includes: adjusting the size of the previous divided spatial block sequentially from back to front to ensure that the number of remaining pixels can meet the size requirement of forming the complete minimum block; if after adjusting all spatial blocks to the size of the minimum block, the number of remaining pixels still cannot meet the size requirement of forming the complete minimum block, the adjustment operation is stopped, and the remaining pixels are formed into a spatial block.

[0009] In one possible implementation, based on Determine the side length of the dynamically differentiating adaptation block; where, To dynamically adapt the block's side length to different dimensions, The side length of the smallest block. The side length of the largest block. For the local dark current variance, The variance of the global dark current.

[0010] In one possible implementation, determining the initial black level calibration coefficient for each spatial block based on the noise level of the dark pixels includes: acquiring the current signal value at the same position of a preset threshold frame before each dark pixel, and calculating the noise standard deviation of each dark pixel; assigning a corresponding weight coefficient to the noise standard deviation of each dark pixel, wherein the weight coefficient is inversely proportional to the noise standard deviation of the dark pixel; normalizing the weight coefficient, and determining the initial black level calibration coefficient for each spatial block based on the normalized weight coefficient and the current signal value of the corresponding dark pixel.

[0011] In one possible implementation, obtaining the current temperature and temperature change rate of each spatial block, and obtaining the temperature adaptive calibration coefficient of each spatial block based on the initial black level calibration coefficient corresponding to each spatial block, includes: based on Obtain the level calibration coefficients after temperature adaptive calibration for each spatial block; where... The level calibration coefficient is the temperature-adaptive calibration coefficient for the current frame space block. This is the initial black level calibration coefficient for the current frame space block. , , , and For temperature coefficient, The current temperature of the current frame space block. The rate of temperature change of the current frame space block. This is the temperature difference value. For time intervals.

[0012] In one possible implementation, fusing the calibration information of the previous frame and the level calibration coefficient of the current frame to obtain a cross-frame stable calibration result includes: obtaining the calibration value of the dark current of the spatial block in the current frame based on the optimal calibration estimate of the dark current of the spatial block in the previous frame; obtaining the predicted covariance of the dark current of the spatial block in the current frame by adding the calibration covariance of the dark current of the spatial block in the previous frame to the process noise covariance; obtaining the gain of the dark current of the spatial block in the current frame based on the predicted covariance of the dark current of the spatial block in the current frame and the observation noise covariance; obtaining the optimal calibration estimate of the dark current of the spatial block in the current frame by combining the calibration value of the dark current of the spatial block in the current frame, the gain of the dark current of the spatial block in the current frame, and the level calibration coefficient after adaptive calibration of the temperature of the spatial block in the current frame; and using the optimal calibration estimate of the dark current of the spatial block in the current frame as the cross-frame stable calibration result; wherein, the current temperature of the spatial block in the current frame, the optimal calibration estimate of the dark current of the spatial block in the current frame, and the predicted covariance of the dark current of the spatial block in the current frame are used as spatial block storage values ​​for image processing in the next frame.

[0013] In one possible implementation, the method further includes: using the smallest block as the base block, storing the current temperature of the current frame base block, the optimal calibration estimate of the dark current of the current frame base block, and the calibration covariance of the dark current of the current frame base block as base block storage values; when the spatial block divisions of the current frame and the previous frame are inconsistent, obtaining the spatial block storage value of the previous frame based on the base block storage value, wherein the acquisition method includes: when the current frame and the previous frame spatial blocks have overlapping base blocks, obtaining the average value of the base block storage values ​​of all overlapping base blocks in the previous frame as the spatial block storage value of the previous frame, which is used for the spatial block calibration of the current frame; when the spatial block of the current frame is completely contained by the spatial block of the previous frame, directly obtaining the spatial block storage value of the previous frame for the spatial block calibration of the current frame.

[0014] In one possible implementation, the step of correcting the effective pixels in each spatial block based on the cross-frame stabilization calibration result to obtain the corrected pixel value and reconstructing the corrected pixel value into an output image includes: subtracting the corresponding cross-frame stabilization calibration result from the current value of the effective pixels in the spatial block to obtain the difference; if the difference is less than 0, setting the corrected pixel value to 0; if the difference is greater than 0, using the difference as the corrected pixel value.

[0015] Secondly, embodiments of this application provide a dynamic adaptive black level correction device, which includes: an acquisition module for acquiring multiple frames of original images output by an image sensor; a division module for dynamically dividing the effective pixel region in each frame of the original image according to the difference in dark current spatial distribution to obtain multiple spatial blocks; a determination module for determining the initial black level calibration coefficient of each spatial block based on the dark pixel noise level; an adjustment module for obtaining the current temperature and temperature change rate of each spatial block, and obtaining the temperature adaptive calibration coefficient of each spatial block based on the initial black level calibration coefficient corresponding to each spatial block; a fusion module for fusing the calibration information of the previous frame and the level calibration coefficient of the current frame to obtain a cross-frame stable calibration result; and an output module for correcting the effective pixels in each spatial block based on the cross-frame stable calibration result to obtain the corrected pixel values, and reconstructing the corrected pixel values ​​into an output image.

[0016] The one or more technical solutions provided in this application embodiment have at least the following technical effects: This application embodiment provides a dynamic adaptive black level correction method, which acquires multiple frames of original images output by an image sensor. Based on the spatial distribution differences of dark current, the effective pixel region in each frame of the original image is dynamically divided to obtain multiple spatial blocks. This allows for fine correction of areas with large dark current differences and efficient processing of areas with small differences, greatly improving the spatial uniformity of the image and making the brightness and detail of each area of ​​the image more balanced. The current temperature and temperature change rate of each spatial block are obtained, and based on the initial black level calibration coefficient corresponding to each spatial block, the level calibration coefficient after temperature adaptive calibration of each spatial block is obtained. This solves the problem of black level correction caused by the lag of dark current growth behind temperature change in scenarios with rapid temperature changes, ensuring accurate calibration under different temperature conditions. The calibration information of the previous frame and the level calibration coefficient of the current frame are fused to obtain a stable cross-frame calibration result. Based on the stable cross-frame calibration result, the effective pixels in each spatial block are corrected to obtain the corrected pixel values, and the corrected pixel values ​​are reconstructed into the output image. The corrected pixel values ​​are more accurate, solving the problems of poor spatial uniformity, insufficient adaptability to temperature changes, and unstable calibration coefficients in existing black level calibration techniques. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a dynamic adaptive black level correction method provided in this application embodiment;

[0019] Figure 2 A schematic diagram illustrating the optimal calibration estimate of the dark current of the current frame spatial block generated by dynamic block partitioning in an embodiment of this application;

[0020] Figure 3 A schematic diagram of a dynamic adaptive black level correction device provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of a dynamic adaptive black level correction server provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0024] This application provides a dynamic adaptive black level correction method, such as... Figure 1 As shown, the method includes steps S101 to S106. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a dynamic adaptive black level correction method. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0025] S101: Acquires multiple frames of raw images output by the image sensor.

[0026] Specifically, multiple frames of raw images output by the image sensor are acquired, including the current frame and the previous N frames (N>1), with each frame consisting of a valid pixel area and a dark pixel area.

[0027] S102: Based on the spatial distribution differences of dark current, the effective pixel region in each frame of the original image is dynamically divided to obtain multiple spatial blocks.

[0028] Based on the spatial distribution differences of dark current, the effective pixel region in each frame of the original image is dynamically divided to obtain multiple spatial blocks, including the following:

[0029] Obtain the average value of dark current in the dark pixel region of the current frame, and obtain the spatial distribution variance of dark current based on the average value, and use it as the global dark current variance.

[0030] Specifically, the spatial distribution variance of dark current reflects the degree of spatial difference in dark current across the entire image.

[0031] Set the size of the minimum and maximum blocks. The side length of the maximum block divided by the side length of the minimum block is an integer.

[0032] Specifically, the minimum block can be set to a fixed-size pixel block, such as 5×5 pixels, with a side length of 5. The maximum block can also be set to a fixed-size pixel block, such as 35×35 pixels, with a side length of 35. The condition is that the side length of the maximum block modulo the side length of the minimum block is zero; that is, the side length of the maximum block divided by the side length of the minimum block equals an integer.

[0033] A sliding window scan is performed with the largest block as the candidate block. When the number of remaining pixels is insufficient to form the size of the complete largest block, the remaining pixels are used to form a candidate block.

[0034] Specifically, by using the largest block as a candidate block for sliding window scanning, the area covered by the window after each scan can be used as a candidate block.

[0035] When the number of remaining pixels is insufficient to form the size of the maximum block, the remaining pixels are used to form a candidate block.

[0036] Obtain the dark current variance of the dark pixels in each candidate block and use it as the local dark current variance.

[0037] Specifically, the local dark current variance reflects the degree of difference in dark current in local images.

[0038] The candidate blocks are divided into spatial blocks based on the ratio of local dark current variance to global dark current variance.

[0039] The candidate blocks are divided into spatial blocks based on the ratio of local dark current variance to global dark current variance, including the following:

[0040] When the ratio of the local dark current variance to the global dark current variance is greater than the first preset threshold, the candidate blocks are divided equally according to the size of the smallest block to obtain spatial blocks.

[0041] When the number of remaining pixels is insufficient to form a complete minimum block size, the remaining pixels are grouped into a spatial block.

[0042] Specifically, the first preset threshold must be greater than 0, and can be set to 1.5. When the ratio of the local dark current variance to the global dark current variance is greater than the first preset threshold, it indicates that the dark current difference of the candidate block is significantly higher than the global level, and further subdivision is required to achieve more refined dark current correction.

[0043] When the ratio of local dark current variance to global dark current variance is less than the second preset threshold, the candidate block is directly used as a spatial block.

[0044] Specifically, the second preset threshold must be greater than 0, and can be set to 0.5. When the ratio of the local dark current variance to the global dark current variance is less than the second preset threshold, it indicates that the dark current difference of the candidate block is relatively low, and no fine correction is required.

[0045] When the ratio of local dark current variance to global dark current variance is between a first preset threshold and a second preset threshold, the candidate blocks are divided equally based on the size of the dynamic difference adaptation to obtain spatial blocks.

[0046] Specifically, the ratio of local dark current variance to global dark current variance being between the first preset threshold and the second preset threshold can be understood as the ratio of local dark current variance to global dark current variance being greater than or equal to the second preset threshold and less than or equal to the first preset threshold.

[0047] based on Determine the side length of the dynamically differentiating adaptation block. Among them, To dynamically adapt the block's side length to different dimensions, The side length of the smallest block. The side length of the largest block. For the local dark current variance, The variance of the global dark current.

[0048] When the number of remaining pixels is insufficient to form the size of the minimum complete block, an adjustment operation is performed.

[0049] The adjustment operation includes: adjusting the size of the previous divided spatial block sequentially from back to front to ensure that the number of remaining pixels can meet the size requirement of forming a complete minimum block.

[0050] If, after adjusting all spatial blocks to the size of the smallest block, the number of remaining pixels still cannot meet the size requirement for forming a complete smallest block, stop the adjustment operation and form a spatial block with the remaining pixels.

[0051] The following example illustrates the adjustment process described above. Assume the minimum block size is 5×5 pixels. The dynamic difference adaptation block size is determined to be 6×6 pixels. After equally dividing the candidate blocks based on the dynamic difference adaptation block size, the remaining portion of a certain block is 4×4 pixels, which is smaller than the minimum block size requirement of 5×5 pixels. In this case, the size of the previously divided spatial block needs to be adjusted. Specifically, the previous 6×6 pixel spatial block can be adjusted to 5×6 pixels, and the remaining portion will also be 5×6 pixels, thus satisfying the minimum block size requirement.

[0052] S103: Determine the initial black level calibration coefficient for each spatial block based on the dark pixel noise level.

[0053] Based on the dark pixel noise level, the initial black level calibration coefficients for each spatial block are determined, including the following:

[0054] Obtain the current signal value at the same position in the preset threshold frame before each dark pixel, and calculate the noise standard deviation of each dark pixel.

[0055] Specifically, the first preset threshold frame can be the first 5 frames. The noise standard deviation can be denoted as... . The larger the value, the greater the noise of the dark pixel.

[0056] Each dark pixel is assigned a corresponding weighting coefficient based on its noise standard deviation. The weighting coefficient is inversely proportional to the noise standard deviation of the dark pixel.

[0057] Specifically, in order to reduce the impact of noisy dark pixels, a corresponding weighting coefficient is assigned to the noise standard deviation of each dark pixel. , That is, dark pixels with high noise have low weight coefficients, while dark pixels with low noise have high weight coefficients.

[0058] The weighting coefficients are normalized, and the initial black level calibration coefficients for each spatial block are determined based on the normalized weighting coefficients and the current signal values ​​of the corresponding dark pixels.

[0059] Specifically, the normalization process for the weight coefficients is as follows: Calculate the sum of all weight coefficients, divide each weight coefficient by the sum of the weight coefficients to obtain the normalized weight coefficients, ensuring that the sum of all normalized weight coefficients is 1.

[0060] Specifically, the expression for determining the initial black level calibration coefficient for each spatial block is as follows: .in, This is the initial black level calibration coefficient for the current frame space block. This represents the current signal value of the dark pixel corresponding to the weighting coefficient. These are the weighting coefficients.

[0061] S104: Obtain the current temperature and temperature change rate of each space block, and based on the initial black level calibration coefficient corresponding to each space block, obtain the level calibration coefficient after temperature adaptive calibration of each space block.

[0062] based on Obtain the level calibration coefficients after temperature adaptive calibration for each spatial block. Among them, The level calibration coefficient is the temperature-adaptive calibration coefficient for the current frame space block. This is the initial black level calibration coefficient for the current frame space block. , , , and For temperature coefficient, The current temperature of the current frame space block. The rate of temperature change of the current frame space block. This is the temperature difference value. For time intervals.

[0063] Specifically, the current temperature and temperature change rate of each spatial block can be obtained through a temperature sensor.

[0064] Specifically, during the initial run, no temperature adaptive calibration is performed, and the stability coefficient in the model is set to [value missing]. , , The purpose of this setting is to avoid adverse effects on calibration results due to inaccurate temperature coefficients during the initial stage, ensuring a relatively stable baseline calibration value during initial operation. The above formula can be automatically fitted to determine a suitable temperature coefficient. , , , and Fitting methods can include least squares and backpropagation (BP) neural networks. As image sensors age, device aging occurs, and differences in manufacturing processes and wear rates among different devices mean that a fixed temperature coefficient cannot adapt to all temperatures and temperature change rates. When the calibrated level calibration coefficients show a divergent trend, and the divergence exceeds a threshold limit, the above formula is refitted based on automatically saved data and updated. This ensures that the formula always adapts to changes in device characteristics, providing accurate level calibration coefficients after temperature adaptive calibration for each spatial block.

[0065] S105: Fuse the calibration information from the previous frame with the level calibration coefficient of the current frame to obtain a stable cross-frame calibration result.

[0066] By fusing the calibration information from the previous frame and the level calibration coefficients of the current frame, a stable cross-frame calibration result is obtained, including the following:

[0067] The calibration value of the dark current of the spatial block in the current frame is obtained based on the optimal calibration estimate of the dark current of the spatial block in the previous frame.

[0068] Specifically, the expression for obtaining the calibration value of the spatial block dark current in the current frame based on the optimal calibration estimate of the spatial block dark current in the previous frame is as follows: .in, This is the dark current calibration value for the current frame space block. This is the optimal calibration estimate of the dark current in the spatial block of the previous frame. Here is the state transition matrix. For the control matrix, To control the quantity, , , .

[0069] Furthermore, the core state variables in this application involve only one key state quantity, namely the optimal calibration estimate of the dark current of the current frame space block. Therefore, in this application, the state transition matrix can be... Simplified to 1. Control matrix This is used to describe the effect of the control quantity on the state variable. In this application, there are no additional control factors to change the prediction process of the calibration value; therefore, the control matrix can be used... Simplify to 1, and control quantity It is 0.

[0070] The dark current prediction covariance of the spatial block in the current frame is obtained by adding the dark current calibration covariance of the spatial block in the previous frame to the process noise covariance.

[0071] Specifically, the expression for obtaining the dark current prediction covariance of the current frame spatial block is: .in, The covariance of dark current prediction for the current frame spatial block. To calibrate the covariance of the dark current in the spatial block of the previous frame, For process noise covariance, .

[0072] Based on the dark current prediction covariance and observation noise covariance of the current frame spatial block, the dark current gain of the current frame spatial block is obtained.

[0073] Specifically, the expression for obtaining the dark current gain of the current frame spatial block is: .in, The dark current gain of the current frame spatial block. The covariance of dark current prediction for the current frame spatial block. To observe the noise covariance, For the observation matrix, .

[0074] Furthermore, since the core state variables in this application involve only one key state quantity, namely the optimal calibration estimate of the dark current of the current frame space block, the observation matrix can be... Simplified to 1.

[0075] By combining the current frame spatial block dark current calibration value, the current frame spatial block dark current gain, and the level calibration coefficient after temperature adaptive calibration of the current frame spatial block, the optimal calibration estimate of the current frame spatial block dark current is obtained, and this optimal calibration estimate is used as the cross-frame stable calibration result. The current temperature of the current frame spatial block, the optimal calibration estimate of the current frame spatial block dark current, and the predicted covariance of the current frame spatial block dark current are used as spatial block storage values ​​for image processing in the next frame.

[0076] Specifically, the expression for obtaining the optimal calibration estimate of the dark current of the current frame spatial block is: .in, This is the optimal calibration estimate for the dark current of the current frame space block. This is the dark current calibration value for the current frame space block. The dark current gain of the current frame spatial block. The level calibration coefficient is the temperature-adaptive calibration coefficient for the current frame space block. .

[0077] Update the dark current prediction covariance of the current frame space block based on the dark current gain and dark current prediction covariance of the current frame space block.

[0078] Specifically, the expression for updating the dark current prediction covariance of the current frame spatial block is: .in, The updated covariance for predicting dark current in the current frame spatial block. It is the identity matrix. The dark current gain of the current frame spatial block. For the observation matrix, The covariance for predicting dark current in the current frame spatial block.

[0079] This application also includes the following.

[0080] Using the smallest block as the base block, the current temperature of the base block of the current frame, the optimal calibration estimate of the dark current of the base block of the current frame, and the calibration covariance of the dark current of the base block of the current frame are stored as the base block storage values.

[0081] When the spatial block division of the current frame and the previous frame is inconsistent, the spatial block storage value of the previous frame is obtained based on the basic block storage value. The method for obtaining the storage value includes the following:

[0082] When the current frame and the previous frame have overlapping base blocks in their spatial blocks, the average value of the base block stored in the previous frame for all overlapping base blocks is obtained and used as the spatial block stored value of the previous frame for spatial block calibration in the current frame.

[0083] When the current frame space block is completely contained by the previous frame space block, the stored value of the previous frame space block is directly obtained and used for the calibration of the current frame space block.

[0084] Specifically, this application uses the smallest block as the basic block for storage and updating, which will be explained in detail below with specific examples.

[0085] Since the largest block can be equally divided into the smallest blocks, the smallest block is chosen as the base block to store the current temperature of the current frame's base block, the optimal calibration estimate of the dark current of the current frame's base block, and the calibration covariance of the dark current of the current frame's base block. The advantage of this approach is that regardless of how subsequent blocks are dynamically divided, the new divisions can always establish a correspondence with the base block, ensuring that calibration information is accurately applied to the base block.

[0086] Assume the previous frame's image calibration used a 5x5 pixel block partitioning method, while the current frame is dynamically partitioned into 12x12 pixel blocks. During the value retrieval process, for each dynamically partitioned 12x12 pixel block, it's necessary to find its overlapping area with the 5x5 pixel base blocks. For example, a 12x12 pixel block might overlap with a 3x3 5x5 pixel area, which is 9 base blocks. At this point, the level calibration coefficient after the current frame's spatial block temperature adaptive calibration is combined with... The value is 7, and the closest one is selected from these 9 overlapping basic blocks. The value is 7. Following this method, values ​​are sequentially taken for the overlapping regions of each dynamically partitioned block and the base block. In cross-frame stabilization calibration, the optimal calibration estimate of the dark current in the spatial block of the previous frame is used. After relevant calculations, the value was determined to be 7. Following the completion of the value selection and related calculations, the optimal calibration estimate of the dark current in the current frame's spatial block was generated. Assuming the optimal calibration estimate of the dark current of the current frame space block after dynamic block generation... like Figure 2 As shown, Figure 2 This diagram illustrates the optimal calibration estimate of the dark current of the current frame spatial block generated by dynamic block partitioning, as provided in this embodiment of the application. The black border represents a 12×12 pixel dynamically partitioned block, and the numbers on the arrows represent the optimal calibration estimate of the dark current of the current frame spatial block generated by dynamic block partitioning. Figure 2 The values ​​marked in different regions represent the optimal calibration estimate of the dark current of the current frame spatial block in each region after calibration calculation. Subsequently, this frame is stored according to pixel values, and the optimal calibration estimate of the dark current of the current frame spatial block corresponding to each pixel position is recorded to provide a basis for calibration calculation of subsequent frames.

[0087] S106: Based on the cross-frame stable calibration results, the effective pixels in each spatial block are corrected to obtain the corrected pixel values, and the corrected pixel values ​​are reconstructed into the output image.

[0088] Based on the cross-frame stable calibration results, the effective pixels in each spatial block are corrected to obtain the corrected pixel values. The corrected pixel values ​​are then reconstructed into the output image, which includes the following:

[0089] The difference is obtained by subtracting the corresponding cross-frame stabilization calibration result from the current value of the effective pixels in the spatial block. The expression is: .in, These are the corrected pixel values. This represents the current value of the effective pixels within the spatial block. This is the optimal estimate for the calibration of the dark current in the current frame space block.

[0090] The optimal estimate of the current frame calibration in this application is the cross-frame stable calibration result.

[0091] Since a pixel value less than 0 is not physically valid and has no corresponding color representation, it is necessary to restrict the calculated difference. If the difference is less than 0, the corrected pixel value is set to 0. If the difference is greater than 0, the difference is used as the corrected pixel value.

[0092] This application also provides a dynamic adaptive black level correction device 300, such as... Figure 3 As shown, the device includes: a data acquisition module 301, a division module 302, a determination module 303, an adjustment module 304, a fusion module 305, and an output module 306.

[0093] The acquisition module 301 is used to acquire multiple frames of raw images output by the image sensor.

[0094] The segmentation module 302 is used to dynamically segment the effective pixel region in each frame of the original image according to the difference in the spatial distribution of dark current, so as to obtain multiple spatial blocks.

[0095] The determination module 303 is used to determine the initial black level calibration coefficient for each spatial block based on the dark pixel noise level.

[0096] The adjustment module 304 is used to obtain the current temperature and temperature change rate of each space block, and based on the initial black level calibration coefficient corresponding to each space block, obtain the level calibration coefficient after temperature adaptive calibration of each space block.

[0097] The fusion module 305 is used to fuse the calibration information of the previous frame and the level calibration coefficient of the current frame to obtain a stable calibration result across frames.

[0098] The output module 306 is used to correct the effective pixels in each spatial block based on the cross-frame stable calibration results, obtain the corrected pixel values, and reconstruct the corrected pixel values ​​into an output image.

[0099] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0100] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0101] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, for example, as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0102] like Figure 4 As shown in the figure, this application embodiment also provides a dynamic adaptive black level correction server, including a memory 401 and a processor 402; the memory 401 is used to store computer-executable instructions; the processor 402 is used to execute the computer-executable instructions to implement the dynamic adaptive black level correction method described above in this application embodiment.

[0103] This application also provides a computer-readable storage medium storing executable instructions, which, when executed by a computer, enable the implementation of the dynamic adaptive black level correction method described above in this application.

[0104] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0105] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations.

[0106] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method of dynamic adaptive black level correction, characterized in that, The method comprises the following steps: acquiring a plurality of original images output by an image sensor; dynamically dividing an effective pixel region in each original image according to a dark current spatial distribution difference to obtain a plurality of spatial blocks; determining an initial black level calibration coefficient of each spatial block based on a dark pixel noise level; obtaining a current temperature and a temperature change rate of each spatial block, and obtaining a level calibration coefficient of each spatial block after temperature self-adaptive calibration based on the initial black level calibration coefficient corresponding to each spatial block; fusing calibration information of a previous frame and the level calibration coefficient of a current frame to obtain a cross-frame stable calibration result; correcting effective pixels in each spatial block based on the cross-frame stable calibration result to obtain corrected pixel values, and reconstructing the corrected pixel values into an output image; the step of dynamically dividing the effective pixel region in each original image according to the dark current spatial distribution difference comprises the following steps: obtaining an average value of dark current of a dark pixel region in a current frame, and obtaining a dark current spatial distribution variance based on the average value, which is used as a global dark current variance; the size of a minimum block and the size of a maximum block are set, and the length of the maximum block is an integer multiple of the length of the minimum block; a sliding window scan is performed on the maximum block as a candidate block; when the number of remaining pixels cannot form a complete maximum block, the remaining pixels form a candidate block; a dark current variance of dark pixels in each candidate block is obtained, which is used as a local dark current variance; and the candidate block is divided into spatial blocks based on the ratio of the local dark current variance to the global dark current variance.

2. The dynamic adaptive black level correction method of claim 1, wherein, the step of dividing the candidate block into spatial blocks based on the ratio of the local dark current variance to the global dark current variance comprises the following steps: when the ratio of the local dark current variance to the global dark current variance is greater than a first preset threshold, the candidate block is equally divided into spatial blocks according to the size of the minimum block; when the number of remaining pixels cannot form a complete minimum block, the remaining pixels form a spatial block; when the ratio of the local dark current variance to the global dark current variance is less than a second preset threshold, the candidate block is directly used as a spatial block; when the ratio of the local dark current variance to the global dark current variance is between the first preset threshold and the second preset threshold, the candidate block is equally divided into spatial blocks based on a dynamic difference adaptive block size; when the number of remaining pixels cannot form a complete minimum block, an adjustment operation is performed; the adjustment operation comprises adjusting the size of a previously divided spatial block from back to front to ensure that the number of remaining pixels can meet the requirement of forming a complete minimum block; if the number of remaining pixels still cannot meet the requirement of forming a complete minimum block after all spatial blocks are adjusted to the size of the minimum block, the adjustment operation is stopped, and the remaining pixels form a spatial block.

3. The dynamic adaptive black level correction method of claim 2, wherein, based on determining a side length of a dynamic disparity adaptation tile; wherein, for a side length of a dynamic disparity adaptation tile, for a side length of a minimum tile, for a side length of a maximum tile, for a local dark current variance, for a global dark current variance.

4. The dynamic adaptive black level correction method of claim 3, wherein, the step of determining the initial black level calibration coefficient of each spatial block based on the dark pixel noise level comprises the following steps: obtaining current signal values of each dark pixel at the same position in a preset threshold frame, and calculating a noise standard deviation of each dark pixel. The weight coefficient is inversely proportional to the noise standard deviation of the dark pixel; The weight coefficient is normalized, and based on the normalized weight coefficient and the current signal value of the corresponding dark pixel, an initial black level calibration coefficient of each spatial block is determined.

5. The dynamic adaptive black level correction method of claim 4, wherein, The current temperature and temperature change rate of each spatial block are obtained, and based on the initial black level calibration coefficient corresponding to each spatial block, a level calibration coefficient of each spatial block after temperature self-adaptive calibration is obtained, including: Based on obtaining a level calibration coefficient of each spatial block after temperature self-adaptive calibration; wherein, is a level calibration coefficient of a current frame spatial block after temperature self-adaptive calibration, is an initial black level calibration coefficient of a current frame spatial block, , , , and is a temperature coefficient, is a current temperature of a current frame spatial block, is a temperature change rate of a current frame spatial block, is a temperature difference value, is a time interval.

6. The dynamic adaptive black level correction method of claim 5, wherein, The calibration information of the previous frame and the level calibration coefficient of the current frame are fused to obtain a cross-frame stable calibration result, including: The calibration optimal estimate value of the dark current of the spatial block of the previous frame is obtained to obtain the dark current calibration value of the spatial block of the current frame; The dark current prediction covariance of the spatial block of the current frame is obtained by adding the dark current calibration covariance of the spatial block of the previous frame and the process noise covariance; The dark current gain of the spatial block of the current frame is obtained according to the dark current prediction covariance of the spatial block of the current frame and the observation noise covariance; The calibration optimal estimate value of the dark current of the spatial block of the current frame is obtained by combining the dark current calibration value of the spatial block of the current frame, the dark current gain of the spatial block of the current frame, and the level calibration coefficient of the spatial block of the current frame after temperature self-adaptive calibration, and the calibration optimal estimate value of the dark current of the spatial block of the current frame is taken as the cross-frame stable calibration result; wherein the current temperature of the spatial block of the current frame, the calibration optimal estimate value of the dark current of the spatial block of the current frame, and the dark current prediction covariance of the spatial block of the current frame are used as spatial block storage values for image processing of the next frame.

7. The dynamic adaptive black level correction method of claim 6, wherein, Further comprising: The current temperature of the current frame basic block, the calibration optimal estimate value of the dark current of the current frame basic block, and the dark current calibration covariance of the current frame basic block are saved as basic block storage values based on the minimum block as the basic block; When the spatial block division of the current frame and the previous frame is inconsistent, the previous frame spatial block storage value is obtained based on the basic block storage value, and the obtaining method comprises: When the current frame and the previous frame spatial block have overlapping basic blocks, the average value of the basic block storage values of all overlapping basic blocks in the previous frame is obtained as the previous frame spatial block storage value, which is used for calibration of the current frame spatial block; When the current frame spatial block is completely contained in the previous frame spatial block, the previous frame spatial block storage value is directly obtained, which is used for calibration of the current frame spatial block.

8. The dynamic adaptive black level correction method of claim 7, wherein, The effective pixels in each spatial block are corrected based on the cross-frame stable calibration result to obtain corrected pixel values, and the corrected pixel values are reconstructed into an output image, including: The current value of the effective pixel in the spatial block is subtracted by the corresponding cross-frame stable calibration result to obtain a difference value; If the difference value is less than 0, the corrected pixel value is set to 0; If the difference value is greater than 0, the difference value is taken as the corrected pixel value.

9. A dynamic adaptive black level correction device, characterized by, Comprising: The acquisition module is used for acquiring multiple frames of original images output by the image sensor; The division module is used for dynamically dividing the effective pixel region in each frame of original image according to the dark current spatial distribution difference to obtain multiple spatial blocks; The determination module is used for determining the initial black level calibration coefficient of each spatial block based on the dark pixel noise level; The adjusting module is configured to obtain the current temperature and the temperature change rate of each spatial block, and obtain the level calibration coefficient of each spatial block after adaptive calibration of the temperature based on the initial black level calibration coefficient corresponding to each spatial block; The fusing module is configured to fuse the calibration information of the previous frame and the level calibration coefficient of the current frame to obtain a cross-frame stable calibration result; The output module is configured to correct the effective pixels in each spatial block based on the cross-frame stable calibration result to obtain corrected pixel values, and reconstruct the corrected pixel values into an output image. The method comprises the following steps: obtaining the average value of the dark current of the dark pixel area of the current frame, and obtaining the variance of the spatial distribution of the dark current based on the average value, which is used as the global dark current variance; The size of the minimum block and the maximum block is set, and the length of the maximum block is an integer multiple of the length of the minimum block. The maximum block is used as a candidate block to perform sliding window scanning. When the number of remaining pixels cannot form a complete maximum block, the remaining pixels form a candidate block. The dark current variance of the dark pixels of each candidate block is obtained, which is used as the local dark current variance. The candidate block is divided into spatial blocks based on the ratio of the local dark current variance to the global dark current variance.

Images