Automatic exposure method and control device for adaptively suppressing overexposure and overdarkness

Abstract

The present invention discloses an automatic exposure method and control device for adaptively suppressing overexposure and overdarkness. Aiming at the requirements of image acquisition for the overall brightness of the image and the suppression of overexposure and overdarkness, and taking into account the different requirements for image brightness in different scenes, the method is based on the current frame image, and obtains feature parameters by combining the spatial domain and the value range weighted average method, and the feature parameters are made to approach the target feature parameters through an iterative method, thereby obtaining automatic exposure parameters. Among them, the spatial domain weight is based on the consideration of the impact of the image brightness distribution on human vision, and an image that is more in line with observation habits can be obtained. The value range weight can reduce the problems of overexposure and partial overdarkness by giving higher weights to highlight and extremely dark areas. The method and device can adaptively achieve appropriate brightness for various different scenes.

Description

Technical Field

[0001] The present invention relates to an image acquisition control method and device, and in particular to an automatic exposure control method and device. Background Art

[0002] Digital image acquisition is the process of converting light signals into electrical signals using sensors such as cameras and storing, transmitting and displaying them in digital form. Automatic exposure refers to the method by which the image sensor automatically adjusts parameters such as exposure time and exposure gain according to the ambient brightness so that the brightness of the collected image signal is within a reasonable range.

[0003] Automatic exposure is widely used in all fields that need to collect image signals, such as cameras, surveillance, vehicle-mounted imaging, target detection, etc. In the fields of smart cars, smart homes, smart surveillance, etc., automatic exposure plays a great role in collecting initial images with appropriate brightness and improving image quality, so a more effective automatic exposure method has practical value.

[0004] During automatic exposure, it is usually necessary to comprehensively consider the overall brightness of the image, the overexposure problem of the bright area, the overdarkness problem of the dark area, and the position of areas of different brightness in the image, and achieve a more appropriate exposure intensity through the iteration of multiple frames of images.

[0005] Most of the current mainstream automotive image processors use average or weighted average to calculate the mean value, and then use multiple frame iterations to make it close to the target value to achieve automatic exposure. The advantage is that it is relatively simple to implement, but the disadvantage is that it is difficult to effectively suppress problems such as overexposure and darkening of some areas in more complex scenes. Summary of the invention

[0006] Purpose of the invention: In view of the above-mentioned prior art, an automatic exposure method and control device for adaptively suppressing overexposure and overdarkness are proposed, which can adaptively achieve appropriate brightness for various scenes.

[0007] Technical solution: An automatic exposure method for adaptively suppressing overexposure and overdarkness, comprising: based on the current frame image, obtaining feature parameters by combining the method of spatial domain and value range weighted average, and making the feature parameters approach the target feature parameters by an iterative method, thereby obtaining automatic exposure parameters; wherein the method for obtaining the feature parameters specifically comprises: dividing the image into m × n There are rectangular pixel blocks of the same size, and the average pixel value of the pixel block in the i-th row and j-th column is , the characteristic parameters are calculated according to the following formula c n :

[0008] ; In the formula, Wij is the spatial weight, yes The corresponding range weight; during the iteration process, for the feature parameters c n ,according to The convergence condition of the exposure parameter is determined. When the convergence condition is met, the exposure parameters of the next frame remain the same as the current frame. When the convergence condition is not met, the exposure parameters of the next frame are calculated according to the following formula E n+1 Sure:

[0009] ; In the formula, i To determine the parameters of the convergence range; E n is the exposure intensity of the current frame, E n+1 is the exposure intensity of the next frame; c 0 is the target feature parameter; d is the step length, and its value is i The values ​​are positively correlated.

[0010] Furthermore, d The value range is [0.2,0.8], i The value range is [0.05,0.2].

[0011] Furthermore, the spatial weight W ij The setting of depends on the distance between the pixel block and the center of the image. The pixel block in the middle of the image is given a higher weight. The setting of the range weight is based on the consideration of overexposure and dark suppression. The bright and dark areas in the picture are given higher weights, which makes the image feature parameter values ​​of the overbright parts higher and the image feature parameter values ​​of the dark parts lower.

[0012] Furthermore, the spatial weight W ij Using the Gaussian weighting method, the coordinates of the center point of the image are recorded as ( x 0, y 0), the coordinates of the center point of the rectangular pixel block in the i-th row and j-th column are ( x ij , y ij ),but ;in, s Parameters for determining the degree of weight increase for the center; L is the side length of the rectangular pixel block.

[0013] Further, the range weights are obtained by a sorting-based method or a mapping-based method, or by a combination of sorting-based and mapping-based weights.

[0014] Further, the sorting-based method, that is, m × n The pixel blocks are arranged in order of brightness from large to small, with higher weights given to the brightest blocks and the darkest blocks, and the rest have the same weights.

[0015] Furthermore, a mapping-based method is used, that is, a mapping from a value range to a weight is established, so that the weights for dark and bright areas are larger, and the weights for medium brightness are smaller; the mapping is obtained by setting a function or a table lookup method.

[0016] Furthermore, the setting function is to define a mapping function that meets the target requirements, and the mapping function is one of the following two functions:

[0017] ;in, I max It is the maximum value of the pixel value range; W 0 is a constant, which is the basic weight value without weight increase; k 1. k 2 is the parameter that determines the relative increase ratio of the dark and bright weights; c 1. c 2 is the parameter that determines the shape of the function, reflecting the overall increase in the weight of the bright and dark parts.

[0018] Furthermore, the spatial domain weight, the value domain weight and the target feature parameter are obtained based on a machine learning method, including the following steps:

[0019] Step 1: Obtain a training set, that is, collect images of several application scenarios as training sets according to the requirements of the application scenarios; the size of the images in the training set is the same as the size of the images collected by the image acquisition device;

[0020] The second step: brightness statistics, that is, dividing each image in the training set into m × n rectangular pixel blocks of the same size; for each image in the training set, the average pixel value in each pixel block is calculated; for the pixel block in the i-th row and j-th column of image I in the training set, the average value is , calculate the distance between the center of each pixel block and the center of the image, recorded as ;

[0021] Step 3: Transform the problem into a solution , that is, an optimization problem. By solving this optimization problem, we can get the corresponding W ij , W r , c 0.

[0022] An automatic exposure control device for adaptively suppressing overexposure and overdarkness comprises a memory, a processor and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the automatic exposure method for adaptively suppressing overexposure and overdarkness.

[0023] Beneficial effects: In view of the requirements of image acquisition for the overall brightness of the image and the suppression of overexposure and over-darkness, and taking into account the different requirements for image brightness in different scenes, this method adopts a method combining value range and spatial domain weighted averaging to adjust the exposure parameters to achieve automatic exposure, and provides a weight setting method based on image properties and machine learning.

[0024] Specifically, this method obtains feature parameters by combining spatial domain and range weighted average, and makes the feature parameters approach the target value through iteration, so as to obtain appropriate automatic exposure parameters. Spatial domain weights are based on the consideration of the impact of image brightness distribution on human vision, and can obtain images that are more in line with observation habits. Range weights can reduce overexposure and partial darkening problems by giving higher weights to highlight and extremely dark areas.

[0025] The weights can be obtained by methods based on image properties or machine learning. The method based on image properties provides mapping-based and sorting-based solutions, while the method based on machine learning can obtain a more accurate weight setting solution based on the use of image properties.

[0026] Compared with the existing technology, this method can adaptively achieve appropriate brightness for various scenes (including special and extreme scenes). This method can effectively suppress overexposure and dark problems, and consider the impact of brightness at different positions on the image effect. The main calculation of this method is concentrated in the weight setting step, which can be implemented at low cost and high efficiency when applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flow chart of the method of the present invention in the embodiment. DETAILED DESCRIPTION

[0028] The present invention will be further explained below in conjunction with the accompanying drawings.

[0029] like Figure 1 As shown, an automatic exposure method for adaptively suppressing overexposure and overdarkness is divided into four steps:

[0030] Step 1: Initialize exposure parameters.

[0031] The exposure intensity is EDifferent image acquisition devices have different exposure parameters, and the final exposure intensity is determined by adjusting the exposure parameters. For example, for some image acquisition devices, the exposure time t and the exposure gain representing the light sensitivity of the photosensitive element are g Will affect the final exposure intensity, that is, the exposure intensity E = f 1( g , t ),in f 1 is a function that depends on the hardware properties of the device. For some image acquisition devices, the exposure intensity is changed only by adjusting the exposure time, that is, E = f 2( t ). The relationship between exposure parameters and exposure intensity belongs to the nature of image acquisition equipment. The present invention does not involve innovation of related content. E It can be obtained by adjusting the actual parameters according to the properties of the equipment.

[0032] In the first frame, the exposure parameters are initialized so that the exposure intensity is E 0, the collected image is recorded as P original , which is usually an image in Bayer format. In the automatic exposure process, it is necessary to count the brightness information of the image for adjusting the exposure parameters of the next frame. Before counting the brightness information, the image is usually preprocessed. This preprocessing process is recorded as f , generally including noise reduction, AWB, demosaic and other processes, which are related to the application requirements of the image and are not within the scope of discussion of this invention. P = f ( P original ) is a preprocessed image used to count brightness information.

[0033] Step 2: Image P Perform brightness statistics to obtain characteristic parameters c .

[0034] Specifically, the image P Divide into m × n The image can be divided into 32×18 60×60 pixel blocks, or 64×36 30×30 pixel blocks. If the image cannot be divided evenly, the division can be completed by adding pixels at the edge, which will not be described here. The more detailed the division of the image, the better the final effect, but the higher the requirement for computing resources. Usually, m × nWhen the value is in the range of 200-1000, better results can be achieved without consuming too much computing resources.

[0035] For each rectangular pixel block after division, calculate the average pixel value of the pixel block. The average pixel value of the pixel block in the i-th row and j-th column is , use the following formula to calculate the characteristic parameters c :

[0036]

[0037] in, W ij is the spatial weight, which depends on the position of the pixel block in the image. The spatial weight is set based on the characteristics of the human eye. The human eye is more sensitive to the middle part of the picture, so a higher weight should be given. W ij is the spatial weight, yes The corresponding range weight. The range weight is set for the purpose of suppressing overexposure and dark areas. Overexposure and dark areas in the picture will have a greater impact on the image quality. Therefore, higher weights are given to the bright and dark areas in the picture, which are reflected in the feature parameters. c In the example, the image feature parameter value of the overly bright part is higher, and the brightness is suppressed during the iteration process, and the image feature parameter value of the overly dark part is lower, and the brightness is increased during the iteration process. The specific setting methods for these two sets of weights are given in the subsequent parameter description section.

[0038] Step 3: Based on feature parameters c Update exposure parameters.

[0039] The goal of this step is to make the exposure parameters of the next frame close to the target exposure parameters. If the exposure intensity of the current frame is E n , the corresponding characteristic parameters are calculated as c n , then the exposure intensity of the next frame is obtained according to the following formula .in, c 0 is the predetermined target characteristic parameter; d ∈(0,1] is the step size, and its value is generally set in the range of [0.2,0.8], which is related to the convergence condition.

[0040] According to the properties of the image sensor in the image acquisition device, usually the image brightness and exposure intensity of the acquired image E The relationship is approximately linear, and the characteristic parameters c Characterize the image brightness. This step can make the characteristic parameters of the next frame image c n+1 Closerc 0, that is . Introducing step size d The reason is that the image brightness and exposure intensity are not completely linear. By introducing the step size, we can prevent the non-convergence caused by excessive changes in exposure parameters.

[0041] Step 4: Determine the convergence conditions.

[0042] For characteristic parameters c n ,according to If the convergence condition is not met, the exposure parameters of the next frame are calculated according to the third step. E n+1 When the convergence condition is met, the exposure parameters of the next frame remain the same as the current frame, that is, E n+1 = E n .in, i To determine the parameters of the convergence range, the general value range is [0.05, 0.2]. i The smaller the value, the closer the final result is to the target value, but the d The smaller the value, the slower the convergence, and vice versa. Table 1 gives the different i Corresponding d The purpose of convergence judgment is to maintain the stability of the image brightness. A slight deviation between the brightness and the target value is within an acceptable range, but unstable brightness will have a greater impact on video quality. After the convergence conditions are met, the exposure parameters will remain stable until the scene of the image acquisition changes to a certain extent.

[0043] Table 1

[0044] 0.05 0.08 0.1 0.15 0.2 [0.2,0.4] [0.25,0.5] [0.3,0.6] [0.4,0.7] [0.5,0.8]

[0045] The next frame is exposed according to the convergence condition determination result, and the iteration is continued. Meanwhile, the current frame image is further processed as required, and the processing content varies according to different application scenarios and does not belong to the solution of the present invention.

[0046] In the above process, the spatial domain weight and range weight when calculating the feature parameters in step 2 and the target feature parameters in step 3 c 0 needs to be obtained in advance, which is explained in detail below.

[0047] The above parameters can be obtained through two methods: image property-based method and machine learning-based method. The image property-based method mainly builds a model for the above parameters based on image properties, and then determines the specific parameters through testing. The machine learning-based method is based on statistical learning of a large amount of image information, which is more accurate, but has higher requirements for the training set.

[0048] In the method based on image properties, the spatial weight W ij It is obtained based on the distance between the center of the rectangular pixel block and the center of the image. You can refer to the general center weighting method, such as Gaussian weighting, that is, the coordinates of the center point of the image are recorded as ( x 0, y 0), the coordinates of the center point of the rectangular pixel block in the i-th row and j-th column are ( x ij , y ij ),but .in, s In order to determine the parameters for the degree of weight enhancement of the center, it is generally taken s ≥10; L is the side length of the rectangular pixel block.

[0049] The range weights can be obtained through a sorting-based approach or a mapping-based approach.

[0050] The sorting-based method is m × n Pixel blocks are divided into brightness blocks (the average pixel value calculated ) are arranged in order from large to small. The brightest and darkest blocks are given higher weights, and the rest are given the same weights. Generally, overexposure has a greater impact on image quality, so the bright part is given a higher weight, and the dark part is given a lower weight than the bright part.

[0051] Based on the mapping method, a mapping from a range to a weight is established, so that the weights for dark and bright areas are larger, and the weights for medium brightness are smaller. The mapping can be obtained by setting a function or looking up a table.

[0052] Setting a function means defining a mapping function that meets the target requirements. The following are two examples of mapping functions:

[0053]

[0054] in, Represents the average pixel value of any pixel block, [0, I max ] is the value range of pixel values, where I max is the maximum value of the pixel value range, for example, for 8-bit data,I max =255; W 0 is a constant, which is the basic weight value without any weight increase. k 1. k 2 is the parameter that determines the relative increase ratio of the dark and bright weights. Generally, the bright part needs to be increased more, so k 1∈[0,1 / 2] range. c 1. c 2 is the parameter that determines the shape of the function, reflecting the overall increase in the weight of the bright and dark parts. It should be noted that the range given here is a recommended range.

[0055] The table lookup rule is based on the fact that the pixel value range is generally discrete. A corresponding weight is assigned to each brightness and stored. The weight can be determined by the table lookup method. The table lookup can also be simplified to a piecewise linear function to reduce storage resource consumption.

[0056] The weights of the above parameters and the table lookup method will vary according to the image acquisition scenario and application requirements, so they need to be tested according to specific circumstances. c 0 is also obtained at the same time in this test. In addition, sorting-based and mapping-based weights can be used at the same time, that is, W r ( p )= W order ( p )∙ W map ( p ),in W order ( p ), W map ( p ) are the weights based on sorting and mapping respectively.

[0057] The machine learning method uses a large number of pictures with appropriate brightness as training sets to learn the appropriate weight parameters. The specific implementation is divided into the following steps:

[0058] Step 1: Obtain a training set, that is, according to the needs of the application scenario, collect enough images with appropriate brightness of the application scenario as a training set. The size of each image should be the same as the size of the image acquisition device. The selected scenes should try to include various types of scenes that require image signal acquisition during application, and the more pictures, the better the effect. Among them, the selected pictures should be preprocessed f After the image.

[0059] Step 2: Brightness statistics, that is, divide all images in the training set into m × nFor each image in the training set, the average pixel value in each pixel block is calculated. The average value of the pixel block in the i-th row and j-th column of image I in the training set is , calculate the distance between the center of each pixel block and the center of the image, recorded as .

[0060] Step 3: Transform the problem into a solution , which is an optimization problem. The corresponding W ij , W r , c 0. This can be solved by basic machine learning methods, which I will not go into detail here.

[0061] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An automatic exposure method for adaptively suppressing overexposure and overdarkness, characterized in that: include: Based on the current frame image, characteristic parameters are obtained by combining the spatial domain and the value domain weighted average method, and the characteristic parameters are made to approach the target characteristic parameters by an iterative method, so as to obtain automatic exposure parameters; wherein the method for obtaining the characteristic parameters specifically includes: dividing the image into m × n There are rectangular pixel blocks of the same size, and the average pixel value of the pixel block in the i-th row and j-th column is , the characteristic parameters are calculated according to the following formula c n : ; In the formula, W ij is the spatial weight, yes The corresponding range weight; during the iteration process, for the feature parameters c n ,according to The convergence condition of the exposure parameter is determined. When the convergence condition is met, the exposure parameters of the next frame remain the same as the current frame. When the convergence condition is not met, the exposure parameters of the next frame are calculated according to the following formula E n+1 Sure: ; In the formula, θ To determine the parameters of the convergence range; E n is the exposure intensity of the current frame, E n+1 is the exposure intensity of the next frame; c 0 is the target feature parameter; δ is the step length, and its value is θ The values ​​are positively correlated; Airspace Weight W ij The setting of depends on the distance between the pixel block and the center of the image. The pixel block in the middle of the image is given a higher weight. The setting of the range weight is based on the consideration of overexposure and dark suppression. The bright and dark areas in the picture are given higher weights, which makes the image feature parameter value of the overbright part higher and the image feature parameter value of the dark part lower.

2. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 1, characterized in that: δ The value range is [0.2,0.8], θ The value range is [0.05,0.2].

3. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 1, characterized in that: Airspace Weight W ij Using the Gaussian weighting method, the coordinates of the center point of the image are recorded as ( x 0, y 0), the coordinates of the center point of the rectangular pixel block in the i-th row and j-th column are ( x ij , y ij ),but ;in, σ Parameters for determining the degree of weight increase for the center; L is the side length of the rectangular pixel block.

4. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 1 or 3, characterized in that: The range weights are obtained by a sorting-based method or a mapping-based method, or a combination of sorting-based and mapping-based weights.

5. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 4, characterized in that: The sorting-based method is m × n The pixel blocks are arranged in descending order of brightness, with higher weights given to the brightest blocks and the darkest blocks, and the rest have the same weights.

6. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 4, characterized in that: The mapping-based method is to establish a mapping from a value range to a weight, so that the weights for dark and bright parts are larger, and the weights for medium brightness are smaller; the mapping is obtained by setting a function or a table lookup method.

7. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 6, characterized in that: The setting function is to define a mapping function that meets the target requirements. The mapping function is one of the following two functions: ;in, I max It is the maximum value of the pixel value range; W 0 is a constant, which is the basic weight value without weight increase; k 1. k 2 is the parameter that determines the relative increase ratio of the dark and bright weights; γ 1. γ 2 is the parameter that determines the shape of the function, reflecting the overall increase in the weight of the bright and dark parts.

8. The automatic exposure method for adaptively suppressing overexposure and overdarkness according to claim 1, characterized in that: Spatial weights, range weights, and target feature parameters are obtained based on machine learning methods, including the following steps: Step 1: Obtain a training set, that is, collect images of several application scenarios as training sets according to the requirements of the application scenarios; the size of the images in the training set is the same as the size of the images collected by the image acquisition device; The second step: brightness statistics, that is, dividing each image in the training set into m × n rectangular pixel blocks of the same size; for each image in the training set, the average pixel value in each pixel block is calculated; for the pixel block in the i-th row and j-th column of image I in the training set, the average value is , calculate the distance between the center of each pixel block and the center of the image, recorded as ; Step 3: Transform the problem into a solution , that is, an optimization problem. By solving this optimization problem, we can get the corresponding W ij , W r , c 0.

9. An automatic exposure control device for adaptively suppressing overexposure and overdarkness, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automatic exposure method for adaptively suppressing overexposure and overdarkness as described in any one of claims 1 to 8.

Images