Automatic exposure adjusting method

Abstract

The invention provides an automatic exposure adjusting method, and relates to the technical field of camera shooting, and the method comprises the steps: obtaining a current RAW image outputted by an image sensor; determining a target image state of the current RAW image; the target image state comprises a target overexposure state and a target too dark state; the target overexposure state is used for representing the overexposure degree of the current RAW image, and the target over-dark state is used for representing the over-dark degree of the current RAW image; determining a target exposure adjustment strategy based on the target overexposure state and the target too dark state; and performing exposure adjustment on the RAW image after the current RAW image based on the target exposure adjustment strategy. When the exposure adjustment strategy is determined, determination is directly carried out based on the RAW image output by the image sensor, the exposure adjustment strategy is determined through the overexposure state and the over-dark state of the RAW image, the RAW image does not need to be converted into an RGB image, the picture brightness of the image does not need to be determined, the calculation amount is small, and therefore the power consumption of the monitoring device is reduced.

Description

Technical Field

[0001] The present invention relates to the field of camera technology, and in particular to an automatic exposure adjustment method. Background Art

[0002] At present, monitoring scenarios are becoming more and more extensive, but some scenarios cannot power monitoring equipment through power supply, such as smart locks, doorbells, etc. The use environment of such monitoring equipment cannot be powered by power supply, and can only be powered by batteries. Therefore, low-power monitoring equipment such as battery models are becoming more and more widely used, but in order to ensure the long-term stable operation of monitoring equipment, higher requirements are also put forward for the power consumption of monitoring equipment. How to further reduce the power consumption of monitoring equipment has become an urgent problem to be solved in the industry.

[0003] In the related art, the automatic exposure (AE) adjustment method usually converts the RAW image output by the image sensor into an RGB image, and then divides the entire frame of the RGB image into M*N areas, calculates the statistical information of each area, such as the statistical information including R / Gr, B / Gb or brightness distribution histogram, etc., and then obtains the current picture brightness of the frame image based on the weight table and the statistical information of each area, and compares the current picture brightness with the target brightness to obtain whether the direction of AE adjustment is pressing or brightening; finally, the shutter value and gain value are determined based on the direction of AE adjustment, and the value of the register in the image sensor is changed by adjusting the shutter value and the gain value to perform exposure adjustment until the difference between the current picture brightness and the target brightness is within the tolerance range, and the AE adjustment is completed.

[0004] However, in the above-mentioned related technologies, when determining the AE adjustment direction of the next frame image, it is necessary to convert the RAW image output by the image sensor into an RGB image, and calculate the picture brightness of the current frame image based on the RGB image. The amount of calculation is too large, and a large amount of calculation consumes more electricity, thereby resulting in higher power consumption of the monitoring equipment. Summary of the invention

[0005] In view of the problems existing in the prior art, an embodiment of the present invention provides an automatic exposure adjustment method.

[0006] The present invention provides an automatic exposure adjustment method, comprising:

[0007] Get the current RAW image output by the image sensor;

[0008] Determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0009] Determining a target exposure adjustment strategy based on the target overexposure state and the target underdark state;

[0010] Exposure adjustment is performed on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0011] According to an automatic exposure adjustment method provided by the present invention, the determining of the target image state of the current RAW image includes:

[0012] Sampling pixel points in the current RAW image to obtain at least one sampled pixel point;

[0013] For each of the sampling pixel points, when it is determined that the pixel value of the sampling pixel point is greater than or equal to the initial brightness threshold, updating the number of overexposed pixel points;

[0014] When it is determined that the pixel value of the sampled pixel point is less than the initial brightness threshold, updating the number of dark pixels;

[0015] The target overexposure state is determined based on the number of overexposed pixels after the final update, and the target overdark state is determined based on the number of overdark pixels after the final update.

[0016] According to an automatic exposure adjustment method provided by the present invention, the number of overexposed pixels includes a first number of overexposed pixels at a first overexposure level and a second number of overexposed pixels at a second overexposure level; the first overexposure level is greater than the second overexposure level;

[0017] The updating of the number of overexposed pixels includes:

[0018] When the pixel value of the sampling pixel point is greater than or equal to the first pixel point overexposure threshold, updating the first overexposed pixel point quantity; the first pixel point overexposure threshold is greater than the initial brightness threshold;

[0019] When the pixel value of the sampling pixel point is less than the first pixel point overexposure threshold and greater than or equal to the second pixel point overexposure threshold, updating the second overexposed pixel point number;

[0020] The determining the target overexposure state based on the number of overexposed pixels after the final update includes:

[0021] When it is determined that the number of first overexposed pixel points after the final update is greater than or equal to the first image overexposure threshold, determining that the target overexposure state is the first state;

[0022] When it is determined that the number of the first overexposed pixel points after the final update is less than the first image overexposure threshold, and the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is greater than or equal to the second image overexposure threshold, determining that the target overexposure state is the second state; and the overexposure degree corresponding to the second state is less than the overexposure degree corresponding to the first state;

[0023] When it is determined that the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is less than the overexposure threshold of the second image, the target overexposure state is determined to be the third state; and the overexposure degree corresponding to the third state is less than the overexposure degree corresponding to the second state.

[0024] According to an automatic exposure adjustment method provided by the present invention, the number of overly dark pixels includes a first number of overly dark pixels of a first overly dark level and a second number of overly dark pixels of a second overly dark level; the first overly dark level is greater than the second overly dark level;

[0025] The updating of the number of dark pixels includes:

[0026] When the pixel value of the sampled pixel point is less than or equal to the first pixel point too dark threshold, the first pixel point too dark threshold is updated; and the first pixel point too dark threshold is less than the initial brightness threshold;

[0027] When the pixel value of the sampling pixel point is greater than the first pixel point too dark threshold and less than or equal to the second pixel point too dark threshold, updating the second too dark pixel point number;

[0028] The determining the target over-dark state based on the number of over-dark pixels after the final update includes:

[0029] When it is determined that the number of the first overly dark pixels after the final update is greater than or equal to the first image overly dark threshold, determining that the target overly dark state is the fourth state;

[0030] When it is determined that the number of the first overly dark pixels after the final update is less than the first image overly dark threshold, and the sum of the number of the first overly dark pixels after the final update and the number of the second overly dark pixels after the final update is greater than or equal to the second image overly dark threshold, it is determined that the target overly dark state is the fifth state; and the overly dark degree corresponding to the fifth state is less than the overly dark degree corresponding to the fourth state;

[0031] When it is determined that the sum of the number of the first over-dark pixels after the final update and the number of the second over-dark pixels after the final update is less than the over-dark threshold of the second image, the target over-dark state is determined to be the sixth state; the over-darkness corresponding to the sixth state is less than the over-darkness corresponding to the fifth state.

[0032] According to an automatic exposure adjustment method provided by the present invention, the target exposure adjustment strategy is determined based on the target overexposure state and the target overdark state, including:

[0033] Based on the correspondence between the overexposure state, the too dark state, the exposure adjustment direction and the image adjustment quantity, the target exposure adjustment strategy corresponding to the target overexposure state and the target too dark state is determined; the target exposure adjustment strategy includes a target exposure adjustment direction and a target image adjustment quantity.

[0034] According to an automatic exposure adjustment method provided by the present invention, the exposure adjustment of a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy includes:

[0035] In a process of performing exposure adjustment on RAW images subsequent to the current RAW image and having the same adjustment number as the target image based on the target exposure adjustment direction, when it is determined that an image state of a first RAW image after exposure adjustment is different from an image state of a previous RAW image of the first RAW image after exposure adjustment, exposure adjustment is performed on RAW images subsequent to the first RAW image and having the same adjustment number as the latest image based on the latest image adjustment number included in the exposure adjustment strategy corresponding to the image state of the first RAW image after exposure adjustment.

[0036] According to an automatic exposure adjustment method provided by the present invention, the method further includes:

[0037] When it is determined that the image states of two sequentially adjacent second RAW images after exposure adjustment alternately change between the first image state and the second image state, and the number of repeated changes is equal to a preset number of changes, stopping exposure adjustment for the next RAW image of the last second RAW image, and determining a third image state based on the lock gear strategy; the number of repeated changes is used to represent the number of times the first image state changes to the second image state, and then changes to the first image state from the second image state;

[0038] The image sensor is controlled to perform image acquisition based on the exposure parameters corresponding to the third image state.

[0039] According to an automatic exposure adjustment method provided by the present invention, the determining of the third image state based on the locking gear strategy includes:

[0040] When the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are different, determining the image state corresponding to the overexposure state with a lower degree of overexposure as the third image state;

[0041] When the overexposure state corresponding to the first image state is the same as the overexposure state corresponding to the second image state, and the overdark state corresponding to the first image state is different from the overdark state corresponding to the second image state, the image state corresponding to the overdark state with a lower degree of overdarkness is determined as the third image state.

[0042] According to an automatic exposure adjustment method provided by the present invention, the method further includes:

[0043] When the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy, the image sensor is controlled to be in a locking mode; the locking mode is used to indicate that no operation is performed on the register of the image sensor.

[0044] According to an automatic exposure adjustment method provided by the present invention, the method further includes:

[0045] When it is determined that the image state corresponding to the third RAW image after exposure adjustment is changed from the image state corresponding to the previous RAW image of the third RAW image after exposure adjustment to meet a preset change condition, a new RAW image output by the image sensor is acquired, the new RAW image is set as the current RAW image, and the process of determining the target image state of the current RAW image is returned to determine the target exposure adjustment strategy corresponding to the new RAW image;

[0046] When it is determined that the target exposure adjustment strategy corresponding to the new RAW image is any adjustment strategy except the prohibited adjustment strategy, the image sensor is controlled to be in an unlocked mode, and exposure adjustment is performed on a RAW image after the new RAW image based on the target exposure adjustment strategy corresponding to the new RAW image; the unlocked mode is used to indicate an operation on a register of the image sensor.

[0047] The present invention also provides an automatic exposure adjustment device, comprising:

[0048] An acquisition unit, used for acquiring a current RAW image output by the image sensor;

[0049] A first determining unit is used to determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0050] A second determining unit, configured to determine a target exposure adjustment strategy based on the target overexposure state and the target underdark state;

[0051] The first adjustment unit is configured to perform exposure adjustment on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0052] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the automatic exposure adjustment methods described above is implemented.

[0053] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the automatic exposure adjustment method as described above is implemented.

[0054] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the automatic exposure adjustment method as described above is implemented.

[0055] The automatic exposure adjustment method provided by the present invention determines the target overexposure state and the target too dark state of the current RAW image when the current RAW image output by the image sensor is acquired, and determines the target exposure adjustment strategy based on the target overexposure state and the target too dark state, and performs exposure adjustment on the RAW image after the current RAW image based on the target exposure adjustment strategy. It can be seen that when determining the exposure adjustment strategy, the present invention determines it directly based on the RAW image output by the image sensor, and determines the exposure adjustment strategy through the overexposure state and the too dark state of the RAW image, without converting the RAW image into an RGB image, and without determining the image brightness, the amount of calculation is small, thereby reducing the power consumption of the monitoring device. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1This is one of the flow charts of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0058] Figure 2 This is a second flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0059] Figure 3 This is a third flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0060] Figure 4 This is a fourth flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0061] Figure 5 This is a fifth flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0062] Figure 6 This is a sixth flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0063] Figure 7 FIG7 is a flowchart of the automatic exposure adjustment method provided by an embodiment of the present invention;

[0064] Figure 8 is a schematic structural diagram of an automatic exposure adjustment device provided by an embodiment of the present invention;

[0065] Fig. 9 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] In order to maintain the low power consumption requirement of monitoring equipment, the precision and accuracy of the adjustment can be appropriately relaxed during the AE adjustment process, as long as the brightness of the picture can be kept within a reasonable range after the AE adjustment is stable. For example, for monitoring equipment such as smart locks or cat eyes, an image sensor that runs 24 / 7 is required for liveness detection. For the AE algorithm used by the sensor, as long as the brightness of the picture after stabilization can meet the requirements of liveness detection, it is sufficient.

[0068] Based on this, the present invention proposes an automatic exposure adjustment method, which determines the target overexposure state and target overdark state of the current RAW image when the current RAW image output by the image sensor is acquired, and determines the target exposure adjustment strategy based on the target overexposure state and target overdark state, and performs exposure adjustment on the RAW image after the current RAW image based on the target exposure adjustment strategy. That is, when determining the exposure adjustment strategy, the present invention determines it directly based on the RAW image output by the image sensor, and determines the exposure adjustment strategy through the overexposure state and overdark state of the RAW image, without converting the RAW image into an RGB image, and without determining the image brightness, and the amount of calculation is small, thereby reducing the power consumption of the monitoring device.

[0069] Combine the following Figure 1-Figure 7 The automatic exposure adjustment method of the present invention is described. The execution subject of the embodiment of the present invention can be an electronic device such as a computer, server or monitoring equipment with a camera function, or it can be an automatic exposure adjustment device set in the electronic device. The automatic exposure adjustment device can be implemented by software, hardware or a combination of both.

[0070] Figure 1 FIG. 1 is a flow chart of an automatic exposure adjustment method provided by an embodiment of the present invention. Figure 1 As shown, the automatic exposure adjustment method includes the following steps:

[0071] Step 101: Acquire a current RAW image output by an image sensor.

[0072] Among them, the image sensor is a photosensitive chip, which is a common sensor device. It can sense the specified measured quantity and convert it into a usable output signal according to a certain rule, and is used to output image data. The original data output by the image sensor is called RAW data, which is generally in Bayer format. Each pixel of the image sensor sensor is only sensitive to one color. To obtain an image with color in RGB format, a demosaic algorithm is required to obtain data of the other two colors before it can be converted into an image in RGB (red, green, and blue) format. The current RAW image can be the first frame of the image output by the image sensor after the monitoring device is powered on, or it can be any frame of the image output by the image sensor after the monitoring device is operating normally.

[0073] For example, the image sensor changes the value of the register through the shutter value and sensor gain value sent by the processor of the monitoring device, and then performs image acquisition to ensure that the output flow of the image sensor is normal. The image sensor sends the acquired current RAW image to the processor of the monitoring device, so that the processor of the monitoring device obtains the current RAW image output by the image sensor.

[0074] Step 102: Determine a target image state of the current RAW image; the target image state includes a target overexposed state and a target dark state.

[0075] The target overexposure state is used to characterize the overexposure degree of the current RAW image, and the target dark state is used to characterize the dark degree of the current RAW image.

[0076] For example, when the current RAW image output by the image sensor is acquired, the pixels in the current RAW image are sampled to obtain the sampled pixels, the overexposure state and the overdark state of each sampled pixel are determined, and based on the overexposure state and the overdark state of each sampled pixel, the target overexposure state and the target overdark state of the entire frame of the current RAW image are determined. The overexposure state is divided into overexposure, moderate overexposure, and non-overexposure, and the overdark state is divided into overdark, moderate overdark, and non-overdark.

[0077] Step 103: Determine a target exposure adjustment strategy based on the target overexposure state and the target dark state.

[0078] Optionally, based on the correspondence between the overexposure state, the too dark state, the exposure adjustment direction and the image adjustment quantity, the target exposure adjustment strategy corresponding to the target overexposure state and the target too dark state is determined; the target exposure adjustment strategy includes a target exposure adjustment direction and a target image adjustment quantity.

[0079] For example, Table 1 is a schematic diagram of the strategy mapping table. As shown in Table 1, exposure adjustment strategies corresponding to overexposure states and overdark states are stored. The exposure adjustment strategies include exposure adjustment directions and image adjustment quantities. Assuming that the target overexposure state of the current RAW image is determined to be moderate overexposure, and the target overdark state of the current RAW image is not too dark, it can be found from Table 1 that the exposure adjustment strategy corresponding to moderate overexposure and not too dark is exposure adjustment strategy C, that is, the target exposure adjustment strategy is exposure adjustment strategy C, the exposure adjustment direction of exposure adjustment strategy C is calendering, and the image adjustment quantity is N3.

[0080] Table 1

[0081]

[0082] It should be noted that, in Table 1, ↓ indicates that the exposure adjustment direction is calendering, ↑ indicates that the exposure adjustment direction is brightening, — indicates that the adjustment strategy is prohibited, G11, G12, G13, etc. indicate the positions of the corresponding overexposure states and overdark states in the strategy mapping table, which can be called the gears of the current RAW image, A, B, C, D and E indicate different exposure adjustment strategies, among which, the exposure adjustment strategy A is specifically: calendering each image in N1 consecutive images once, if it is still impossible to enter other gears after the exposure adjustment of the N1 consecutive images is completed, no further adjustment is performed; the exposure adjustment strategy B is specifically: Each of the N2 consecutive images is brightened once. If the exposure adjustment of the N2 consecutive images is still unable to enter other gears, no further adjustment is performed. The specific exposure adjustment strategy C is: each of the N3 consecutive images is calendered once. If the exposure adjustment of the N3 consecutive images is still unable to enter other gears, no further adjustment is performed. The specific exposure adjustment strategy D is: each of the N4 consecutive images is brightened once. If the exposure adjustment of the N4 consecutive images is still unable to enter other gears, no further adjustment is performed. The exposure adjustment strategy E is a prohibited adjustment strategy, that is, no exposure adjustment is performed. N1, N2, N3 and N4 are the image adjustment quantities of the corresponding adjustment strategies.

[0083] It should be noted that the above N1, N2, N3 and N4 are all configurable parameters, which can be adjusted according to the usage scenario, etc., and need to satisfy N1>N3, N2>N4. For example, it can be set to: N1=N2=8, N3=N4=4.

[0084] Step 104: Perform exposure adjustment on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0085] For example, when determining the target exposure adjustment strategy based on the current RAW image, the shutter value and sensor gain value corresponding to each calendering or brightening are determined, the value of the register of the image sensor is changed by the determined shutter value and sensor gain value, and the value of the corresponding secondary register is changed based on the target exposure adjustment strategy to achieve exposure adjustment of the RAW image after the current RAW image. For example, if the target exposure adjustment strategy is exposure adjustment strategy C, the shutter value and sensor gain value corresponding to each calendering are determined, and the N3 RAW images after the current RAW image are calendered once based on the shutter value and sensor gain value corresponding to the calendering to achieve exposure adjustment of the RAW images after the current RAW image; if the exposure adjustment of the N3 RAW images after the current RAW image is completed and it is still impossible to enter other gears, the exposure adjustment of the subsequent RAW images will no longer be performed.

[0086] The automatic exposure adjustment method provided by the present invention determines the target overexposure state and the target too dark state of the current RAW image when the current RAW image output by the image sensor is acquired, and determines the target exposure adjustment strategy based on the target overexposure state and the target too dark state, and performs exposure adjustment on the RAW image after the current RAW image based on the target exposure adjustment strategy. It can be seen that when determining the exposure adjustment strategy, the present invention determines it directly based on the RAW image output by the image sensor, and determines the exposure adjustment strategy through the overexposure state and the too dark state of the RAW image, without converting the RAW image into an RGB image, and without determining the image brightness, the amount of calculation is small, thereby reducing the power consumption of the monitoring device.

[0087] In one embodiment, Figure 2 FIG. 2 is a flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 2 As shown, the above step 102 can be specifically implemented by the following steps:

[0088] Step 1021: Sample pixel points in the current RAW image to obtain at least one sampled pixel point.

[0089] For example, when the current RAW image is acquired, the pixels in the current RAW image are sampled, and the specific sampling methods include the following methods: the first method is to divide the current RAW image into M*M regions, where M is an odd number, traverse each region, and select the pixel point at the center of each region as the sampling pixel point; the second method is to use the pixel point of the G component in the current RAW image as the sampling pixel point; the third method is to use the pixel point after every preset number of pixels in the current RAW image as the sampling pixel point; the fourth method is to use all the pixel points in the current RAW image as the sampling pixel point. In addition, other sampling methods can also be used to determine the sampling pixel points, and the present invention is not limited to this.

[0090] Step 1022: for each of the sampling pixels, when it is determined that the pixel value of the sampling pixel is greater than or equal to the initial brightness threshold, update the number of overexposed pixels.

[0091] The initial brightness threshold may be represented by THR, and the initial brightness threshold is used to determine whether a pixel is dark or bright. For example, THR may be set to 1 / 2 of the maximum pixel value.

[0092] For example, traverse each sampling pixel point and convert the pixel value Pix (i,j) Compare with the initial brightness threshold THR to determine Pix (i,j) If it is greater than or equal to THR, it means that the sampled pixel is too bright. At this time, the number of overexposed pixels is updated, that is, the number of overexposed pixels is increased by one; Pix(i,j) Represents the pixel value of the pixel in the i-th row and j-th column in the current RAW image, where i and j are both integers.

[0093] Step 1023: When it is determined that the pixel value of the sampled pixel point is less than the initial brightness threshold, the number of dark pixels is updated.

[0094] For example, in determining Pix (i,j) When it is less than THR, it means that the sampled pixel is too dark. At this time, the number of dark pixels is updated, that is, the number of dark pixels is increased by one.

[0095] Step 1024: determine the target overexposure state based on the final updated number of overexposed pixels, and determine the target overdark state based on the final updated number of overdark pixels.

[0096] Among them, the number of overexposed pixels after the final update refers to the corresponding number of overexposed pixels after the evaluation of all sampling pixels is completed; the number of dark pixels after the final update refers to the corresponding number of dark pixels after the evaluation of all sampling pixels is completed.

[0097] For example, after traversing all the sampled pixel points, the overexposure state of the current RAW image is determined based on the number of overexposed pixel points after the final update, that is, the target overexposure state is determined, and the overdark state of the current RAW image is determined based on the number of overdark pixel points after the final update, that is, the target overdark state is determined.

[0098] In this embodiment, the sampling pixels in the overexposed state and the sampling pixels in the too dark state in the current RAW image are counted based on the initial brightness and darkness thresholds, the target overexposure state of the current RAW image is determined based on the final number of overexposed pixels obtained by counting, and finally the target too dark state of the current RAW image is determined based on the number of too dark pixels obtained by counting, thereby realizing the determination of the overexposed state and the too dark state of the current RAW image. In addition, the present invention reduces the amount of calculation through the sampling method, thereby further reducing the power consumption.

[0099] In one embodiment, the number of overexposed pixels includes a first number of overexposed pixels at a first overexposure level and a second number of overexposed pixels at a second overexposure level; the first overexposure level is greater than the second overexposure level; Figure 3 FIG. 3 is a flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 3 As shown, in the above step 1022, the number of overexposed pixels is updated, which can be specifically implemented by the following steps:

[0100] Step 10221: When the pixel value of the sampling pixel is greater than or equal to the first pixel overexposure threshold, update the number of the first overexposed pixel points; the first pixel overexposure threshold is greater than the initial brightness threshold.

[0101] Among them, the first overexposure level can be overexposure, the second overexposure level can be moderate overexposure, the first overexposed pixel number can be represented by NUM1, the first pixel overexposure threshold can be represented by THR1, the second overexposed pixel number can be represented by NUM2, the second pixel overexposure threshold can be represented by THR2, and both THR1 and THR2 are greater than THR.

[0102] For example, each sampling pixel point is traversed, and the pixel value Pix of the sampling pixel point is determined. (i,j) When it is greater than or equal to the initial brightness threshold THR, Pix (i,j) Compared with THR1, in determining Pix (i,j) When it is greater than or equal to THR1, the sampled pixel is considered to be an overexposed pixel, and the first overexposed pixel number NUM1 is updated, that is, NUM1 is increased by one.

[0103] Step 10222: When the pixel value of the sampling pixel point is less than the first pixel point overexposure threshold and greater than or equal to the second pixel point overexposure threshold, update the second overexposed pixel point quantity.

[0104] For example, in determining Pix (i,j) When it is less than THR1, Pix (i,j) Compared with THR2, in determining Pix (i,j) When it is greater than or equal to THR2, the sampled pixel is considered to be a moderately overexposed pixel, and the second overexposed pixel number NUM2 is updated, that is, NUM2 is increased by one.

[0105] like Figure 3 As shown, in the above step 1024, the target overexposure state is determined based on the number of overexposed pixels after the final update, which can be specifically achieved by the following steps:

[0106] Step 10241: When it is determined that the number of first overexposed pixel points after the final update is greater than or equal to the first image overexposure threshold, determine that the target overexposure state is the first state.

[0107] The first image overexposure threshold may be represented by THR5.

[0108] For example, after traversing all sampling pixels, the finally updated NUM1 is compared with THR5. When it is determined that the finally updated NUM1 is greater than or equal to THR5, it is determined that the target overexposure state of the current RAW image is the first state, and the first state is overexposure.

[0109] Step 10242: When it is determined that the number of the first overexposed pixel points after the final update is less than the first image overexposure threshold, and the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is greater than or equal to the second image overexposure threshold, determine that the target overexposure state is the second state; and the overexposure degree corresponding to the second state is less than the overexposure degree corresponding to the first state.

[0110] The second image overexposure threshold may be represented by THR6.

[0111] For example, when it is determined that the final updated NUM1 is less than THR5, the sum of the final updated NUM1 and the final updated NUM2 is compared with THR6. When it is determined that the sum of the final updated NUM1 and the final updated NUM2 is greater than or equal to THR6, the target overexposure state is determined to be the second state, and the second state is moderate overexposure.

[0112] Step 10243: When it is determined that the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is less than the overexposure threshold of the second image, determine that the target overexposure state is a third state; and the overexposure degree corresponding to the third state is less than the overexposure degree corresponding to the second state.

[0113] For example, when it is determined that the sum of the finally updated NUM1 and the finally updated NUM2 is less than THR6, it is determined that the target overexposure state is the third state, and the third state is non-overexposure.

[0114] In this embodiment, based on the initial brightness threshold, the first pixel overexposure threshold and the second pixel overexposure threshold, the number of first overexposed pixels in the current RAW image whose overexposure state is overexposed and the number of second overexposed pixels whose overexposure state is moderately overexposed are counted, and then the target overexposure state of the current RAW image is determined based on the number of first overexposed pixels and the number of second overexposed pixels, which improves the accuracy of determining the overexposure state with small calculation amount.

[0115] In one embodiment, the number of overly dark pixels includes a first number of overly dark pixels of a first overly dark level and a second number of overly dark pixels of a second overly dark level; the first overly dark level is greater than the second overly dark level; Figure 3 As shown, updating the number of dark pixels in the above step 1023 can be specifically implemented by the following steps:

[0116] Step 10231: When the pixel value of the sampled pixel point is less than or equal to the first pixel point too dark threshold, update the first pixel point too dark threshold number; the first pixel point too dark threshold is less than the initial brightness threshold.

[0117] Among them, the first too dark level can be too dark, the second too dark level can be moderately too dark, the first too dark pixel number can be represented by NUM3, the first pixel too dark threshold can be represented by THR3, the second too dark pixel number can be represented by NUM4, the second pixel too dark threshold can be represented by THR4, and both THR3 and THR4 are less than THR.

[0118] For example, each sampling pixel point is traversed, and the pixel value Pix of the sampling pixel point is determined. (i,j) When it is less than the initial brightness threshold THR, Pix (i,j) Compared with THR3, in determining Pix (i,j) When it is less than or equal to THR3, the sampled pixel is considered to be an overly dark pixel, and the first overly dark pixel number NUM3 is updated, that is, NUM3 is increased by one.

[0119] Step 10232: When the pixel value of the sampled pixel is greater than the first pixel too dark threshold and less than or equal to the second pixel too dark threshold, update the number of the second too dark pixels.

[0120] For example, in determining Pix (i,j) When it is greater than THR3, Pix (i,j) Compared with THR4, in determining Pix (i,j) When it is less than or equal to THR4, the sampled pixel is considered to be a moderately dark pixel, and the second number of dark pixels NUM4 is updated, that is, NUM4 plus one.

[0121] like Figure 3 As shown, in the above step 1024, determining the target over-dark state based on the number of over-dark pixels after the final update can be specifically achieved by the following steps:

[0122] Step 10244: When it is determined that the number of the first overly dark pixels after the final update is greater than or equal to the first image overly dark threshold, determine that the target overly dark state is the fourth state.

[0123] The first image too dark threshold may be represented by THR7.

[0124] For example, after traversing all sampling pixels, the finally updated NUM3 is compared with THR7. When it is determined that the finally updated NUM3 is greater than or equal to THR7, the target too dark state of the current RAW image is determined to be the fourth state, and the fourth state is too dark.

[0125] Step 10245, when it is determined that the number of the first over-dark pixels after the final update is less than the first image over-dark threshold, and the sum of the number of the first over-dark pixels after the final update and the number of the second over-dark pixels after the final update is greater than or equal to the second image over-dark threshold, determine that the target over-dark state is the fifth state; the degree of over-darkness corresponding to the fifth state is less than the degree of over-darkness corresponding to the fourth state.

[0126] The second image too dark threshold may be represented by THR8.

[0127] For example, when it is determined that the final updated NUM3 is less than THR7, the sum of the final updated NUM3 and the final updated NUM4 is compared with THR8. When it is determined that the sum of the final updated NUM3 and the final updated NUM4 is greater than or equal to THR8, the target too dark state is determined to be the fifth state, and the fifth state is moderately too dark.

[0128] Step 10246, when it is determined that the sum of the number of the first over-dark pixels after the final update and the number of the second over-dark pixels after the final update is less than the over-dark threshold of the second image, determine that the target over-dark state is the sixth state; the over-darkness corresponding to the sixth state is less than the over-darkness corresponding to the fifth state.

[0129] For example, when it is determined that the sum of the finally updated NUM3 and the finally updated NUM4 is less than THR8, it is determined that the target over-dark state is the sixth state, and the sixth state is not over-dark.

[0130] It should be noted that the above-mentioned initial brightness and darkness thresholds, the first pixel overexposure threshold, the second pixel overexposure threshold, the first image overexposure threshold, the second image overexposure threshold, the first pixel overdark threshold, the second pixel overdark threshold, the first image overdark threshold and the second image overdark threshold can all be adaptively adjusted according to the specific sampling method of the pixels, the resolution of the monitoring equipment, the operating parameters and the use environment, and the present invention is not limited to this.

[0131] In this embodiment, based on the initial brightness threshold, the first pixel over-dark threshold and the second pixel over-dark threshold, the number of first over-dark pixels in the current RAW image whose over-dark state is too dark and the number of second over-dark pixels whose over-dark state is moderately too dark are counted, and then based on the number of the first over-dark pixels and the second over-dark pixels, the target over-dark state of the current RAW image is determined, thereby improving the accuracy of determining the over-dark state with a small amount of calculation.

[0132] In one embodiment, Figure 4 FIG. 4 is a flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 4 As shown, the above step 104 can be specifically implemented by the following steps:

[0133] Step 1041. In the process of performing exposure adjustment on the RAW images that follow the current RAW image and have the same adjustment number as the target image based on the target exposure adjustment direction, when it is determined that the image state of the first RAW image after the exposure adjustment is different from the image state of the previous RAW image after the first RAW image after the exposure adjustment, exposure adjustment is performed on the RAW images that follow the first RAW image and have the same adjustment number as the latest image based on the latest image adjustment number included in the exposure adjustment strategy corresponding to the image state of the first RAW image after the exposure adjustment.

[0134] The first RAW image is any image in the process of performing exposure adjustment based on the target exposure adjustment strategy.

[0135] For example, in a process of performing exposure adjustment on RAW images that follow a current RAW image and have the same adjustment number as the target image based on a target exposure adjustment direction, after each exposure adjustment is performed on a first RAW image, it is necessary to determine an image state of the first RAW image after the exposure adjustment, and compare the image state of the first RAW image after the exposure adjustment with the image state of a previous RAW image after the exposure adjustment. When it is determined that the image state of the first RAW image after the exposure adjustment is different from the image state of the previous RAW image after the exposure adjustment, it is necessary to recount the image adjustment number, that is, the exposure adjustment strategy corresponding to the image state of the first RAW image after the exposure adjustment includes the latest The number of image adjustments is to perform exposure adjustment on the RAW images after the first RAW image that have the same number of adjustments as the latest image; for example, assuming that the target exposure adjustment strategy includes a target image adjustment number of 5 times, the exposure adjustment direction is calendering, and the first RAW image P1 is the third image after the current RAW image. After performing exposure adjustment of calendering on P1, if the image state corresponding to the exposure-adjusted P1 is different from the image state corresponding to the previous RAW image P0 of the exposure-adjusted P1, assuming that the latest image adjustment number included in the exposure adjustment strategy corresponding to the exposure adjustment P1 after the exposure adjustment is 3, then exposure adjustment is performed on the three RAW images after P1 based on the exposure adjustment strategy corresponding to the exposure adjustment P1 after the exposure adjustment, so as to re-count the number of image adjustments.

[0136] In this embodiment, in the process of performing exposure adjustment on the RAW images that follow the current RAW image and have the same adjustment amount as the target image based on the target exposure adjustment direction, when it is determined that the image state of the first RAW image after the exposure adjustment is different from the image state of the previous RAW image after the exposure adjustment, exposure adjustment is performed on the RAW images that follow the first RAW image based on the latest image adjustment amount included in the exposure adjustment strategy corresponding to the image state of the first RAW image after the exposure adjustment, thereby realizing real-time updating of the exposure adjustment strategy, so that the RAW image can be adjusted to the appropriate brightness more quickly, thereby improving the accuracy of the exposure adjustment.

[0137] In one embodiment, Figure 5 FIG. 5 is a flowchart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 5 As shown, after executing the above step 104, the automatic exposure adjustment method further includes the following steps:

[0138] Step 105: When it is determined that the image states of two adjacent second RAW images after exposure adjustment alternately change between the first image state and the second image state, and the number of repeated changes is equal to the preset number of changes, stop exposure adjustment for the next RAW image of the last second RAW image, and determine the third image state based on the locking gear strategy.

[0139] Among them, the number of repeated changes is used to characterize the number of times the first image state changes to the second image state, and then changes to the first image state from the second image state; the preset number of changes can be set based on demand, for example, the preset number of changes can be 3, and the second RAW image is any image in the exposure adjustment process.

[0140] For example, during the exposure adjustment process, after each exposure adjustment, it is necessary to determine the image state of the current second RAW image after the exposure adjustment. When it is determined that the image states of two consecutively adjacent second RAW images alternate between the first image state and the second image state, the number of repeated changes corresponding to the alternating change is compared with the preset number of changes. When the number of repeated changes is equal to the preset number of changes, the exposure adjustment is stopped, and the third image state is determined based on the gear locking strategy; wherein the gear locking strategy is to select an image state between the first image state and the second image state as the third image state; for example, taking the preset number of changes as 3 as an example, assuming that 7 consecutive frames of second RAW images are P1, P2, P3, P4, P5, P6 and P7, as shown in Table 1, assuming that the gear position of P1 after the exposure adjustment is G22, the gear position of P2 after the exposure adjustment is G23, and the gear position of P3 after the exposure adjustment is G24. The gear position is G22, the gear position of P4 after exposure adjustment is G23, the gear position of P5 after exposure adjustment is G22, the gear position of P6 after exposure adjustment is G23, and the gear position of P7 after exposure adjustment is G22. It can be determined that the image status changes of the 7 consecutive frames of the second RAW images are: P1 to P2 changes from G22 to G23, P2 to P3 changes from G23 to G22, P3 to P4 changes from G22 to G23, P4 to P5 changes from G23 to G22, P5 to P6 changes from G22 to G23, P6 to P7 changes from G23 to G22, and from G22 to G23, and then from G23 to G22 are recorded as one repeated change number. It can be determined that the repeated change number of the gear position of the 7 consecutive frames of the second RAW images is 3. When the preset upper limit of the change number is reached, the gear lock strategy takes effect. At this time, the exposure adjustment is not performed on the RAW images after P7, and the exposure adjustment ends.

[0141] Step 106: Control the image sensor to perform image acquisition based on the exposure parameters corresponding to the third image state.

[0142] For example, after the exposure adjustment is stopped, the image sensor may be controlled to subsequently capture an image based on the gain value and shutter value corresponding to the third image state.

[0143] In this embodiment, if during the exposure adjustment process, consecutive RAW images repeatedly change between two image states, and the number of repeated changes reaches a preset number of changes, in order to avoid instability in the image brightness, the gear lock strategy is implemented, the image state is fixed at the third image state, and the image sensor is controlled to perform image acquisition based on the exposure parameters corresponding to the third image state, thereby reducing power consumption while improving the stability of the image brightness.

[0144] In one embodiment, the third image state is determined based on the locking gear strategy in step 105, which can be implemented in the following manner:

[0145] In a case where the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are different, the image state corresponding to the overexposure state with a lower degree of overexposure is determined as the third image state.

[0146] When the overexposure state corresponding to the first image state is the same as the overexposure state corresponding to the second image state, and the overdark state corresponding to the first image state is different from the overdark state corresponding to the second image state, the image state corresponding to the overdark state with a lower degree of overdarkness is determined as the third image state.

[0147] By way of example, the gear locking strategy is as follows: comparing the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state, and when it is determined that the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are different, regardless of whether the over-dark state corresponding to the first image state and the over-dark state corresponding to the second image state are the same, determining the image state with a lower degree of overexposure between the first image state and the second image state as the third image state; as shown in Table 1 above, assuming that the overexposure state corresponding to the first image state is moderate overexposure, and the over-dark state is moderate overdark, and the overexposure state corresponding to the second image state is overexposed, and the over-dark state is moderate overdark, then the first image state including moderate overexposure and moderate overdark is determined as the third image state. There are several reasons why the third image state is selected based on the overexposure state when the overexposure state changes: the first reason is that people are more acceptable to darker pictures in their perception of picture brightness and darkness. Therefore, when it is not possible to ensure that both the overdark state and the overexposure state are normal at the same time, priority is given to ensuring that the picture is not overexposed. Therefore, the image state corresponding to moderate overexposure is selected from overexposure and moderate overexposure as the third image state; the second reason is that for darker areas, the post-stage image signal processing (ISP) algorithm can be used to brighten the area, but for the overexposed area, the picture details have disappeared, and the post-stage ISP algorithm cannot obtain the picture details after processing; the third reason is that if the monitoring device stays in a darker picture state, the effective sensor gain value is small and the power consumption is relatively low.

[0148] When it is determined that the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are the same, the overdark state corresponding to the first image state and the overdark state corresponding to the second image state are compared; when it is determined that the overdark state corresponding to the first image state and the overdark state corresponding to the second image state are different, the image state with a lower degree of overdarkness in the first image state and the second image state is determined as the third image state; as shown in Table 1 above, assuming that the overexposure state corresponding to the first image state is moderate overexposure and the overdark state is too dark, and the overexposure state corresponding to the second image state is moderate overexposure and the overdark state is moderate too dark, the second image state including moderate overexposure and moderate too dark is determined as the third image state.

[0149] In this embodiment, when the overexposure state corresponding to the first image state is different from the overexposure state corresponding to the second image state, the image state with a lower degree of overexposure is determined as the third image state; and when the overexposure state corresponding to the first image state is the same as the overexposure state corresponding to the second image state, and the too dark state corresponding to the first image state is different from the too dark state corresponding to the second image state, the image state with a lower degree of overexposure is determined as the third image state. The third image state determined in this way can ensure that the image is neither overexposed nor too dark, thereby reducing power consumption while ensuring the brightness of the image.

[0150] In one embodiment, Figure 6 FIG. 6 is a flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 6 As shown, after executing the above step 104, the automatic exposure adjustment method further includes the following steps:

[0151] Step 107: When the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy, control the image sensor to be in a locking mode; the locking mode is used to indicate that the register of the image sensor is not operated.

[0152] As shown in Table 1 above, the exposure adjustment strategies include prohibited adjustment strategies and individual adjustment strategies. For example, A, B, C and D in Table 1 are all adjustment strategies, and E is a prohibited adjustment strategy.

[0153] For example, when the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy, in order to further reduce power consumption, the image sensor is controlled to be in a locked mode. In the locked mode, the monitoring device operates normally, the image sensor can output the flow normally, but no register operation is performed, and the power consumption is lower.

[0154] In this embodiment, when the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy, the image sensor is controlled to be in a locking mode to further reduce power consumption.

[0155] In one embodiment, Figure 7 FIG. 7 is a flow chart of the automatic exposure adjustment method provided by an embodiment of the present invention. Figure 7 As shown, after executing the above step 104, the automatic exposure adjustment method further includes the following steps:

[0156] Step 108: When it is determined that the image state corresponding to the third RAW image after exposure adjustment and the image state corresponding to the previous RAW image of the third RAW image after exposure adjustment meet a preset change condition, a new RAW image output by the image sensor is obtained, the new RAW image is set as the current RAW image, and the process of determining the target image state of the current RAW image is returned to determine the target exposure adjustment strategy corresponding to the new RAW image.

[0157] Among them, the preset change condition is a condition pre-set based on the demand, and the number of preset change conditions can be at least one, as long as the preset change condition can represent a large change in ambient brightness. For example, as shown in Table 1 above, the preset change condition can be a change from the image state corresponding to the gear position G11 to the image state corresponding to the gear position G13, or a change from the image state corresponding to the gear position G13 to the image state corresponding to the gear position G11, or a change from the image state corresponding to the gear position G11 to the image state corresponding to the gear position G31, or a change from the image state corresponding to the gear position G31 to the image state corresponding to the gear position G11, or a change from the image state corresponding to the gear position G12 to the image state corresponding to the gear position G32, or a change from the image state corresponding to the gear position G32 to the image state corresponding to the gear position G12, and so on.

[0158] For example, assuming that the image state corresponding to the third RAW image after exposure adjustment is the image state corresponding to gear position G11, and the image state corresponding to the previous RAW image of the third RAW image after exposure adjustment is the image state corresponding to gear position G13, then it is determined that the image state corresponding to the third RAW image after exposure adjustment and the image state corresponding to the previous RAW image of the third RAW image after exposure adjustment meet the preset change condition, indicating that the ambient brightness has changed significantly at this time, then a new RAW image output by the image sensor is obtained, and the new RAW image is determined as the current RAW image in the above step 101, and the process returns to step 102 to determine the target overexposure state and the target too dark state of the new RAW image, and then determine the corresponding target exposure adjustment strategy based on the target overexposure state and the target too dark state of the new RAW image.

[0159] It should be noted that the specific method for determining the target overexposure state and the target dark state of the new RAW image is the same as the specific method for determining the target overexposure state and the target dark state of the current RAW image, and the present invention will not be repeated here.

[0160] It should be noted that the specific method of determining the corresponding target exposure adjustment strategy based on the target overexposure state and the target too dark state of the new RAW image is the same as the specific method of determining the corresponding target exposure adjustment strategy based on the target overexposure state and the target too dark state of the current RAW image, and the present invention will not be repeated here.

[0161] Step 109: When it is determined that the target exposure adjustment strategy corresponding to the new RAW image is any adjustment strategy except the prohibited adjustment strategy, control the image sensor to be in a non-locked mode, and perform exposure adjustment on a RAW image after the new RAW image based on the target exposure adjustment strategy corresponding to the new RAW image; the non-locked mode is used to indicate an operation on a register of the image sensor.

[0162] For example, when it is determined that the target exposure adjustment strategy corresponding to the new RAW image is any adjustment strategy except the prohibited adjustment strategy, it means that exposure adjustment is required. At this time, the image sensor is controlled to be in the unlocked mode, that is, the lock mode of the image sensor is released, and the exposure adjustment of the RAW image after the new RAW image is performed based on the target exposure adjustment strategy corresponding to the new RAW image, that is, the shutter value and sensor gain value corresponding to each calendering or brightening are determined, and the value of the register of the image sensor is changed by the determined shutter value and sensor gain value, and the value of the corresponding secondary register is changed based on the target exposure adjustment strategy corresponding to the new RAW image, so as to realize the exposure adjustment of the RAW image after the new RAW image.

[0163] It should be noted that, when the exposure adjustment is completed or the target exposure adjustment strategy corresponding to the new RAW image is a prohibited adjustment strategy, the image sensor is controlled to be in the locking mode again, and the cycle is repeated.

[0164] It should be noted that the exposure adjustment in the above embodiment can be the exposure adjustment process when the monitoring device is just powered on, and the exposure adjustment process when the ambient brightness changes significantly during the normal operation of the monitoring device. The present invention is not limited to this.

[0165] In this embodiment, the image states of the two frames of RAW images after exposure adjustment are detected in real time. When the change between the image states of the two frames of RAW images after exposure adjustment meets the preset change condition, it means that the ambient brightness has changed significantly. At this time, the lock mode of the image sensor is released, and the exposure adjustment is continued until the exposure adjustment is completed or the adjustment strategy is prohibited. In this case, the ambient brightness is considered to be stable. At this time, the image sensor is controlled to be in the lock mode again, thereby realizing the adaptive change of the working state of the image sensor and further reducing power consumption.

[0166] The automatic exposure adjustment device provided by the present invention is described below. The automatic exposure adjustment device described below and the automatic exposure adjustment method described above can be referred to each other.

[0167] Figure 8 is a schematic diagram of the structure of an automatic exposure adjustment device provided by an embodiment of the present invention. Figure 8 As shown, the automatic exposure adjustment device 800 includes an acquisition unit 801, a first determination unit 802, a second determination unit 803 and a first adjustment unit 804; wherein:

[0168] An acquisition unit 801 is used to acquire a current RAW image output by an image sensor;

[0169] A first determining unit 802 is used to determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0170] A second determining unit 803 is used to determine a target exposure adjustment strategy based on the target overexposure state and the target dark state;

[0171] The first adjustment unit 804 is configured to perform exposure adjustment on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0172] The automatic exposure adjustment device provided by the present invention determines the target overexposure state and target overdark state of the current RAW image when the current RAW image output by the image sensor is acquired, and determines the target exposure adjustment strategy based on the target overexposure state and target overdark state, and performs exposure adjustment on the RAW image after the current RAW image based on the target exposure adjustment strategy. It can be seen that when determining the exposure adjustment strategy, the present invention determines it directly based on the RAW image output by the image sensor, and determines the exposure adjustment strategy through the overexposure state and overdark state of the RAW image, without converting the RAW image into an RGB image, and without determining the image brightness, the amount of calculation is small, thereby reducing the power consumption of the monitoring device.

[0173] Based on any of the foregoing embodiments, the first determining unit 802 is specifically configured to:

[0174] Sampling pixel points in the current RAW image to obtain at least one sampled pixel point;

[0175] For each of the sampling pixel points, when it is determined that the pixel value of the sampling pixel point is greater than or equal to the initial brightness threshold, updating the number of overexposed pixel points;

[0176] When it is determined that the pixel value of the sampled pixel point is less than the initial brightness threshold, updating the number of dark pixels;

[0177] The target overexposure state is determined based on the number of overexposed pixels after the final update, and the target overdark state is determined based on the number of overdark pixels after the final update.

[0178] Based on any of the above embodiments, the number of overexposed pixels includes a first number of overexposed pixels at a first overexposure level and a second number of overexposed pixels at a second overexposure level; the first overexposure level is greater than the second overexposure level;

[0179] The first determining unit 802 is further specifically configured to:

[0180] When the pixel value of the sampling pixel point is greater than or equal to the first pixel point overexposure threshold, updating the first overexposed pixel point quantity; the first pixel point overexposure threshold is greater than the initial brightness threshold;

[0181] When the pixel value of the sampling pixel point is less than the first pixel point overexposure threshold and greater than or equal to the second pixel point overexposure threshold, updating the second overexposed pixel point number;

[0182] When it is determined that the number of first overexposed pixel points after the final update is greater than or equal to the first image overexposure threshold, determining that the target overexposure state is the first state;

[0183] When it is determined that the number of the first overexposed pixel points after the final update is less than the first image overexposure threshold, and the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is greater than or equal to the second image overexposure threshold, determining that the target overexposure state is the second state; and the overexposure degree corresponding to the second state is less than the overexposure degree corresponding to the first state;

[0184] When it is determined that the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is less than the overexposure threshold of the second image, the target overexposure state is determined to be the third state; and the overexposure degree corresponding to the third state is less than the overexposure degree corresponding to the second state.

[0185] Based on any of the above embodiments, the number of too dark pixels includes a first number of too dark pixels of a first too dark level and a second number of too dark pixels of a second too dark level; the first too dark level is greater than the second too dark level;

[0186] The first determining unit 802 is further specifically configured to:

[0187] When the pixel value of the sampled pixel point is less than or equal to the first pixel point too dark threshold, the first pixel point too dark threshold is updated; and the first pixel point too dark threshold is less than the initial brightness threshold;

[0188] When the pixel value of the sampling pixel point is greater than the first pixel point too dark threshold and less than or equal to the second pixel point too dark threshold, updating the second too dark pixel point number;

[0189] When it is determined that the number of the first overly dark pixels after the final update is greater than or equal to the first image overly dark threshold, determining that the target overly dark state is the fourth state;

[0190] When it is determined that the number of the first overly dark pixels after the final update is less than the first image overly dark threshold, and the sum of the number of the first overly dark pixels after the final update and the number of the second overly dark pixels after the final update is greater than or equal to the second image overly dark threshold, it is determined that the target overly dark state is the fifth state; and the overly dark degree corresponding to the fifth state is less than the overly dark degree corresponding to the fourth state;

[0191] When it is determined that the sum of the number of the first over-dark pixels after the final update and the number of the second over-dark pixels after the final update is less than the over-dark threshold of the second image, the target over-dark state is determined to be the sixth state; the over-darkness corresponding to the sixth state is less than the over-darkness corresponding to the fifth state.

[0192] Based on any of the foregoing embodiments, the second determining unit 803 is further specifically configured to:

[0193] Based on the correspondence between the overexposure state, the too dark state, the exposure adjustment direction and the image adjustment quantity, the target exposure adjustment strategy corresponding to the target overexposure state and the target too dark state is determined; the target exposure adjustment strategy includes a target exposure adjustment direction and a target image adjustment quantity.

[0194] Based on any of the above embodiments, the first adjusting unit 804 is specifically configured to:

[0195] In a process of performing exposure adjustment on RAW images subsequent to the current RAW image and having the same adjustment number as the target image based on the target exposure adjustment direction, when it is determined that an image state of a first RAW image after exposure adjustment is different from an image state of a previous RAW image of the first RAW image after exposure adjustment, exposure adjustment is performed on RAW images subsequent to the first RAW image and having the same adjustment number as the latest image based on the latest image adjustment number included in the exposure adjustment strategy corresponding to the image state of the first RAW image after exposure adjustment.

[0196] Based on any of the above embodiments, the automatic exposure adjustment device 800 further includes:

[0197] a second adjustment unit, configured to stop exposure adjustment for a next RAW image of the last second RAW image after the exposure adjustment, and determine a third image state based on a lock gear strategy when it is determined that the image states of two consecutively adjacent second RAW images after exposure adjustment alternately change between a first image state and a second image state, and the number of repeated changes is equal to a preset number of changes; the number of repeated changes is used to represent the number of times the first image state changes to the second image state, and then changes to the first image state from the second image state;

[0198] The first control unit is used to control the image sensor to capture images based on the exposure parameters corresponding to the third image state.

[0199] Based on any of the foregoing embodiments, the second adjusting unit is specifically configured to:

[0200] When the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are different, determining the image state corresponding to the overexposure state with a lower degree of overexposure as the third image state;

[0201] When the overexposure state corresponding to the first image state is the same as the overexposure state corresponding to the second image state, and the overdark state corresponding to the first image state is different from the overdark state corresponding to the second image state, the image state corresponding to the overdark state with a lower degree of overdarkness is determined as the third image state.

[0202] Based on any of the above embodiments, the automatic exposure adjustment device 800 further includes:

[0203] The second control unit is used to control the image sensor to be in a locking mode when the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy; the locking mode is used to indicate that the register of the image sensor is not operated.

[0204] Based on any of the above embodiments, the automatic exposure adjustment device 800 further includes:

[0205] a third determining unit, configured to, when it is determined that a change between an image state corresponding to a third RAW image after exposure adjustment and an image state corresponding to a previous RAW image of the third RAW image after exposure adjustment satisfies a preset change condition, obtain a new RAW image output by the image sensor, set the new RAW image as the current RAW image, and return to the step of determining a target image state of the current RAW image to determine a target exposure adjustment strategy corresponding to the new RAW image;

[0206] A third adjustment unit is used to control the image sensor to be in an unlocked mode when it is determined that the target exposure adjustment strategy corresponding to the new RAW image is any adjustment strategy except the prohibited adjustment strategy, and to perform exposure adjustment on a RAW image after the new RAW image based on the target exposure adjustment strategy corresponding to the new RAW image; the unlocked mode is used to indicate an operation on a register of the image sensor.

[0207] Fig. 9 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the automatic exposure adjustment method, which includes: obtaining a current RAW image output by the image sensor;

[0208] Determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0209] Determining a target exposure adjustment strategy based on the target overexposure state and the target underdark state;

[0210] Exposure adjustment is performed on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0211] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0212] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the automatic exposure adjustment method provided by the above methods, the method includes: obtaining a current RAW image output by an image sensor;

[0213] Determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0214] Determining a target exposure adjustment strategy based on the target overexposure state and the target underdark state;

[0215] Exposure adjustment is performed on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0216] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being implemented when executed by a processor to perform the automatic exposure adjustment method provided by the above methods, the method comprising: acquiring a current RAW image output by an image sensor;

[0217] Determine a target image state of the current RAW image; the target image state includes a target overexposure state and a target dark state; the target overexposure state is used to characterize the degree of overexposure of the current RAW image, and the target dark state is used to characterize the degree of darkness of the current RAW image;

[0218] Determining a target exposure adjustment strategy based on the target overexposure state and the target underdark state;

[0219] Exposure adjustment is performed on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

[0220] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0221] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic exposure adjustment method, characterized in that: include: Get the current RAW image output by the image sensor; Determining a target image state of the current RAW image; The target image state includes a target overexposed state and a target too dark state; The target overexposure state is used to characterize the overexposure degree of the current RAW image, and the target dark state is used to characterize the dark degree of the current RAW image; Determining a target exposure adjustment strategy based on the target overexposure state and the target underdark state; Exposure adjustment is performed on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy.

2. The automatic exposure adjustment method according to claim 1, characterized in that: The determining a target image state of the current RAW image includes: Sampling pixel points in the current RAW image to obtain at least one sampled pixel point; For each of the sampling pixel points, when it is determined that the pixel value of the sampling pixel point is greater than or equal to the initial brightness threshold, updating the number of overexposed pixel points; When it is determined that the pixel value of the sampled pixel point is less than the initial brightness threshold, updating the number of dark pixels; The target overexposure state is determined based on the number of overexposed pixels after the final update, and the target overdark state is determined based on the number of overdark pixels after the final update.

3. The automatic exposure adjustment method according to claim 2, characterized in that: The number of overexposed pixels includes a first number of overexposed pixels at a first overexposure level and a second number of overexposed pixels at a second overexposure level; The first overexposure level is greater than the second overexposure level; The updating of the number of overexposed pixels includes: When the pixel value of the sampling pixel point is greater than or equal to the first pixel point overexposure threshold, updating the first overexposed pixel point quantity; the first pixel point overexposure threshold is greater than the initial brightness threshold; When the pixel value of the sampling pixel point is less than the first pixel point overexposure threshold and greater than or equal to the second pixel point overexposure threshold, updating the second overexposed pixel point number; The determining the target overexposure state based on the number of overexposed pixels after the final update includes: When it is determined that the number of first overexposed pixel points after the final update is greater than or equal to the first image overexposure threshold, determining that the target overexposure state is the first state; When it is determined that the number of the first overexposed pixel points after the final update is less than the first image overexposure threshold, and the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is greater than or equal to the second image overexposure threshold, determining that the target overexposure state is the second state; and the overexposure degree corresponding to the second state is less than the overexposure degree corresponding to the first state; When it is determined that the sum of the number of the first overexposed pixel points after the final update and the number of the second overexposed pixel points after the final update is less than the overexposure threshold of the second image, the target overexposure state is determined to be the third state; and the overexposure degree corresponding to the third state is less than the overexposure degree corresponding to the second state.

4. The automatic exposure adjustment method according to claim 2, characterized in that: The number of too dark pixels includes a first number of too dark pixels of a first too dark level and a second number of too dark pixels of a second too dark level; The first too dark level is greater than the second too dark level; The updating of the number of dark pixels includes: When the pixel value of the sampled pixel point is less than or equal to the first pixel point too dark threshold, the first pixel point too dark threshold is updated; and the first pixel point too dark threshold is less than the initial brightness threshold; When the pixel value of the sampling pixel point is greater than the first pixel point too dark threshold and less than or equal to the second pixel point too dark threshold, updating the second too dark pixel point number; The determining the target over-dark state based on the number of over-dark pixels after the final update includes: When it is determined that the number of the first overly dark pixels after the final update is greater than or equal to the first image overly dark threshold, determining that the target overly dark state is the fourth state; When it is determined that the number of the first overly dark pixels after the final update is less than the first image overly dark threshold, and the sum of the number of the first overly dark pixels after the final update and the number of the second overly dark pixels after the final update is greater than or equal to the second image overly dark threshold, it is determined that the target overly dark state is the fifth state; and the overly dark degree corresponding to the fifth state is less than the overly dark degree corresponding to the fourth state; When it is determined that the sum of the number of the first over-dark pixels after the final update and the number of the second over-dark pixels after the final update is less than the over-dark threshold of the second image, the target over-dark state is determined to be the sixth state; the over-darkness corresponding to the sixth state is less than the over-darkness corresponding to the fifth state.

5. The automatic exposure adjustment method according to any one of claims 1 to 4, characterized in that: The determining of a target exposure adjustment strategy based on the target overexposure state and the target underdark state includes: Based on the correspondence between the overexposure state, the too dark state, the exposure adjustment direction and the image adjustment quantity, the target exposure adjustment strategy corresponding to the target overexposure state and the target too dark state is determined; the target exposure adjustment strategy includes a target exposure adjustment direction and a target image adjustment quantity.

6. The automatic exposure adjustment method according to claim 5, characterized in that: The step of performing exposure adjustment on a RAW image subsequent to the current RAW image based on the target exposure adjustment strategy includes: In a process of performing exposure adjustment on RAW images subsequent to the current RAW image and having the same adjustment number as the target image based on the target exposure adjustment direction, when it is determined that an image state of a first RAW image after exposure adjustment is different from an image state of a previous RAW image of the first RAW image after exposure adjustment, exposure adjustment is performed on RAW images subsequent to the first RAW image and having the same adjustment number as the latest image based on the latest image adjustment number included in the exposure adjustment strategy corresponding to the image state of the first RAW image after exposure adjustment.

7. The automatic exposure adjustment method according to any one of claims 1 to 4, characterized in that: The method further comprises: When it is determined that the image states of two sequentially adjacent second RAW images after exposure adjustment alternately change between the first image state and the second image state, and the number of repeated changes is equal to a preset number of changes, stopping exposure adjustment for the next RAW image of the last second RAW image, and determining a third image state based on the lock gear strategy; the number of repeated changes is used to represent the number of times the first image state changes to the second image state, and then changes to the first image state from the second image state; The image sensor is controlled to perform image acquisition based on the exposure parameters corresponding to the third image state.

8. The automatic exposure adjustment method according to claim 7, characterized in that: The determining the third image state based on the locking gear strategy includes: When the overexposure state corresponding to the first image state and the overexposure state corresponding to the second image state are different, determining the image state corresponding to the overexposure state with a lower degree of overexposure as the third image state; When the overexposure state corresponding to the first image state is the same as the overexposure state corresponding to the second image state, and the overdark state corresponding to the first image state is different from the overdark state corresponding to the second image state, the image state corresponding to the overdark state with a lower degree of overdarkness is determined as the third image state.

9. The automatic exposure adjustment method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the exposure adjustment is completed or the target exposure adjustment strategy is a prohibited adjustment strategy, the image sensor is controlled to be in a locking mode; the locking mode is used to indicate that no operation is performed on the register of the image sensor.

10. The automatic exposure adjustment method according to claim 9, characterized in that: The method further comprises: When it is determined that the image state corresponding to the third RAW image after exposure adjustment is changed from the image state corresponding to the previous RAW image of the third RAW image after exposure adjustment to meet a preset change condition, a new RAW image output by the image sensor is acquired, the new RAW image is set as the current RAW image, and the process of determining the target image state of the current RAW image is returned to determine the target exposure adjustment strategy corresponding to the new RAW image; When it is determined that the target exposure adjustment strategy corresponding to the new RAW image is any adjustment strategy except the prohibited adjustment strategy, the image sensor is controlled to be in an unlocked mode, and exposure adjustment is performed on a RAW image after the new RAW image based on the target exposure adjustment strategy corresponding to the new RAW image; the unlocked mode is used to indicate an operation on a register of the image sensor.

Images