Face brightness statistical method, exposure control method, equipment and storage medium

Abstract

The invention discloses a human face brightness statistical method, an exposure control method, equipment and a storage medium, and the method comprises the steps: carrying out the human face region brightness statistics in a YUV image outputted by an image processor, feeding back the calculated exposure offset to the image processor, and achieving the purpose of controlling the exposure of a human face region in real time. According to the method, brightness statistics can be carried out on multiple face areas of the output image, the Otsu method and the image morphology opening operation are sequentially adopted during statistics, the face pixel points are separated, interference of the background pixel points is eliminated, the problem that the statistical value does not conform to the actual face brightness during face brightness statistics can be effectively solved, and the face brightness statistics accuracy is improved. And meanwhile, the proposed exposure control strategy can effectively cope with various complex video scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to image processing technology, in particular to a face brightness statistical method and an exposure control method, device and storage medium. BACKGROUND

[0002] Generally, there is a difference in brightness between the original data output by the image sensor and the image expected by human beings, and the image processor needs to adjust the exposure state of each frame of image. For images that need to be detected for faces, the image processor should focus on the brightness of the face area to adjust the exposure.

[0003] At present, in the exposure processing based on face detection images, the related technology is to directly count the brightness of the face area in the image in the image processor, and compare it with the preset target brightness to adjust the image exposure. Generally, there are several frame buffers when the image processor is working, and the image for face detection comes from the frame buffer queue of the image processor. If the brightness of the face area fed back by the face detection module is directly counted in the image processor, the existence of frame delay will cause the feedback face area not to conform to the actual face position. When the face is in a moving state or suddenly disappears in the image, this way of counting brightness will make the exposure control unreasonable, which is easy to cause the problem of image flicker. Moreover, the traditional way of counting face brightness does not consider the separation of face and background, and is easily disturbed by background pixels during counting, for example, the brightness of the face area is easily disturbed by sunlight when counting the brightness of the face area in the outdoor backlight or light measurement environment, which leads to too low exposure of the face. Secondly, the previous exposure adjustment method is also relatively simple and cannot adapt to complex and variable video scenes, for example, there are multiple face areas in the same image or the scene where the face cannot be detected. The above scene changes need to consider the reasonable adjustment of face exposure, otherwise it will cause the brightness of the face area to change smoothly. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a face brightness statistical method and an exposure control method, device and storage medium, so as to feed back the calculated exposure offset to the image processor and achieve the purpose of real-time control of face area exposure.

[0005] To achieve the above purpose, the technical scheme of the present application is:

[0006] In a first aspect, the present application provides a face brightness statistical method and an exposure control method, comprising:

[0007] obtaining a frame of image to be processed, detecting the face, and obtaining the face area in the image;

[0008] determining whether the number of face areas in the image is greater than zero;

[0009] If the number of face regions in the image is zero, and the number of face regions detected in continuous n frames of images is zero, then the exposure adjustment amount fed back to the image processor is set to zero; if the number of face regions in the image is zero, but the cumulative number of image frames in which the number of face regions detected is zero does not reach n frames, then the exposure adjustment amount fed back to the image processor remains unchanged; n is a positive integer;

[0010] If the number of face regions in the image is greater than zero, according to each face region in the image, the Otsu method and the morphological method are used to segment the face pixel points, the Y channel pixel average value of the face pixel points is obtained, and the average brightness of each face region is determined to obtain the face brightness of the image.

[0011] If the brightness difference between the face brightness and the preset target brightness is within the preset tolerance, the exposure adjustment amount fed back to the image processor remains unchanged; if the brightness difference between the face brightness and the preset target brightness exceeds the preset tolerance, the exposure adjustment amount is updated and fed back to the image processor.

[0012] Optionally, the Otsu method and the morphological method are used to segment the face pixel points according to each face region in the image, the Y channel pixel average value of the face pixel points is obtained, and the average brightness of each face region is determined to obtain the face brightness of the image, including:

[0013] 201, obtaining each face region in an image;

[0014] 202, using the Otsu method to obtain the segmentation threshold of the Y channel pixel points of the candidate face and the background in the i-th face region; i is a natural number greater than zero;

[0015] 203, performing binaryzation processing on the face region according to the segmentation threshold to obtain a binaryzation image; performing morphological opening operation on the binaryzation image to obtain the largest connected region in the image after the morphological opening operation, and the pixel points on the connected region are the screened face pixel points, and the Y channel pixel average value of the face pixel points is counted to obtain the face average brightness of the i-th face region;

[0016] 204, if multiple face regions are detected in an image, repeating steps 202-204 to obtain the face pixel points of each face region, counting the Y channel pixel average value of the face pixel points of each face region, and determining the face brightness of the image.

[0017] Optionally, the step 202 includes:

[0018] Let the segmentation threshold of Y channel pixel points belonging to the candidate face and the background in the i-th face region of the current image be T(i), the pixel points with Y channel pixel value lower than T(i) in the face region are marked as candidate face pixel points, the proportion of the total number of the candidate face pixel points in the total number of pixels in the face region is w0(i), and the average value of Y channel pixels of the candidate face pixel points is Y0(i); the pixel points with Y channel pixel value greater than or equal to T(i) in the face region are marked as background pixel points, the proportion of the total number of the background pixel points in the total number of pixels in the face region is w1(i), and the average value of Y channel pixels of the background pixel points is Y1(i), then the average value of Y channel pixels of the face region is represented as:

[0019]

[0020] Optionally, the step 202 further comprises:

[0021] The inter-class variance G(i) of the face region is represented as:

[0022]

[0023] The segmentation threshold T(i) of the face region is obtained by maximizing the inter-class variance G(i), and the candidate face pixel points of the face region are obtained according to the segmentation threshold T(i).

[0024] Optionally, in the step 203, the morphological opening operation on the binary image is represented as:

[0025]

[0026] Wherein, P″(i) represents the face region image after the morphological opening operation, P′(i) represents the binary image of the face region, B represents a structure element, and the symbols represent the morphological opening operation, and Θ and represent the morphological erosion and dilation respectively.

[0027] Optionally, in the step 204, the face brightness of the image is:

[0028]

[0029] Wherein, Y represents the face brightness of the current image, and N represents the number of face regions detected in the current image; represents the average face brightness of the i-th face region.

[0030] Optionally, the brightness difference between the face brightness and the preset target brightness is:

[0031] ΔY=Y′-Y

[0032] Y' is a preset target brightness, Y is a face brightness.

[0033] Optionally, if the brightness difference between the face brightness and the preset target brightness is within a preset tolerance, the exposure adjustment amount fed back to the image processor remains unchanged; if the brightness difference between the face brightness and the preset target brightness exceeds the preset tolerance, the exposure adjustment amount is updated and fed back to the image processor, expressed as:

[0034]

[0035] u t = u t-1 + Δu

[0036] wherein d represents the change rate of the exposure offset adjustment amount of the t-th frame of image, Δu represents the exposure adjustment amount corresponding to the t-th frame of image, u t represents the exposure offset amount of the t-th frame of image, u t-1 represents the exposure offset amount of the (t-1)-th frame of image, and k0 and k1 represent preset exposure adjustment intensities.

[0037] In a second aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.

[0038] The memory is used for storing a computer program.

[0039] The processor is used for executing the program stored on the memory, so as to realize the steps of the face brightness statistical method and the exposure control method according to any one of the above.

[0040] In a third aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the face brightness statistical method and the exposure control method according to any one of the above.

[0041] Compared with the prior art, the present application has the following beneficial effects:

[0042] The method provided by the present application can solve the problem that the face region does not conform to the actual face position when the brightness is directly counted in the image processor, and can effectively avoid the problem of image flicker caused by unreasonable face brightness counting mode when the face is in a moving state or suddenly disappears in the image. When counting the face brightness, considering the premise that there are multiple face regions in the image, the set of Y channel pixel points of each face region is respectively subjected to the Otsu method and the morphological opening operation method in sequence, the pixel points belonging to the face are separated from the pixel points, the average value of the Y channel pixels of the set of face pixel points is calculated to obtain the average brightness of each face region, and finally the face brightness of the whole image is obtained, so that the interference of background factors during counting can be effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 Main flow chart of the face brightness counting method and the exposure control method provided by the embodiments of the present application;

[0044] Figure 2 Flow chart of the face brightness counting method;

[0045] Figure 3 Flow chart of the exposure offset updating method;

[0046] Figure 4 Schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0047] Embodiment:

[0048] The technical solutions of the present application will be further described below in combination with the drawings and embodiments.

[0049] As shown in the drawings, a detailed description is made of a face region brightness counting method and an exposure control method according to one embodiment of the present application. Figure 1

[0050] A frame of image to be processed is acquired, and a face detection algorithm is run to detect the face and acquire the face region in the image.

[0051] The number of face regions in the image is counted and judged. If the number of face regions in the current image is equal to zero, that is, no face is detected, it is needed to confirm whether no face is detected in n frames (usually n≥10) of images in succession. If so, the exposure offset value fed back to the image processor is set to zero, otherwise the exposure offset value is maintained unchanged.

[0052]

[0053] Wherein, u t ​an exposure offset of a t-th frame image, u t-1 an exposure offset of a t-1-th frame image.

[0054] If the number of face regions in the image is greater than zero, the Otsu method and the morphological method are used to segment the face pixel points according to each face region in the image, and the Y channel pixel average value of the face pixel points is obtained to determine the average brightness of each face region, so as to obtain the face brightness of the frame image.

[0055] If the number of face regions in the image is greater than zero and the brightness difference between the face brightness and the preset target brightness is within the preset tolerance, the exposure adjustment amount fed back to the image processor remains unchanged; if the number of face regions in the image is greater than zero and the brightness difference between the face brightness and the preset target brightness exceeds the preset tolerance, the exposure adjustment amount is updated and fed back to the image processor.

[0056] It can be seen that the method can solve the problem that the face region does not conform to the actual face position when the brightness is directly counted in the image processor, and can effectively avoid the problem of image flicker caused by unreasonable face brightness counting method when the face is in a motion state or suddenly disappears in the image. When counting the face brightness, considering the premise that there are multiple face regions in the image, the Otsu method and the morphological opening operation method are used on the Y channel pixel point set of each face region in turn, the pixel points belonging to the face are separated from these pixel points, the Y channel pixel average value of the above face pixel point set is calculated to obtain the average brightness of each face region, and finally the face brightness of the whole image is obtained, which can effectively reduce the interference of background factors during counting.

[0057] The Otsu method is an image segmentation algorithm for automatically obtaining the segmentation threshold of target and background according to the maximum principle of inter-class variance. As shown in Figure 2 the face brightness is counted in the following way:

[0058] 201, obtaining each face region in a frame image;

[0059] 202, using the Otsu method to obtain the segmentation threshold of the Y channel pixel points of the candidate face and the background in the i-th face region; i is a natural number greater than zero;

[0060] Specifically, let the segmentation threshold of Y channel pixel points belonging to the candidate face and the background in the i-th face region of the current image be T(i), the pixel points with Y channel pixel value lower than T(i) in the face region are marked as candidate face pixel points, the proportion of the total number of these candidate face pixel points in the total pixels of the face region is w0(i), and the Y channel pixel average value of these candidate face pixel points is Y0(i); in addition, the pixel points with Y channel pixel value greater than or equal to T(i) in the face region are marked as background pixel points, the proportion of the total number of these background pixel points in the total pixels of the face region is w1(i), and the Y channel pixel average value of these background pixel points is Y1(i), then the Y channel pixel average value of the face region is is expressed as:

[0061]

[0062] Further, the inter-class variance G(i) of the face region is expressed as:

[0063]

[0064] By the principle of maximizing the inter-class variance G(i), the segmentation threshold T(i) of the face region is obtained, and the candidate face pixel points of the face region are obtained according to the segmentation threshold T(i).

[0065] The candidate face pixel points may include isolated background pixel points, which appear as scattered points or burrs in the image, and their existence will also affect the accuracy of face brightness statistics. In order to remove the interference of these background pixel points, the following step 203 is required;

[0066] 203, binaryzation processing is performed on the face region according to the segmentation threshold to obtain a binaryzation image; morphological opening operation is performed on the binaryzation image, which will first perform morphological erosion and then perform morphological dilation, can smooth the background noise, and obtain an image with smoother face contour, which can be expressed as:

[0067]

[0068] Wherein, P″(i) represents the face region image after morphological opening operation, P′(i) represents the binaryzation image of the face region, B represents the structure element, and the symbols represent morphological opening operation, and Θ and represent morphological erosion and morphological dilation respectively.

[0069] The largest connected region in the image after morphological opening operation is obtained, and the pixel points on the connected region are the screened face pixel points, and the Y channel pixel average value of the face pixel points is obtained to obtain the face average brightness of the i-th face region

[0070] 204, if multiple face regions are detected in a frame of image, repeating steps 202-204 to obtain face pixel points of each face region, counting the average value of Y channel of face pixel points of each face region, and determining the face brightness of the image:

[0071]

[0072] wherein Y represents the face brightness of the current image, and N represents the number of face regions detected in the current image.

[0073] Specifically, as shown in the flow chart of the exposure offset updating method according to the method of the present application; Figure 3

[0074] According to the face brightness Y obtained by the above counting and the preset target brightness Y', the brightness difference ΔY is calculated:

[0075] ΔY = Y' - Y

[0076] It is judged whether the brightness difference ΔY is within the preset tolerance R. If the brightness difference between the two is within the tolerance, it is considered that the exposure control of the frame image is within a reasonable range, and the exposure offset u does not need to be updated, i.e. the adjustment amount Δu is zero; if the brightness difference between the two is outside the tolerance, it is considered that the exposure offset of the frame image needs to be updated:

[0077]

[0078] u t = u t-1 + Δu

[0079] wherein d represents the change rate of the exposure offset adjustment amount of the t-th frame of image, Δu represents the exposure offset adjustment amount corresponding to the t-th frame of image, ut represents the exposure offset of the t-th frame of image, ut-1 represents the exposure offset of the t-1-th frame of image, and k0 and k1 represent preset exposure adjustment intensities. In the above updating process, if the value of Δy of the t-th frame of image is large, the value of d representing the change rate of the adjustment amount will also be large, so the adjustment amount Δu of the exposure offset of the t-th frame of image will also increase. In addition, the preset k0 and k1 can control the updating amplitude of the adjustment amount Δu, and reasonable configuration of these two parameters in different environments is conducive to the smooth change of the exposure of the face region, so as to realize reasonable updating of the exposure offset under the premise of considering the difference between the statistical brightness of the face region and the preset target brightness.

[0080] ​In summary, the method can perform face region brightness statistics in the YUV image output by the image processor, can solve the problem that the face region statistics does not conform to the actual face position when the brightness is directly counted in the image processor, and can effectively avoid the problem of image flicker caused by unreasonable face brightness counting mode when the face is in a motion state or suddenly disappears in the image. When counting the face brightness, under the premise that there are multiple face regions in the image, the Otsu method and the morphological opening operation method are used on the Y channel pixel point set of each face region in turn, the pixel points belonging to the face are separated from these pixel points, the Y channel pixel average value of the above face pixel point set is calculated to obtain the average brightness of each face region, and finally the face brightness of the whole image is obtained, so that the interference of background factors during counting can be effectively reduced. In addition, the exposure control strategy provided by the method can cope with various scenes, and exposure offset is updated according to the face detection condition and the face brightness statistics result. For example, when no face is detected in several consecutive frames of the image, the exposure offset is set to zero or unchanged; or when a face is detected, whether the statistical brightness and the target brightness are within the preset tolerance is judged to select exposure offset updating or unchanged. When updating the exposure offset, the difference between the statistical brightness and the target brightness is considered, and a preset parameter for controlling exposure smoothness is added, so that the exposure offset is reasonably updated, and the updated exposure offset is fed back to the image processor in real time, so that the exposure degree of the face region is accurately controlled.

[0081] As shown in Figure 4 The embodiment of the present application provides an electronic device, which comprises a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114; the memory 113 is used for storing a computer program; and the processor 111 is used for executing the program stored in the memory 113 to realize the steps of the face brightness statistics method and the exposure control method provided by any one of the preceding method embodiments.

[0082] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the face brightness statistics method and the exposure control method provided by any one of the preceding method embodiments.

[0083] It should be noted that, in the present document, relational terms such as "first" and "second", and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0084] The above examples are only used to illustrate the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made according to the essence of the present application should be covered within the protection scope of the present application.

Claims

1. A face brightness statistics method and exposure control method, characterized in that, The method comprises the following steps: acquiring a frame of image to be processed, detecting a face, and acquiring a face region in the image; determining whether the number of face regions in the image is greater than zero; if the number of face regions in the image is zero, and the number of face regions detected in continuous n frames of images is zero, setting the exposure adjustment amount fed back to the image processor to zero; if the number of face regions in the image is zero, but the cumulative number of image frames in which the number of face regions detected is zero does not reach n frames, maintaining the exposure adjustment amount fed back to the image processor unchanged; n is a positive integer; if the number of face regions in the image is greater than zero, segmenting face pixel points from each face region in the image by using the Otsu method and the morphological method, acquiring the average value of Y channel pixels of the face pixel points, determining the average brightness of each face region, and obtaining the face brightness of the image; if the brightness difference between the face brightness and a preset target brightness is within a preset tolerance, maintaining the exposure adjustment amount fed back to the image processor unchanged; if the brightness difference between the face brightness and the preset target brightness exceeds the preset tolerance, updating the exposure adjustment amount and feeding it back to the image processor.

2. The face brightness statistics method and exposure control method according to claim 1, wherein, The method of segmenting face pixel points from each face region in the image by using the Otsu method and the morphological method, acquiring the average value of Y channel pixels of the face pixel points, determining the average brightness of each face region, and obtaining the face brightness of the image comprises the following steps: 201, acquiring each face region in a frame of image; 202, using the Otsu method to obtain the segmentation threshold of Y channel pixels of candidate faces and backgrounds in the i-th face region; i is a natural number greater than zero; 203, performing binaryzation processing on the face region according to the segmentation threshold to obtain a binaryzation image, performing morphological opening operation on the binaryzation image, acquiring the largest connected region in the image after the morphological opening operation, the pixel points on the connected region being the screened face pixel points, and acquiring the average value of Y channel pixels of the face pixel points to obtain the face average brightness of the i-th face region; 204, if multiple face regions are detected in a frame of image, repeating steps 202-204 to acquire face pixel points of each face region, and acquiring the average value of Y channel pixels of the face pixel points of each face region to determine the face brightness of the image.

3. The face brightness statistics method and exposure control method according to claim 2, characterized by, The step 202 comprises the following steps: Let the segmentation threshold of Y channel pixel points belonging to the candidate face and the background in the i-th face region of the current image be T(i), the pixel points with Y channel pixel value lower than T(i) in the face region are marked as candidate face pixel points, the proportion of the total number of the candidate face pixel points in the total number of pixels in the face region is w0(i), and the average value of Y channel pixels of the candidate face pixel points is Y0(i); the pixel points with Y channel pixel value greater than or equal to T(i) in the face region are marked as background pixel points, the proportion of the total number of the background pixel points in the total number of pixels in the face region is w1(i), and the average value of Y channel pixels of the background pixel points is Y1(i), then the average value of Y channel pixels of the face region is is represented as:

4. The face brightness statistics method and exposure control method according to claim 3, wherein, The step 202 further comprises the following steps: The interval variance G(i) of the face region is represented as: The segmentation threshold of the face region is obtained by maximizing the principle of the interval variance G(i), and the candidate face pixel points of the face region are obtained according to the segmentation threshold T(i).

5. The face brightness statistics method and exposure control method according to claim 2, wherein, In the step 203, the morphological opening operation on the binaryzation image is represented as: where P"(i) represents the face region image after morphological opening operation, P'(i) represents the binary image of the face region, B represents the structure element, and the symbols represent morphological opening operation, and Θ and respectively represent morphological erosion and dilation.

6. The face brightness statistical method and exposure control method according to claim 2, wherein In the step 204, the face brightness of the image is represented as: Wherein, Y represents the face brightness of the current image, N represents the number of face regions detected in the current image; represents the average face brightness of the i-th face region.

7. The face brightness statistics method and exposure control method according to claim 1, wherein, The brightness difference between the face brightness and a preset target brightness is represented as: ΔY=Y'-Y Y' is the preset target brightness, and Y is the face brightness.

8. The face brightness statistics method and exposure control method according to claim 7, characterized by, If the brightness difference between the face brightness and the preset target brightness is within a preset tolerance, the exposure adjustment amount fed back to the image processor is maintained unchanged. If the luminance difference between the human face luminance and the preset target luminance exceeds a preset tolerance, the exposure adjustment amount is updated and fed back to the image processor, denoted as: u t = u t-1 + Δu wherein d represents a change rate of the exposure offset adjustment amount of the tth frame image, Δu represents an exposure adjustment amount corresponding to the tth frame image, u t represents an exposure offset amount of the tth frame image, u t-1 represents an exposure offset amount of the t-1th frame image, k0 and k1 represent preset exposure adjustment intensities.

9. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used for storing a computer program; The processor is used for executing the program stored on the memory, and realizes the steps of the human face luminance statistical method and the exposure control method according to any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the human face luminance statistical method and the exposure control method according to any one of claims 1-8.

Images