Method for rapidly improving brightness of face area under strong light background

Abstract

The invention provides a method for quickly improving the brightness of a human face area under a strong light background. The method comprises the following steps: S1, detecting a human face; s2, whether a result exists or not is judged, if not, the default exposure value is kept, and the step S2 is executed again; if so, carrying out the next step; s3, acquiring coordinate values of the face region; s4, judging whether the AE value in the coordinate area exceeds a normal value or not, and if not, returning; if so, carrying out the next step; s5, adjusting the AE exposure value in the ROI to reach a target value; and S6, seeing the human face clearly, and returning to the step S1. According to the method, the exposure value of the target area can be improved by simply changing the AE weight value of the designated area, and meanwhile, other exposure values except the target area in the picture are reduced, so that image details of a darker area are displayed.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for quickly increasing the brightness of a face area under a strong light background. Background Art

[0002] When high-brightness areas under strong light and relatively low-brightness areas such as shadows and backlight exist in the image at the same time, the camera output image will show that the bright areas become white due to overexposure, and the black areas become black due to underexposure, which seriously affects the image quality. The camera has limitations in the performance of the brightest area and the darker area in the same scene. This limitation is usually referred to as "lack of dynamic range". When this phenomenon exists, the strong light suppression function can be realized by using DSP technology, that is, weakening the strong light part and quantizing the dark light part to achieve light balance, adjust the brightness of the video signal to normal, and avoid too much contrast between the front and back in the same image.

[0003] Strong light suppression is a kind of "light suppression" used in low-light environments when the background light is extremely strong. It is suitable for areas such as toll booths / parking lot entrances and exits. It can capture faces in dark environments under extreme light.

[0004] The purpose of wide dynamic range / glare suppression and background light compensation is to adapt the camera to more scenes. The main performance is: when the light intensity is seriously unbalanced and the change range is large, wide dynamic range can capture this sudden change, making the picture texture more perfect and avoiding the situation of inconsistent light and dark changes; and the camera with glare suppression processes sunlight or other direct light sources. Under certain light conditions, for the camera subject that you want to highlight, glare suppression suppresses the intensity of light that interferes with the subject.

[0005] However, strong light suppression is not sensitive to areas with large light changes, and has certain defects in monitoring the main outline of the object. When there are both high exposure intensity and dark scenes, the exposure intensity is high, and even if the wide dynamic range is turned on, it is not enough to balance the scene. The overexposed areas are still white, and the face areas in the dark are still black, making it difficult to see the details of the face or the target area.

[0006] In addition, the commonly used technical terms in the prior art are:

[0007] AE (Auto Exposure) is automatic exposure. The AE statistical information module obtains the brightness of the current image in real time, compares it with the set image brightness, and dynamically adjusts the sensor's exposure time, analog gain, and digital gain to achieve the same brightness as the set brightness.

[0008] ISP (Image Signal Processor) is mainly used to process the signals output by the front-end image sensor. It directly affects the quality of the image presented to the user. After the image is collected by CCD or CMOS, it needs to be processed in the later stage to better adapt to different environments and restore the details of the scene under different optical conditions. In ISP, it will complete various functions such as automatic white balance / automatic exposure.

[0009] Wide dynamic range: Both particularly bright and dark areas in a scene can be seen clearly at the same time.

[0010] WDR, wide dynamic range, is the ratio of the brightest brightness signal value that an image can distinguish to the darkest brightness signal value that it can distinguish. Wide dynamic images are currently divided into digital wide dynamic and sensor wide dynamic. Sensor wide dynamic requires the support of both the sensor and the backend ISP, and uses a multi-frame exposure method, which works on both the bright and dark areas. Bright light suppression: One of the modern terms for bright light suppression refers to the process of processing the video information of the bright light part of the image through DSP, adjusting the signal brightness of the video to the normal range, and avoiding too much contrast between the front and back of the same image.

[0011] ROI (region of interest) is used to specify the area for calculating statistical brightness. The whole image is commonly used, but a central area can also be used. The area to be processed is outlined in the image to be processed in the form of a box, circle, ellipse, irregular polygon, etc., which is called the region of interest, ROI. Various operators and functions are often used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI and perform the next step of image processing. Summary of the invention

[0012] In order to solve the above problems, the purpose of this application is to add face recognition technology to low-power cameras such as doorbells, cat-eye cameras and other electronic products. When a face is recognized against a strong light background, the strong light will be overexposed and the face will be dark. At this time, the face that appears in the video is very dark, and it is difficult to see who it is, and the user experience will be very poor. The purpose of this invention is to increase the brightness of the designated area, i.e. the face area, against a strong light background, so that the face can be clearly seen. When the pedestrian walks away, normal AE convergence will be restored to ensure normal video image effects.

[0013] Specifically, the present invention provides a solution in which the AE weight value based on ISP can be set and adjusted, and a method for seeing the details of the subject is achieved by judging the exposure condition of the scene, identifying the designated area, and adjusting the AE weight.

[0014] The method comprises the following steps:

[0015] S1, face detection: determine the presence and location of a face by detecting the positions of key feature points of the face; the algorithms used include the active shape model ASM and the active appearance model AAM based on the shape model;

[0016] S2, determine whether there is a result. If the following judgment conditions are met, it means that a face is detected: It can be expressed as: if (score>threshold&&overlap <threshold_overlap){

[0017] printf("Face detected\n");

[0018] }else{

[0019] print("No results detected\n");

[0020] }

[0021] Where score represents the score of the face area, threshold represents the set threshold, overlap represents the overlap between the face area and other face areas, and threshold_overlap represents the set overlap threshold.

[0022] If not, keep the current environment exposure value unchanged;

[0023] Repeat step S2;

[0024] If yes, proceed to the next step;

[0025] S3, obtain the coordinate value of the face area, which can be obtained through the following code:

[0026] Get yuv data

[0027] IMP_FrameSource_GetFrame(1,&frame_bak);

[0028] Use the acquired data for face detection

[0029] interface->preProcessSync(interface,frame_bak);

[0030] interface->processAsync(interface,frame_bak);

[0031] Get face detection results

[0032] interface->getResult(interface,(void**)&result);

[0033] main_result contains specific face coordinate points

[0034] memcpy(&main_result,result,sizeof(persondet_param_output_t));

[0035] Release Processing

[0036] ret=interface->releaseResult(interface,(void**)&result);

[0037] S4, determine whether the brightness value in the coordinate area exceeds the normal value, that is, less than the minimum value or greater than the maximum value, and use the coordinate value obtained in step S3 to determine the coordinate area. The following code can be used to parse the regional brightness:

[0038] Among them, x0 y0 x1 y1 are the coordinate points of the region, which are obtained by step S3; fram_bak is the YUV data address; target_min is the minimum brightness value of the normal range, 90, and target_max is the maximum brightness value of the normal range, 160:

[0039]

[0040]

[0041] If no, return to S1; if yes, proceed to the next step;

[0042] S5, adjust the AE exposure value in the ROI area to reach the target value, which can be achieved through the following code:

[0043]

[0044] S6, see the face clearly, can be achieved through the following code:

[0045] Type definition structure typedef struct{

[0046] Face recognition enable: IMPISPTuningOpsMode enable;

[0047] The left point of the face area for face recognition: unsigned int left;

[0048] The top point of the face area for face recognition: unsigned int top;

[0049] Right point of the face area for face recognition: unsigned int right;

[0050] The bottom point of the face area for face recognition: unsigned int bottom;

[0051] Target of face recognition face area: unsigned int target;

[0052] }IMPISPFaceAttr;

[0053] IMP_ISP_Tuning_SetFaceAe(IMPISPFaceAttr*saeattr)

[0054] Return to step S1.

[0055] The key feature points in step S1 include eyes, nose, and mouth.

[0056] The parameters involved in step S1 include:

[0057] Detection threshold: The threshold used to determine whether a face exists. When the score of the detected face is higher than the threshold, it is considered that the face is detected. The threshold range is usually between 0 and 1, and 0.8 is taken here. Minimum face size: It is used to limit the minimum size of the detected face to between 20 and 50 pixels. Faces smaller than this size will be ignored.

[0058] Image pyramid: used for multi-scale detection. It detects faces of different sizes by scaling the input image to different scales.

[0059] Non-maximum suppression (NMS): used to remove overlapping detection frames. When multiple detection frames overlap, only the detection frame with the highest score is retained.

[0060] In step S2, the current environmental exposure value is kept unchanged, generally 300lux-500lux, based on the stable exposure value of the actual scene.

[0061] The method sets and adjusts the AE weight value based on the backlight compensation, wide dynamic adjustment, and exposure compensation characteristics of the ISP; the AE weight is set by calling the following API, IMP_ISP_Tuning_SetAeWeight (IMPISPAEWeightAttr*ae_weight); the ae_weight structure includes the weight setting enable of the region of interest, the global weight setting enable, the weight value of the region of interest 0 to 8, and the global weight value 0 to 8; the region recognition obtains the YUV data and sends it to the face recognition algorithm to output the desired region coordinates;

[0062] Then, the target brightness exposure is set to automatically converge the AE ROI function algorithm to achieve a balanced effect. When the face moves away from the video scene and the face recognition algorithm does not output a detection result, it is considered that the face has moved away from the camera scene. At this time, the default brightness and automatic exposure are restored by calling the interface IMP_ISP_Tuning_SetBrightness(128).

[0063] The face recognition algorithm may adopt Ingenic-t40-facepersonpetDet; the setting target brightness may be performed through an ISP hardware module exposure automatic convergence AE ROI function algorithm.

[0064] Therefore, the advantage of the present application is that the method can simply increase the exposure value of the target area by changing the AE weight value of the specified area while reducing the exposure values ​​of other areas in the picture except the target area, so that the image details of the darker area can be displayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0066] Figure 1 It is a flow chart of the method of the present application.

[0067] Figure 2 It is a schematic diagram of the code implementation of the method embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0069] like Figure 1 As shown, the present application relates to a method for quickly increasing the brightness of a face area under a strong light background, the method comprising the following steps:

[0070] S1, face detection; the presence and position of a face are determined by detecting the positions of key feature points of the face (such as eyes, nose, mouth, etc.); commonly used algorithms include Active Shape Model (ASM) and Active Appearance Model (AAM) based on shape models; the parameters involved include:

[0071] Detection threshold: the threshold used to determine whether a face exists; a face is considered detected only when the score of the detected face is higher than the threshold. The threshold range is usually between 0 and 1, and 0.8 is taken here; Minimum face size: the minimum size used to limit the detected face; the minimum size is between 20 and 50 pixels, and faces smaller than this size will be ignored;

[0072] Image pyramid: used for multi-scale detection; by scaling the input image to different scales, faces of different sizes can be detected;

[0073] Non-maximum suppression (NMS): used to remove overlapping detection frames; when multiple detection frames overlap, only the detection frame with the highest score is retained;

[0074] S2: Determine whether the result satisfies the following judgment conditions to indicate that a face has been detected:

[0075] It can be expressed as:

[0076] if(score>threshold&&overlap <threshold_overlap){

[0077] printf("Face detected\n");

[0078] }else{

[0079] print("No results detected\n");

[0080] }

[0081] Where score represents the score of the face area, threshold represents the set threshold, overlap represents the overlap between the face area and other face areas, and threshold_overlap represents the set overlap threshold.

[0082] If not, keep the current ambient exposure value unchanged

[0083] Generally, the exposure range is 300lux-500lux, depending on the stable exposure value of the actual scene.

[0084] Repeat step S2; if yes, proceed to the next step;

[0085] S3, obtain the coordinate value of the face area

[0086] Get the coordinate value through the following code:

[0087] / *Get yuv data* /

[0088] IMP_FrameSource_GetFrame(1,&frame_bak);

[0089] / *Use the acquired data for face detection* /

[0090] interface->preProcessSync(interface,frame_bak);

[0091] interface->processAsync(interface,frame_bak);

[0092] / *Get face detection results* /

[0093] interface->getResult(interface,(void**)&result);

[0094] / *main_result contains specific face coordinate points* /

[0095] memcpy(&main_result,result,sizeof(persondet_param_output_t));

[0096] / *Release process* /

[0097] ret=interface->releaseResult(interface,(void**)&result);

[0098] S4, determine whether the brightness value in the coordinate area exceeds the normal value, that is, less than the minimum value or greater than the maximum value, use the coordinate value obtained in S3 to determine the coordinate area, and parse the regional brightness through the following code: x0 y0 x1 y1 is the regional coordinate point, obtained from step S3. fram_bak is the yuv data address.

[0099] target_min is the minimum brightness value in the normal range, 90, and target_max is the maximum brightness value in the normal range, 160.

[0100]

[0101]

[0102] If no, return to S1; if yes, proceed to the next step; S5, adjust the AE exposure value in the ROI area to reach the target value

[0103]

[0104] S6, see the face clearly

[0105]

[0106]

[0107] IMP_ISP_Tuning_SetFaceAe(IMPISPfaceAttr*saeattr);

[0108] The method sets and adjusts the AE weight value based on the backlight compensation, wide dynamic adjustment, exposure compensation and other characteristics of the ISP; the AE weight is set by calling the following API, IMP_ISP_Tuning_SetAeWeight (IMPISPAEWeightAttr*ae_weight); the ae_weight structure contains the weight setting enable of the region of interest, the global weight setting enable, the weight value of the region of interest (0-8) and the global weight value (0-8). Region recognition obtains yuv data and sends it to the face recognition algorithm Ingenic-t40-facepersonpetDet to output the desired region coordinates. The Ingenic-t40-facepersonpetDet is the name of the face recognition algorithm involved in Ingenic products.

[0109] Then, by setting the target brightness, the tiziano2.0 exposure automatic convergence AE ROI function algorithm is used to converge to achieve a balanced effect; the tiziano2.0 is the name of the Junzheng ISP algorithm module. When the face in the video scene moves away, the face recognition algorithm does not output a detection result, and it is considered that the face has moved away from the camera scene. At this time, the default brightness is restored and the exposure is automatically performed by calling the interface IMP_ISP_Tuning_SetBrightness(128), such as Figure 2 shown.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for quickly increasing the brightness of a face area under a strong light background, characterized in that: The method comprises the following steps: S1, face detection: determine the presence and location of a face by detecting the positions of key feature points of the face; the algorithms used include the active shape model ASM and the active appearance model AAM based on the shape model; S2, determine whether there is a result. If the following judgment conditions are met, it means that a face is detected: It can be expressed as: if(score>threshold&&overlap <threshold_overlap){ printf("Face detected\n"); }else{ print("No results detected\n"); } Where score represents the score of the face area, threshold represents the set threshold, overlap represents the overlap between the face area and other face areas, and threshold_overlap represents the set overlap threshold. If not, keep the current environment exposure value unchanged; Repeat step S2; If yes, proceed to the next step; S3, obtain the coordinate value of the face area, which can be obtained through the following code: Get yuv data IMP_FrameSource_GetFrame(1,&frame_bak); Use the acquired data for face detection interface->preProcessSync(interface,frame_bak); interface->processAsync(interface,frame_bak); Get face detection results interface->getResult(interface,(void**)&result); main_result contains specific face coordinate points memcpy(&main_result,result,sizeof(persondet_param_output_t)); Release process ret=interface->releaseResult(interface,(void**)&result); S4, determine whether the brightness value in the coordinate area exceeds the normal value, that is, less than the minimum value or greater than the maximum value, and use the coordinate value obtained in step S3 to determine the coordinate area. The following code can be used to parse the regional brightness: Among them, x0 y0 x1 y1 are the coordinate points of the region, which are obtained by step S3; fram_bak is the YUV data address; target_min is the minimum brightness value of the normal range, 90, and target_max is the maximum brightness value of the normal range, 160: If no, return to S1; if yes, proceed to the next step; S5, adjust the AE exposure value in the ROI area to reach the target value, which can be achieved through the following code: S6, see the face clearly, can be achieved through the following code: Type definition structure typedef struct{ Face recognition enable: IMPISPTuningOpsMode enable; The left point of the face area for face recognition: unsigned int left; The top point of the face area for face recognition: unsigned int top; Right point of the face area for face recognition: unsigned int right; The bottom point of the face area for face recognition: unsigned int bottom; Target of face recognition face area: unsigned int target; }IMPISPFaceAttr; IMP_ISP_Tuning_SetFaceAe(IMPISPFaceAttr*saeattr) Return to step S1.

2. The method for rapidly increasing the brightness of a face area under a strong light background according to claim 1, characterized in that: The key feature points in step S1 include eyes, nose, and mouth.

3. The method for rapidly increasing the brightness of a face area under a strong light background according to claim 1, characterized in that: The parameters involved in step S1 include: Detection threshold: The threshold used to determine whether a face exists. When the score of the detected face is higher than the threshold, it is considered that the face is detected. The threshold range is usually between 0 and 1, and 0.8 is taken here. Minimum face size: It is used to limit the minimum size of the detected face to between 20 and 50 pixels. Faces smaller than this size will be ignored. Image pyramid: used for multi-scale detection. It detects faces of different sizes by scaling the input image to different scales. Non-maximum suppression (NMS): used to remove overlapping detection frames. When multiple detection frames overlap, only the detection frame with the highest score is retained.

4. The method for rapidly increasing the brightness of a face area under a strong light background according to claim 1, characterized in that: In step S2, the current environmental exposure value is kept unchanged, generally 300lux-500lux, based on the stable exposure value of the actual scene.

5. The method for rapidly increasing the brightness of a face area under a strong light background according to claim 1, characterized in that: The method sets and adjusts the AE weight value based on the backlight compensation, wide dynamic adjustment, and exposure compensation characteristics of the ISP; the AE weight is set by calling the following API, IMP_ISP_Tuning_SetAeWeight (IMPISPAEWeightAttr*ae_weight); the ae_weight structure includes the weight setting enable of the region of interest, the global weight setting enable, the weight value of the region of interest 0 to 8, and the global weight value 0 to 8; the region recognition obtains the YUV data and sends it to the face recognition algorithm to output the desired region coordinates; Then, the target brightness exposure is set to automatically converge the AE ROI function algorithm to achieve a balanced effect. When the face moves away from the video scene and the face recognition algorithm does not output a detection result, it is considered that the face has moved away from the camera scene. At this time, the default brightness and automatic exposure are restored by calling the interface IMP_ISP_Tuning_SetBrightness(128).

6. The method for rapidly increasing the brightness of a face area under a strong light background according to claim 5, characterized in that: The face recognition algorithm may adopt Ingenic-t40-facepersonpetDet; the setting target brightness may be performed through an ISP hardware module exposure automatic convergence AE ROI function algorithm.

Images