Method for snapshot and optimal selection through face attributes

Through multiple screening of face detection and attribute model, the problems of low quality and false detection of face images in the video surveillance system are solved, and efficient and accurate selection of face captures and excellent images are achieved, ensuring the clarity and integrity of the image.

CN119027993APending Publication Date: 2024-11-26INGENIC SEMICON CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310606610.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the video surveillance system, the storage quality of face images is low and incomplete, the analysis is cumbersome and time-consuming, and the confidence and characteristic value filtering effect of target detection are poor, so it cannot be completely filtered with high confidence and low image quality. false detection image.

Method used

The face detection frame is obtained and expanded through the face detection algorithm, and the eyes and key points in the face attribute model are filtered, combined with the image blur model for three screenings, filtering the face images without eyes, side faces and blurred faces to ensure output image quality and accuracy.

Benefits of technology

It realizes the rapid and efficient screening of qualified face images, improves the accuracy and image quality of the best shot selection, avoids mis-checking and blurring problems, and ensures the reliability and efficiency of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027993A_ABST
    Figure CN119027993A_ABST
Patent Text Reader

Abstract

The invention provides a method for snapshot and optimization selection through face attributes. The method comprises the following steps: S1, obtaining a face detection frame by using a face detection model and expanding the face detection frame; s2, performing first screening by using eye shielding in the face attribute model; s3, performing second screening by using key points in the face attribute model; and S4, performing third screening by using the picture fuzzy model, and outputting. Obtaining a face detection frame based on a face detection algorithm and expanding the face detection frame, judging whether two eyes of a face exist or not through eyes in a face attribute model, performing first screening on a face image to filter a face image which does not have eyes and is falsely detected, and performing second screening on side face data by using key points in face attributes to obtain a face detection result; and scores of different angles of each face image can be obtained, and the higher the score is, the more right the face is. And then performing third screening on the scored face images meeting the conditions by using a fuzzy model, filtering the relatively fuzzy face images by using a fuzzy threshold value, and finally outputting the face image as a result after snapshot and optimal selection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for selecting the best shot based on facial attributes. Background Art

[0002] In the prior art, with the accelerated application of video surveillance systems in all walks of life, many surveillance devices will store thousands of face images of different scenes. Due to different storage environments, the stored images will be of low quality, incomplete, and of poor usability. Analyzing these data will be cumbersome, and it will take a lot of time to screen them. Therefore, this method can filter out unqualified faces by using the attributes of the face, such as the position of the eyes and key points of the face, to quickly and efficiently retain qualified faces. It can also be used as input data for face recognition, create conditions for investigation and evidence collection by relevant departments, and provide reliable and effective clues and basis for later data tracing, and be applied to other high-quality face scenes.

[0003] The existing snapshot selection methods generally use the confidence of target detection to filter out face images with poor effects. There are also methods that rely on the characteristic values ​​of the target to distinguish the effect quality of the face image through the characteristic values. Some snapshot methods use Sony professional-grade sensors to support strong light suppression and 3D noise reduction functions, which effectively ensure the image quality. It can better guarantee the snapshot implementation effect in various application scenarios under complex environments.

[0004] However, when using the confidence and feature values ​​of target detection for optimization, this solution will only filter out faces with low confidence, and cannot completely filter out faces with high confidence and low image quality, as well as faces with false detections. Using a high-resolution camera ensures image clarity, which will improve the capture effect to a certain extent, but there are also some scene factors, such as people in motion, unstable camera focus, etc., which will also reduce the image quality, and the cost of using professional lenses will be relatively high.

[0005] In addition, commonly used technical terms include:

[0006] Face attributes: mainly include 72 pairs of key points of the face, left and right eye occlusion, and image blur attributes. Snapshot: capture the "fleeting" image of the target scene.

[0007] Selection: Select the images that meet the facial attribute requirements from the captured images.

[0008] Facial key points: Facial key points refer to the key areas of the face located on the face image, including eyebrows, eyes, nose, mouth, face, etc. The facial key point model is learned by a neural network through a large number of facial key point graphs, and then predicts the positions of 72 pairs of key points.

[0009] Image blur model: The captured image may be blurred due to the camera's own pixels, camera movement, focus not suitable for the current scene, etc. The image blur model classifies the image blur from low to high, learns through a neural network, and then predicts the classification level of the image blur.

[0010] Face and eye occlusion: When neither the left nor right eye or both eyes are visible or the eyes are occluded with an image, the image data is classified, the neural network is used to learn, and then the classification level of the face and eyes in the image is predicted. Summary of the invention

[0011] In order to solve the above problems, the purpose of this application is to obtain and expand the face detection frame based on the face detection algorithm, determine whether the two eyes of the face exist through the eyes in the face attribute model, perform the first screening of the face image, filter the face images without eyes and misdetected, use the key points in the face attributes, and perform the second screening of the side face data. It will get the scores of each face image at different angles. The higher the score, the more positive the face. Then, the scored face images that meet the conditions are screened for the third time by the fuzzy model, and the fuzzy threshold is used to filter the fuzzier face images. The face image output at the end is the result after the capture and selection.

[0012] Specifically, the present invention provides a method for selecting snapshots based on facial attributes, the method comprising the following steps:

[0013] S1. Use the face detection model to get the face detection frame and expand it:

[0014] The face frame of the captured image is determined by the face detection model, and the face is expanded from the center point of the face frame to the surrounding area. The expansion length is half of the largest side of the face frame after random expansion. Finally, an equilateral square is obtained, and then it is proportionally scaled to the image size required by the face attribute. Here, the width and height are (96,96). This image is a face expansion image to ensure that the effective area of ​​the input face model is consistent; S2. Use the eye occlusion in the face attribute model to perform the first screening:

[0015] The face expansion image is sent to the face attribute model to obtain the score value of the face eye occlusion predicted by the image. Through testing, it is found that when the score threshold is less than 0.5, the closer it is to 0, the eyes will be completely invisible, and the closer it is to 1, the more obvious the effect of seeing the eyes is; through the two eye score thresholds, the false detection images of face detection in the snapshot and the snapshot images with serious face eye occlusion are filtered out. After testing, the score threshold is 0.7. When it is greater than 0.7, the process will continue. When it is less than 0.7, the current face will be terminated and the next face will be continued, so that the snapshot results are more neat and it is guaranteed that the snapshot images are all faces; S3. Use the key points in the face attribute model to perform a second screening:

[0016] After the first filtering, the temple, iris, and nose tip of the face key points are used to calculate the ratio score of the side face.

[0017] First, calculate the distance between the left and right temples (x1, y1), (x11, y11) and the nose (x, y) respectively: L1 = math.sqrt((x-x11)**2+(y-y11)**2).

[0018] R1=math.sqrt((x-x1)**2+(y-y1)**2) to determine the direction of the side face.

[0019] Then calculate the distance ratio between the temple to the left and right irises (x2, y2), (x22, y22) and the distance between the iris and the nose (x, y).

[0020] scale1=math.sqrt((x22-x11)**2+(y22-y11)**2) / math.sqrt(x-x22)**2+(y-y22)**2);

[0021] Scale2 = math.sqrt((x2-x1)**2+(y2-y1)**2) / math.sqrt(x-x2)**2+(y-y2)**2); Finally, the ratio is normalized (divided by the maximum value of the ratio 1.8) to 0 to 1. After actual test results, when the side face score threshold is greater than 0.83, the more upright the face is, and when it is less than 0.83, the side face effect is more obvious. Key point screening is used to filter out the side face snapshots with a score less than 0.83, making the snapshot results more accurate;

[0022] S4. Use the image fuzzy model to perform the third screening:

[0023] After the second screening through key points, the image blur model is used to input the face expansion image to obtain the blur score of this image. This score is the predicted value after model learning and training. The score is a decimal between 0 and 1. Through testing results, it is found that when the blur score threshold is greater than 0.4, the original face snapshot is clearer, and when it is less than 0.4, the face snapshot will have a blurry effect. After this screening, a clear, frontal face snapshot will be obtained in the end.

[0024] The process of face expansion in step S1 further includes:

[0025] S1.1, after face detection, the coordinates of the face detection frame in the figure are obtained: upper left (x0, y0), lower right (x1, y1);

[0026] S1.2, find the maximum side h of the detection frame: max((y1-y0), (x1-x0)), find the coordinates of the center point of the detection frame (x, y); x = (x0+x1) / 2, y = (y0+y1) / 2; find the range of values ​​of the random expansion length radius: (0, h / 2); update the expanded face frame: x0 = x-radius; x1 = x+radius; y0 = y-radius; y1 = y+radius;

[0027] S1.3, cropping the expanded face frame on the captured image;

[0028] S1.4, scale the expanded graph to (96,96);

[0029] S1.5, obtain the final face expansion image.

[0030] The process of using the key to perform the second screening in step S3 further includes:

[0031] S3.1, facial key point model, is a model that can predict key points and other results by learning and training a large amount of data with labeled information such as facial key points and eye occlusion;

[0032] S3.2, 72 pairs of key points on the face:

[0033] The points on the contour line from one cheek to the other are marked as: 0, 1, 2, ..., 12, where 0-12 are symmetrical, 1 and 12 are symmetrical, ... 5 and 7 are symmetrical, and 6 is marked as the vertex of the chin on the cheek contour line; the two eyes are: 12 to 20 are marked as points on the contour line of one eye, and 30 to 37 are marked as points on the contour line of the other eye;

[0034] The distance between 13 and 21 is the distance from the edge of the eye to the edge of the iris in one eye;

[0035] The points on the outline of the eyebrow above one eye are labeled: 22 to 29;

[0036] The distance between 30 and 38 is the distance from the edge of the eye to the edge of the iris of the other eye;

[0037] The points on the outline of the eyebrow on the other eye are labeled: 39 to 46;

[0038] The points on the outline of the nose are labeled: 47 to 56;

[0039] The points at the nose tip are marked as: 57;

[0040] The distance between 51 and 57 is the distance from one nostril to the tip of the nose;

[0041] The distance between 52 and 57 is the distance from the other nostril to the tip of the nose;

[0042] The distance between 0 and 57 is the distance from one cheek to the tip of the nose;

[0043] The distance between 21 and 57 is the distance from the edge of the iris of one eye to the tip of the nose;

[0044] The distance between 38 and 57 is the distance from the edge of the iris of the other eye to the tip of the nose;

[0045] The distance between 12 and 57 is the distance from the other cheek to the tip of the nose;

[0046] The point at one corner of the mouth is labeled: 58; the point at the other corner of the mouth is labeled: 62;

[0047] The points on the outer contour of the lips are marked as follows: 58, 59 to 65;

[0048] The points on the inner contour of the lips are marked as follows: 58, 66 to 68, 62, 69 to 71;

[0049] S3.3, calculate the distance between key points 0 and 21: left1;

[0050] Calculate the distance between key points 21 and 57: left2;

[0051] Calculate the distance between key points 12 and 38: right1;

[0052] Calculate the distance between key points 38 and 57: right2;

[0053] Calculate the distance between key points 0 and 57: L1;

[0054] Calculate the distance between key points 0 and 12: R1;

[0055] S3.4, Scale1 = left1 / left2

[0056] Scale2=right1 / right2

[0057] S3.5, determine whether L1 is greater than R1?

[0058] If yes, then Scale = Scale1;

[0059] If not, then Scale = Scale2;

[0060] S3.6, Scale = 1-Scale / 1.8;

[0061] S3.7, Scale greater than 0.83, the front face effect is better; Scale less than 0.83, the side face effect is more obvious.

[0062] Therefore, the advantages of this application are:

[0063] 1. When using the eye occlusion in the face attribute model for the first screening, the model can determine the state of the face through the score values ​​of the left and right eyes, filter out the misdetected pictures without eyes, and make sure that the captured pictures are all face pictures, thus avoiding the false detection effect of face detection and making the captured face pictures more neat.

[0064] 2. When using the key points in the face attribute model for the second screening, the ratio between the temple to the iris and the iris to the nose tip is used to control the degree of the side face, thereby filtering out images with more obvious left and right side faces, so that the captured face images are all in a frontal state, ensuring the accuracy of the captured images.

[0065] 3. Although most of the images after the second screening are in a frontal state, many of the collected images are not clearly focused and have motion blur due to lens blur and motion scenes. In order to filter this part of the data, the image blur model is used here for the third screening to filter out images with scores less than the blur threshold. The current blur score threshold and angle score threshold are both adjustable, which makes the control of the captured images flexible and convenient, greatly improving the quality of the captured images. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0067] Figure 1 is a schematic diagram of the process of this method.

[0068] Figure 2 is a schematic diagram of step S1 in the method.

[0069] FIG3( a ) is a schematic diagram of the eye occlusion effect of a face with a score less than 0.5 measured in this method.

[0070] FIG3( b ) is a schematic diagram of the eye occlusion effect of a face with a score greater than 0.5 measured in this method.

[0071] FIG. 4( a ) is one of the illustrations of distance calculation of facial attribute key points.

[0072] FIG4(b) is the second diagram of distance calculation of facial attribute key points.

[0073] Figure 5 It is a schematic flow chart of step S3 of the method.

[0074] FIG6( a ) is one of the illustrations of the effect of the actual capture of the second screening of key points.

[0075] FIG6( b ) is one of the illustrations of the effect of the actual capture of the second screening without key points. DETAILED DESCRIPTION

[0076] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0077] This application provides a method for selecting the best face capture by face attributes, such as Figure 1 The flowchart of the method is shown in FIG. 1 . The specific implementation steps of the method are described as follows: Step S1. Use the face detection model to obtain the face detection frame and expand

[0078] The camera collects data and performs face detection, cropping and expansion. That is, the data collected by the camera is detected through the face detection model to detect the position of the face frame, calculate the width, height and center coordinates of the face detection frame, and expand the face frame up, down, left and right at the center point. The expanded size is half of the maximum side of the face detection frame. The expanded face detection frame is used to crop the original image to obtain the expanded face image. That is, the face frame of the captured image is determined by the face detection model, and the face frame is expanded around the center point of the face frame. The expansion length is half of the maximum side of the face frame after random expansion. Finally, an equilateral square is obtained, and then it is scaled proportionally to the image size required for the face attributes. Here, the size is (96,96). This image is an expanded face image to ensure the consistency of the effective area of ​​the input face model.

[0079] Specific face expansion process, such as Figure 2 As shown: the process of face expansion in step S1 further includes:

[0080] S1.1, after face detection, the coordinates of the face detection frame in the figure are obtained: upper left (x0, y0), lower right (x1, y1);

[0081] S1.2, find the maximum side h of the detection frame: max((y1-y0), (x1-x0)), find the coordinates of the center point of the detection frame (x, y); x = (x0+x1) / 2, y = (y0+y1) / 2; find the range of values ​​of the random expansion length radius: (0, h / 2); update the expanded face frame: x0 = x-radius; x1 = x+radius; y0 = y-radius; y1 = y+radius;

[0082] S1.3, cropping the expanded face frame on the captured image;

[0083] S1.4, scale the expanded graph to (96,96);

[0084] S1.5, obtain the final face expansion image.

[0085] Step S2. Use the eye occlusion in the face attribute model to perform the first screening

[0086] The expanded face image is sent to the face attribute model, and the predicted scores of the left and right eyes are output. Through large-scale testing, it is found that the human images with eyes smaller than the score threshold of 0.5 are almost all falsely detected, distant, and blurred faces. The face images with the left and right eye score thresholds less than 0.5 are filtered separately to eliminate falsely detected non-face images and distant and blurred faces. That is, the face expanded image is sent to the face attribute model to obtain the score value of the face eye occlusion predicted by the image. Through testing, it is found that when the score threshold is less than 0.5, the closer it is to 0, the eyes will be completely invisible, and the closer it is to 1, the more obvious the effect of seeing the eyes. Through the two eye score thresholds, the falsely detected images of face detection in the snapshot image and the snapshot images with serious face eye occlusion are filtered to make the snapshot results more neat and ensure that the snapshot images are all faces. The specific effect is shown in Figure 3 (a) (b). Step S3. Use the key points in the face attribute model to perform a second screening. The expanded face image is sent to the face attribute model, and the positions of the 72 key points of the face are obtained at the same time. By analyzing the positions of the key points of the face, it is determined whether the face is on the left or right side. Then, by calculating the distance ratio between the corners of the eyes, the iris, and the nose, the degree of the side face is determined. The larger the ratio, the more obvious the side face effect. After the second screening, the side faces with large left and right amplitudes are filtered out. That is, after the first filtering, the temple, iris, and nose tip of the face key points are used to calculate the ratio score of the side face. First, the distance from the temple to the nose is calculated to determine the direction of the side face. Then, the ratio of the distance from the temple to the iris and the distance from the iris to the nose in this direction is calculated. Finally, the ratio is normalized to 0 to 1. After the actual test results, when the side face score threshold is greater than 0.83, the more upright the face is, and when it is less than 0.83, the more obvious the side face effect is. This screening can filter out the snapshots with a high degree of side face, making the snapshot results more accurate. The specific implementation process is as follows Figure 5 As shown, the process of using the key to perform a second screening in step S3 further includes:

[0087] S3.1, face key point model;

[0088] S3.2, 72 pairs of key points of the face, the key points are shown in Figure 4(a) and Figure 4(b):

[0089] The points on the contour line from one cheek to the other are marked as: 0, 1, 2, ..., 12, where 0-12 are symmetrical, 1 and 11 are symmetrical, ... 5 and 7 are symmetrical, and 6 is marked as the vertex of the chin on the cheek contour line; the two eyes are: 13 to 20 are marked as points on the contour line of one eye, and 30 to 37 are marked as points on the contour line of the other eye;

[0090] The distance between 13 and 21 is the distance from the edge of the eye to the edge of the iris of one eye; the points on the outline of the eyebrow on one eye are marked as 22 to 29; the distance between 30 and 38 is the distance from the edge of the eye to the edge of the iris of the other eye; the points on the outline of the eyebrow on the other eye are marked as 39 to 46;

[0091] The points on the outline of the nose are labeled: 47 to 56;

[0092] The points at the nose tip are marked as: 57;

[0093] The distance between 51 and 57 is the distance from one nostril to the tip of the nose;

[0094] The distance between 52 and 57 is the distance from the other nostril to the tip of the nose;

[0095] The distance between 0 and 57 is the distance from one cheek to the tip of the nose;

[0096] The distance between 21 and 57 is the distance from the edge of the iris of one eye to the tip of the nose;

[0097] The distance between 38 and 57 is the distance from the edge of the iris of the other eye to the tip of the nose;

[0098] The distance between 12 and 57 is the distance from the other cheek to the tip of the nose;

[0099] The point at one corner of the mouth is labeled: 58; the point at the other corner of the mouth is labeled: 62;

[0100] The points on the outer contour of the lips are marked as follows: 58, 59 to 65;

[0101] The points on the inner contour of the lips are marked as: 58, 66 to 68, 62, 69 to 71; S3.3, calculate the distance between the key points 0 and 21: left1;

[0102] Calculate the distance between key points 21 and 57: left2;

[0103] Calculate the distance between key points 12 and 38: right1;

[0104] Calculate the distance between key points 38 and 57: right2;

[0105] Calculate the distance between key points 0 and 57: L1;

[0106] Calculate the distance between key points 0 and 12: R1;

[0107] S3.4, Scale1 = left1 / left2

[0108] Scale2=right1 / right2

[0109] S3.5, determine whether L1 is greater than R1?

[0110] If yes, then Scale = Scale1;

[0111] If not, then Scale = Scale2;

[0112] S3.6, Scale = 1-Scale / 1.8;

[0113] S3.7: If the Scale is greater than 0.83, the frontal face effect is better; if the Scale is less than 0.83, the side face effect is more obvious. Step S4: Use the image blur model to perform the third screening

[0114] The face image after the second screening is sent to the blur model. This model will output the face blur score, which is a decimal between 0 and 1. Through large-scale data testing, it is found that the blur score threshold is set to 0.4. The higher the score, the clearer the face image. The face images with a score less than 0.4 are all noise-prone, blurry and unclear face images due to the long distance and the influence of the data acquisition lens. After the third screening, the final face capture result image will be obtained. That is, after the second screening through the key points, the image blur model is used to input the face expansion image to obtain the blur score of this image. Through the test results, it is found that when the blur score threshold is greater than 0.4, the original face capture image is clearer, and when it is less than 0.4, the face capture image will have a blur effect. After this screening, a clear, frontal face capture image will be finally obtained and output. The actual capture effect is shown in Figure 6(a) and Figure 6(b).

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for selecting snapshots based on facial attributes, characterized in that: The method comprises the following steps: S1. Use the face detection model to get the face detection frame and expand it: The camera collects data and performs face detection, cropping and expansion. That is, the face frame of the captured image is determined by the face detection model, and the face is expanded from the center point of the face frame to the surrounding area. The expansion length is half of the largest side of the face frame after random expansion. Finally, an equilateral square is obtained, and then it is proportionally scaled to the image size required by the face attributes. Here, the width and height are (96,96). This image is a face expansion image to ensure the consistency of the effective area of ​​the input face model. S2. Use the eye occlusion in the face attribute model to perform the first screening: The face expansion image is sent to the face attribute model to obtain the score value of the face eye occlusion predicted by the image. Through testing, it is found that when the score threshold is less than 0.5, the closer it is to 0, the eyes will be completely invisible, and the closer it is to 1, the more obvious the effect of seeing the eyes is. Through the two eye score thresholds, the false detection images of face detection in the snapshots and the snapshots with serious face eye occlusion are filtered out. After testing, the score threshold is 0.

7. When it is greater than 0.7, the process will continue. When it is less than 0.7, the current face will be terminated and the next face will be continued. This filtering makes the snapshot results more neat and ensures that the snapshots are all faces. S3. Use the key points in the face attribute model to perform a second screening: After the first filtering, the temple, iris, and nose tip of the face key points are used to calculate the ratio score of the side face. First, calculate the distance between the left and right temples (x1, y1), (x11, y11) and the nose (x, y) respectively: L1 = math.sqrt((x-x11)**2+(y-y11)**2). R1=math.sqrt((x-x1)**2+(y-y1)**2) to determine the direction of the side face. Then calculate the distance ratio between the temple to the left and right irises (x2, y2), (x22, y22) and the distance between the iris and the nose (x, y). scale1=math.sqrt((x22-x11)**2+(y22-y11)**2) / math.sqrt(x-x22)**2+(y-y22)**2); Scale2 = math.sqrt((x2-x1)**2+(y2-y1)**2) / math.sqrt(x-x2)**2+(y-y2)**2); Finally, the ratio is normalized, that is, divided by the maximum value of the ratio 1.8 to 0 to 1. After actual test results, when the side face score threshold is greater than 0.83, the more positive the face is, and when it is less than 0.83, the side face effect is more obvious. Use key point screening to filter out the side face snapshots with a score less than 0.83, so that the snapshot results are more accurate; S4. Use the image fuzzy model to perform the third screening: After the second screening through key points, the image blur model is used to input the face expansion image to obtain the blur score of this image. The blur score is the predicted value after model learning and training, and the score is a decimal between 0 and 1. Through testing results, it is found that when the blur score threshold is greater than 0.4, the original face snapshot is clearer, and when it is less than 0.4, the face snapshot will have a blurry effect. After this screening, a clear, frontal face snapshot will be obtained and output.

2. The method for selecting the best snapshots based on facial attributes according to claim 1, characterized in that: The process of face expansion in step S1 further includes: S1.1, after face detection, the coordinates of the face detection frame in the figure are obtained: upper left (x0, y0), lower right (x1, y1); S1.2, find the maximum side h of the detection frame: max((y1-y0), (x1-x0)), find the coordinates of the center point of the detection frame (x, y); x = (x0+x1) / 2, y = (y0+y1) / 2; find the range of values ​​of the random expansion length radius: (0, h / 2); update the expanded face frame: x0 = x-radius; x1 = x+radius; y0 = y-radius; y1 = y+radius; S1.3, cropping the expanded face frame on the captured image; S1.4, scale the expanded graph to (96,96); S1.5, obtain the final face expansion image.

3. The method for selecting the best snapshots based on facial attributes according to claim 1, characterized in that: The process of using the key points for the second screening in step S3 further includes: S3.1, a face key point model, wherein the model is a model capable of predicting key point results obtained by learning and training a large amount of data with labeled information such as face key points and eye occlusion; S3.2, 72 pairs of key points on the face: The points on the contour line from one cheek to the other are marked as: 0, 1, 2, ..., 12, where 0-12 are symmetrical, 1 and 11 are symmetrical, ... 5 and 7 are symmetrical, and 6 is marked as the vertex of the cheek contour line at the chin; the two eyes are: 13 to 20 are marked as points on the contour line of one eye, and 30 to 37 are marked as points on the contour line of the other eye; The distance between 13 and 21 is the distance from the edge of the eye to the edge of the iris in one eye; The points on the outline of the eyebrow above one eye are labeled: 22 to 29; The distance between 30 and 38 is the distance from the edge of the eye to the edge of the iris of the other eye; The points on the outline of the eyebrow on the other eye are labeled: 39 to 46; The points on the outline of the nose are labeled: 47 to 56; The points at the nose tip are marked as: 57; The distance between 51 and 57 is the distance from one nostril to the tip of the nose; The distance between 52 and 57 is the distance from the other nostril to the tip of the nose; The distance between 0 and 57 is the distance from one cheek to the tip of the nose; The distance between 21 and 57 is the distance from the iris edge of one eye to the tip of the nose; the distance between 38 and 57 is the distance from the iris edge of the other eye to the tip of the nose; The distance between 12 and 57 is the distance from the other cheek to the tip of the nose; The point at one corner of the mouth is labeled: 58; the point at the other corner of the mouth is labeled: 62; The points on the outer contour of the lips are marked as follows: 58, 59 to 65; The points on the inner contour of the lips are marked as follows: 58, 66 to 68, 62, 69 to 71; S3.3, calculate the distance between key points 0 and 21: left1; Calculate the distance between key points 21 and 57: left2; Calculate the distance between key points 12 and 38: right1; Calculate the distance between key points 38 and 57: right2; Calculate the distance between key points 0 and 57: L1; Calculate the distance between key points 0 and 12: R1; S3.4, Scale1 = left1 / left2 Scale2=right1 / right2 S3.5, determine whether L1 is greater than R1? If yes, then Scale = Scale1; If not, Scale = Scale2; S3.6, Scale = 1-Scale / 1.8; S3.7, Scale greater than 0.83, the front face effect is better; Scale less than 0.83, the side face effect is more obvious.

Citation Information

Cited By

  • Face special effect display method and device, equipment and medium

    CN121603693A