An automatic labeling device for a face detection model

Abstract

The application discloses a training method of a face detection model, which comprises the following steps: connecting the boundary points of the facial features of a face image to form a facial feature boundary line, and then expanding the facial feature boundary line to obtain a first facial feature detection frame; measuring and calculating the adjustment amount of the face posture angle and the coordinate axis; adjusting the size and coordinates of the first facial feature detection frame according to the adjustment amount of the coordinate axis to obtain a more accurate second facial feature detection frame; repeating the foregoing steps to obtain a plurality of second facial feature detection frames, generating a mark file according to the plurality of second facial feature detection frames to complete a training set, and training a face detection model according to the training set. The application provides a facial feature automatic marking tool of the face detection model trained by the method, which can reduce the labor cost consumption and improve the applicability of the face detection model.

Description

Technical Field

[0001] This application relates to the field of face recognition technology, and in particular to an automatic labeling tool for face detection models in face recognition. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. It involves using cameras or webcams to capture images or videos containing faces, automatically detecting and tracking faces within the images, and then performing facial recognition on the detected faces. This technology is also commonly referred to as image recognition or face identification.

[0003] During the process of facial recognition, the inventors discovered:

[0004] Current facial recognition relies on face detection models, which require a large amount of training data. Currently, all training data is manually collected and labeled, resulting in high labor costs and low efficiency. The amount of manually collected data is limited, and face detection models trained with this limited data have limited recognition capabilities. For example, they often fail to recognize faces with decorations, degrading the user experience.

[0005] Therefore, in order to reduce labor costs and improve the applicability of face detection models, it is necessary to provide an automatic labeling tool for face detection models. Summary of the Invention

[0006] This application provides a technical solution for an automatic labeling method for face detection models, which solves the technical problems of high labor costs and low efficiency in the prior art of manually collecting data to train models.

[0007] This application provides an automatic labeling method for a face detection model, comprising the following steps:

[0008] S01: Acquire a face image and collect the boundary points of facial features;

[0009] S02: Connect the boundary points of the facial features in the face image to form the boundary line of the facial features, and then expand the boundary line of the facial features to obtain the first facial feature detection box;

[0010] S03: Measure the facial pose angle and calculate the adjustment amount of the point coordinates on the first facial feature detection box based on the facial pose angle;

[0011] S04: Adjust the first facial feature detection box according to the adjustment amount to obtain a more accurate second facial feature detection box;

[0012] S05: Repeat steps S01-S04 to obtain multiple second facial feature detection boxes, and generate a label file based on the multiple second facial feature detection boxes to complete the training set;

[0013] S06: Train a face detection model based on the training set.

[0014] Furthermore, the boundary line of the facial features is expanded by 0.1 times to the first facial feature detection frame.

[0015] Furthermore, the facial feature boundary points were obtained using the ResNet50 face detection 106-point model.

[0016] Furthermore, the facial pose angles include pitch angle, yaw angle, and roll angle.

[0017] Furthermore, the pitch angle, yaw angle, and roll angle are obtained by combining the solvePnP method of OpenCV with the Euler angle calculation formula.

[0018] This application also provides an automatic labeling device for a face detection model, comprising:

[0019] The face image acquisition module (10) is used to acquire face images and collect the boundary points of facial features;

[0020] The detection processing module (11) is used to connect the facial feature boundary points of the face image to form a facial feature boundary line, and then expand the facial feature boundary line to obtain a first facial feature detection box; it is used to adjust the first facial feature detection box according to the coordinate adjustment amount of the points on the first facial feature detection box to obtain a more accurate second facial feature detection box.

[0021] The calculation module (12) is used to measure and calculate the face pose angle and the coordinate adjustment of the points on the first facial feature detection box;

[0022] The determination module (13) is used to train a face detection model based on the training set; wherein, multiple second facial feature detection boxes generate a label file to complete the training set.

[0023] Furthermore, the boundary line of the facial features is expanded by 0.1 times to the first facial feature detection frame.

[0024] Furthermore, the facial feature boundary points were obtained using the ResNet50 face detection 106-point model.

[0025] This application also provides a face recognition system, characterized in that it includes an automatic labeling device for the aforementioned face detection model.

[0026] This application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program that can execute the aforementioned automatic labeling method for the face detection model.

[0027] The embodiments provided in this application have at least the following beneficial effects:

[0028] This application discloses an automatic labeling method or apparatus for a face detection model. The method involves connecting the boundary points of facial features in a face image to form a facial feature boundary line, then expanding this boundary line to obtain a first facial feature detection box. The method then measures and calculates the facial pose angle and the coordinate adjustment amount of points on the first facial feature detection box. Based on the adjustment amount of the coordinate axes, the size and coordinates of the facial feature detection box are adjusted to obtain a more accurate second facial feature detection box. The aforementioned steps are repeated to obtain multiple second facial feature detection boxes. A labeling file is generated based on these multiple second facial feature detection boxes to complete the training set. Finally, a face detection model is trained using the training set. This automatic labeling method or apparatus for face detection models reduces labor costs and improves the applicability of the face detection model. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 A flowchart illustrating an automatic labeling method for a face detection model provided in this application embodiment;

[0031] Figure 2 A schematic diagram showing the directions of pitch, yaw, and roll angles in a human face pose;

[0032] Figure 3 A schematic diagram for obtaining the second facial sensory detection frame;

[0033] Figure 4 This is a schematic diagram of the automatic labeling device for a face detection model.

[0034] Figure 1-4 The system consists of: 1. First facial feature detection box; 2. Second facial feature detection box; 10. Face image acquisition module; 11. Detection and processing module; 12. Calculation module; 13. Determination module. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Please refer to Figure 1 This application provides an automatic labeling method for a face detection model, characterized by the following steps:

[0037] S01: Acquire a face image and collect the boundary points of facial features;

[0038] It should be noted that facial images can be obtained through facial photo registration, by reading facial images from locally stored images, or by real-time capture using camera equipment. Camera equipment refers to devices capable of acquiring image data in real time, such as webcams, cameras, camcorders, or video cameras. The facial images can be photographs of the same person at different times or in different attire; they can also be facial images of people of different ages and genders.

[0039] S02: Connect the facial feature boundary points of the face image to form a facial feature boundary line, and then expand the facial feature boundary line to obtain the first facial feature detection box (1);

[0040] It should be noted that while the ResNet50 network is used to collect 106 key points to form the facial feature boundary points, other networks can also be used. The number of facial points collected is not limited to this; 240, 1000, or even more points can be collected. Collecting more key points will result in a more accurate face detection model.

[0041] It should be noted that, as Figure 2 As shown, the process of obtaining the second facial feature detection box (2) is illustrated by taking the eyes as an example. First, the boundary points around the eyes of the face image are connected to form the facial feature boundary line. The facial feature boundary line can be an actual connection or a virtual connection (not actually connected). The facial feature boundary line is expanded to obtain the first facial feature detection box (1). The expansion can be a non-uniform expansion of each part of the boundary line; or it can be a uniform expansion, such as 0.1 times, 0.11 times, 0.12 times, etc. The specific multiple can be determined by those skilled in the art based on the actual situation.

[0042] It should be noted that the enlarged first facial feature detection box (1) is to leave enough space for subsequent adjustments to the first facial feature detection box (1).

[0043] S03: Measure the facial pose angle and calculate the adjustment amount of the point coordinates on the first facial feature detection box (1) based on the facial pose angle;

[0044] It should be noted that facial poses can be upright or irregular; the degree of irregularity affects the applicability of face detection models. The industry typically uses pitch, yaw, and roll angles (represented by Euler angles) to represent the uprightness of facial poses. Please refer to [link to relevant documentation]. Figure 3 , Figure 3This is a schematic diagram of a face coordinate system in an embodiment of this application. The origin is the center or center of gravity of the human head. The direction from one ear to the other is the X-axis, the direction from the top of the head to the neck is the Y-axis, and the direction from the face to the back of the head is the Z-axis. Euler angles include three angles: pitch, yaw, and roll. The pitch angle θ is the angle of rotation around the X-axis, the yaw angle ψ is the angle of rotation around the Y-axis, and the roll angle Φ is the angle of rotation around the Z-axis.

[0045] It should be further explained that the pitch, yaw, and roll angles of the face pose are obtained by combining the solvePnP method of OpenCV with the Euler angle calculation formula; furthermore, the pitch, yaw, and roll angles of the face pose can be obtained by other calculation methods such as data fitting.

[0046] S04: Adjust the coordinates of the first facial feature detection box (1) according to the adjustment amount to obtain a more accurate second facial feature detection box (2);

[0047] Because of incorrect sitting or standing postures, the resulting facial images are not always straight and frontal, leading to inaccurate acquisition of facial feature points and the first face detection bounding box. To obtain a more accurate facial feature detection bounding box, the first face detection bounding box needs to be adjusted. For example, if the face tilts too much, adjust it upwards; if the face tilts too much, adjust it downwards.

[0048] Furthermore, the specific adjustment method of this embodiment is as follows:

[0049] Assuming the coordinates of any point on the first facial feature detection box (1) are (x, y, z), the pitch angle of the face is θ, the yaw angle is ψ, and the roll angle is Φ, and the coordinates of any point on the second facial feature detection box (2) are (x′, y′, z′), then

[0050] x′=x+Δx;

[0051] y′=y+Δy;

[0052] z′=z+Δz;

[0053] Where Δx, Δy, and Δz are the adjustment amounts for the three coordinate axes x, y, and z, respectively.

[0054] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll);

[0055] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll);

[0056] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll);

[0057] a1, a2, and a3 are the weights of the pitch angle θ relative to the x, y, and z axes, respectively; they can be positive or negative weights. b1, b2, and b3 are the weights of the yaw angle ψ relative to the x, y, and z axes, respectively; they can be positive or negative weights. c1, c2, and c3 are the weights of the roll angle Φ relative to the x, y, and z axes, respectively; they can be positive or negative weights.

[0058] Furthermore, appropriate thresholds are set for the face pose angles: θ(pitch)[-τ°, τ°], ψ(yaw)[-θ°, θ°], Φ(roll)[-ε°, ε°]. In order to further improve the training capability of the multi-face model, the coordinate adjustment amount is corrected by dividing the above angles into several segments.

[0059] For example, θ(pitch)[-90°, 90°] will be divided into three segments: θ(pitch)[-45°, 45°], θ(pitch)[-90°, -45°], and θ(pitch)[45°, 90°].

[0060] If the pitch angle is in the range θ(pitch)[-45°, 45°], the adjustment amounts for the three coordinates Δx, Δy, and Δz are:

[0061] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll);

[0062] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll);

[0063] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll);

[0064] If the pitch angle is in the range θ(pitch)[-90°, -45°] or θ(pitch)[45°, 90°], and a correction function d*cosθ(pitch) is introduced, then Δx, Δy, and Δz are respectively:

[0065] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll)+d1*cosθ(pitch);

[0066] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll)+d2*cosθ(pitch);

[0067] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll)+d3*cosθ(pitch);

[0068] Where d1, d2, and d3 are all pitch angle function correction coefficients.

[0069] For example, a yaw angle in the range of ψ(yaw)[-90°, 90°] will be divided into three segments: ψ(yaw)[-45°, 45°], ψ(yaw)[-90°, -45°], and ψ(yaw)[45°, 90°].

[0070] If the yaw angle is within ψ(yaw) [-45°, 45°], the adjustment amounts for the three coordinates Δx, Δy, and Δz are:

[0071] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll);

[0072] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll);

[0073] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll);

[0074] If the yaw angle is in the range of ψ(yaw)[-90°, -45°] or ψ(yaw)[45°, 90°], a correction function e*cosψ(yaw) is introduced.

[0075] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll)+e1*cosψ(yaw);

[0076] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll)+e2*coSψ(yaw);

[0077] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll)+e3*coSψ(yaw);

[0078] Where e1, e2, and e3 are all yaw angle function correction coefficients.

[0079] For example, a roll angle of Φ(roll)[-90°, 90°] will be divided into three segments: Φ(roll)[-45°, 45°], Φ(roll)[-90°, -45°], and Φ(roll)[45°, 90°].

[0080] If the roll angle is within Φ(roll) [-45°, 45°], then the adjustment amounts for the three coordinates Δx, Δy, and Δz are:

[0081] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll);

[0082] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll);

[0083] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll);

[0084] If the roll angle is in the range of Φ(roll)[-90°, -45°] or Φ(roll)[45°, 90°], a correction function f*cosΦ(roll) is introduced, and the adjustment amounts for the three coordinates Δx, Δy, and Δz are:

[0085] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll)+f1*cosΦ(roll);

[0086] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll)+f2*cosΦ(roll);

[0087] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll)+f3*cosΦ(roll);

[0088] Where f1, f2, and f3 are the roll angle function correction coefficients;

[0089] It should be noted that the adjustments for Δx, Δy, and Δz can be made by selecting the increase or decrease correction function based on the actual angles of the face's pitch, yaw, and roll angles. For example, if the pitch angle is in θ (pitch) [-90°, -45°], the yaw angle is in ψ (yaw) [-45°, 45°], and the roll angle is in Φ (roll) [45°, 90°], then the adjustments for the three coordinates Δx, Δy, and Δz are:

[0090] Δx=a1*θ(pitch)+b1*ψ(yaw)+c1*Φ(roll)+d1*cosθ(pitch)+f1*cosΦ(roll);

[0091] Δy=a2*θ(pitch)+b2*ψ(yaw)+c2*Φ(roll)+d2*cosθ(pitch)+f2*cosΦ(roll);

[0092] Δz=a3*θ(pitch)+b3*ψ(yaw)+c3*Φ(roll)+d3*cosθ(pitch)+f3*cosΦ(roll);

[0093] According to the above formula, the selected points on the first facial feature detection box (1) are traversed and adjusted. The coordinates of the selected points are adjusted according to the above formula to obtain new points and connect them to obtain a more accurate second facial feature detection box (2).

[0094] Other angles of facial pose will not be listed here, and those skilled in the art can adjust the corrections according to the actual situation.

[0095] Furthermore, pitch, yaw, and roll angles are generally applicable within a certain range. In this embodiment, it is necessary to filter out face images with large attitude angles to improve the recognition accuracy of the face recognition system; for example, pitch angles greater than 90 degrees need to be filtered out. This is because face images with large attitude angles represent large-angle side views of the face, with fewer feature points, resulting in a higher probability of recognition failure. Therefore, by filtering out face images with large attitude angles, the problem of decreased face angle recognition accuracy caused by the failure to recognize facial key points at large attitude angles can be avoided.

[0096] It should be noted that, alternatively, those skilled in the art can also directly obtain the second facial feature detection box (2) line by connecting the offset key points. The specific offset method can be the same as described above in this application.

[0097] S05: Repeat steps S01-S04 to obtain multiple second facial feature detection boxes (2), and generate a label file based on the multiple second facial feature detection boxes (2) to form a training set;

[0098] It should be noted that in order to obtain enough training data, it is necessary to repeat steps S01-S04 to obtain multiple second facial feature detection boxes (2), the number of which can be 1000, 10000 or 100000, etc., and then aggregate multiple second facial feature detection boxes (2) to form a label file and thus form a training set.

[0099] S06: Train a face detection model based on the training set.

[0100] Based on the above methods, such as Figure 4 As shown, this application also provides an automatic labeling device for a face detection model, comprising:

[0101] The face image acquisition module (10) is used to acquire face images and collect the boundary points of facial features;

[0102] The detection processing module (11) is used to connect the facial feature boundary points of the face image to form a facial feature boundary line, and then expand the facial feature boundary line to obtain a first facial feature detection box (1); it is used to adjust the first facial feature detection box (1) according to the coordinate adjustment amount of the points on the first facial feature detection box (1) to obtain a more accurate second facial feature detection box (2).

[0103] The calculation module (12) is used to measure and calculate the face pose angle and the coordinate adjustment of the points on the first facial feature detection box (1);

[0104] The determination module (13) is used to train a face detection model based on the training set; wherein, the training set is completed by generating a label file using multiple second facial feature detection boxes (2).

[0105] Furthermore, this application also provides a computer-readable storage medium storing instruction / program data. When executed, this instruction / program data implements the method provided by any embodiment of the applicant's face recognition method and any non-conflicting combination thereof. The instruction / program data can be formed into a program file and stored in the aforementioned storage medium as a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0106] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0107] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An automatic labeling method of a face detection model, characterized by, The method comprises the following steps: S01: acquiring a face image, and collecting face key points as facial feature boundary points; S02: connecting the facial feature boundary points of the face image to form facial feature boundary lines, and then expanding the facial feature boundary lines to obtain a first facial feature detection frame; S03: measuring a face posture angle, and calculating an adjustment amount of point coordinates on the first facial feature detection frame according to the face posture angle; S04: adjusting the first facial feature detection frame according to the adjustment amount to obtain a more accurate second facial feature detection frame; S05: repeating steps S01-S04 to obtain a plurality of second facial feature detection frames, and generating a mark file according to the plurality of second facial feature detection frames to complete a training set; S06: training to obtain a face detection model according to the training set. 2.The method of claim 1, wherein, The facial feature boundary lines are expanded by 0.1 times to the first facial feature detection frame. 3.The method of claim 1, wherein, The facial feature boundary points are obtained by using a ResNet50 face detection 106-point model. 4.The method of Claim 1, wherein, The face posture angle comprises a pitch angle, a yaw angle and a roll angle.

5. The method of claim 4, wherein the automatic labeling of the face detection model is performed by a machine learning algorithm. The pitch angle, the yaw angle and the roll angle are obtained by using a solvePnP method of opencv and an Euler angle calculation formula.

6. An apparatus for automatically labeling a face detection model, the apparatus comprising: The method comprises the following steps: A face image acquisition module (10) is configured to acquire a face image and collect face key points as facial feature boundary points; A detection processing module (11) is configured to connect the facial feature boundary points of the face image to form facial feature boundary lines, and then expand the facial feature boundary lines to obtain a first facial feature detection frame; and adjust the first facial feature detection frame according to an adjustment amount of point coordinates on the first facial feature detection frame to obtain a more accurate second facial feature detection frame; A calculation module (12) is configured to measure and calculate a face posture angle and an adjustment amount of point coordinates on the first facial feature detection frame; A determination module (13) is configured to train to obtain a face detection model according to a training set; wherein a mark file is generated according to a plurality of second facial feature detection frames to complete the training set.

7. The apparatus for automatic labeling of a face detection model according to claim 6, wherein, The facial feature boundary lines are expanded by 0.1 times to the first facial feature detection frame.

8. The apparatus for automatic labeling of a face detection model according to claim 6, wherein, The facial feature boundary points are obtained by using a ResNet50 face detection 106-point model.

9. A face recognition system, characterized by, An automatic marking device comprising the face detection model according to any one of claims 6-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program can execute the automatic marking method of the face detection model according to any one of claims 1-5.

Images