A face key point labeling method

Abstract

A face key point labeling method, characterized in that, comprising the following steps: S11: five key points are labeled on small sample data selected from a face image data set; S12: pixel-level positioning and labeling are performed on small sample data of different sizes by using inference of a multi-task model, and a first labeling model is formed; S13: labeling effect of the small sample data is checked, and parameters in the previous step are adjusted until the labeling effect reaches an expectation; S14: a second labeling model is obtained by training the first labeling model by using large sample data selected from the face image data set; S15: the second labeling model is used to pre-label the face image data set; and S16: supplementary labeling is performed on the pre-labeling to improve the labeling effect. The effect of a small model can also be obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of face detection, in particular, to a face key point labeling method. BACKGROUND

[0002] Traditional face recognition technology is mainly based on visible light image face recognition, which is also the familiar recognition method and has a history of 30 years of research and development. However, this method has a difficult-to-overcome defect, especially when the environmental light changes, the recognition effect will decrease sharply, which cannot meet the needs of actual systems.

[0003] A rapidly developed solution in recent years is a multi-light source face recognition technology based on active near-infrared images. It can overcome the influence of light changes and has achieved excellent recognition performance. The overall system performance in terms of precision, stability and speed has reached the requirements of practicality. It is this technology that has developed rapidly in the past two years, making face recognition technology gradually practical.

[0004] Face recognition technology can be divided into two technical routes from the application. One is to take the traditional single RGB image recognition as the core, to obtain the corresponding technical features in the image through the analysis and processing of the image data, and to recognize and judge, so as to mark and provide the face. In this way, some models can better obtain face feature information, but the recognition effect is poor for background interference, large-angle face and the like. The other is to process a variety of data such as living body detection and face quality detection, and confirm the face information through a series of data change degrees. In this way, the face data and features can be better obtained, and the security is very high, so it is widely used in face payment and the like. However, this technology has a large amount of data processing and a long cycle, and is difficult to apply to mass data processing.

[0005] At present, face key point positioning models can be mainly divided into traditional methods based on manual features and regression methods based on deep learning. Traditional methods mainly include AAMs, CLMs, ESR, SDM and the like, which mainly fit the positions of face key points through manually designed features. In recent years, with the rapid development of deep learning methods represented by convolutional neural networks and recurrent neural networks, face key point positioning algorithms have also rapidly shifted to deep learning-based methods. Due to the powerful feature extraction capability of deep neural networks, deep learning-based face key point positioning algorithms have rapidly surpassed traditional methods in evaluation indicators such as NME and FR.

[0006] Deep learning algorithm needs a large amount of data support, and the evaluation of an algorithm is meaningless without a large amount of effective data support. The result quality of the algorithm depends entirely on the fitting degree with the real data. If there is not enough data support and verification, the algorithm often does not have good generalization performance. In order to ensure that the algorithm has good robustness, a large amount of data is needed to verify the model. SUMMARY

[0007] Therefore, the present application labels key points on small sample data for multi-task learning, and the finally formed labeled data can be used for training of other models, especially various small models. Since the data labeling is iterated for many times, the effect of the small model can also be obviously improved.

[0008] The present application provides a face key point labeling method, characterized in that it comprises the following steps:

[0009] S11: five key points are labeled on small sample data selected from a face image data set; wherein the five key points include two left and right corners of the mouth, the centers of two eyes and a nose; the face image data set includes front face photos and side face photos;

[0010] S12: pixel-level positioning and labeling are performed on small sample data of different sizes by using inference of a multi-task model, and a first labeling model is formed; wherein the multi-task learning includes face frame position detection function, face frame confidence classification function and face key point regression function, wherein the face key point can be further used for face angle calculation;

[0011] S13: the labeling effect of the small sample data is checked, and the parameters in the previous step are adjusted until the labeling effect reaches the expectation;

[0012] S14: a second labeling model is obtained by training the first labeling model by using large sample data selected from the face image data set;

[0013] S15: the second labeling model is used for pre-labeling of the face image data set;

[0014] S16: supplementary labeling is performed on the pre-labeling to improve the labeling effect.

[0015] Optionally, the face key point labeling method is characterized in that for the side face photos, auxiliary key points, i.e. ears, are added.

[0016] Optionally, the face key point labeling method is characterized in that when the five key points are labeled on the small sample data selected from the face image data set, a manual selection method is adopted.

[0017] Optionally, the face key point labeling method has the feature that when the small sample data in the face image data set is selected to label five key points, a method of automatic selection first and manual adjustment later is adopted.

[0018] Optionally, the face key point labeling method has the feature that the images in the face image data set are depth images.

[0019] Optionally, the face key point labeling method has the feature that the RGB image and the depth data corresponding to the RGBD image are respectively stored.

[0020] Optionally, the face key point labeling method has the feature that the first labeling model and the second labeling model are obtained by learning from depth images.

[0021] Optionally, the face key point labeling method has the feature that the first labeling model and the second labeling model are respectively obtained by learning from the RGB image and the depth data, and the first labeling model and the second labeling model are finally obtained by fusion according to the correspondence between the RGB image and the depth data.

[0022] Optionally, the face key point labeling method has the feature that the first labeling model and the second labeling model also identify a face contour line for face angle calculation.

[0023] Optionally, the face key point labeling method has the feature that:

[0024] The first labeling model and the second labeling model adopt a feature pyramid mode to obtain.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] The present application labels key points by using small sample data, trains a model, improves the accuracy of samples, and fundamentally guarantees the accuracy of the model. The model obtained by using small sample data can be applied to the training of other models, especially various small models, and improves the application range of the model. The data labeling of the present application is iterated for multiple times, and the effect of the small model can also be obviously improved. The present application classifies faces and provides a new dimension for labeling, which is helpful to improve the labeling accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim at the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort on the basis of the provided drawings. Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-restrictive embodiments with reference to the accompanying drawings:

[0028] Figure 1 A flow chart of a face recognition method in an embodiment of the present application;

[0029] Figure 2 A flow chart of a face key point labeling method in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a face labeling method in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of manually selecting and adjusting a single labeling point in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of manually selecting a single labeling point and adjusting it as a whole in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of selecting a labeling point in an automatic labeling manner in an embodiment of the present application;

[0034] Figure 7 A light spot used in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made. These all belong to the protection scope of the present application.

[0036] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is only to distinguish the comparable objects and not necessarily for describing a particular sequential or chronological order. Unless otherwise defined, all terms used herein have the same meaning as they would have to a person skilled in the art pertaining to the present application. It will be further understood that any definitions of terms in this background section or elsewhere herein are intended to apply cumulatively to the description and claims unless otherwise indicated. It is also to be understood that the description and the examples, while indicating embodiments of the application, are given by way of illustration only, since various changes and modifications within the spirit and scope of the application will become apparent to the skilled in the art from this detailed description. Further, the term "comprising" and the like does not exclude the presence of elements or steps other than those listed in a process, method, system, product or apparatus which comprises the steps. The word "comprising" and the like does not exclude the presence of additional elements or steps than those listed in a process, method, system, product or apparatus which comprises the steps. Furthermore, the words "a" or "an" shall not exclude the presence of zero, one, or more than one element, irrespective of whether the reference is in the singular or plural.

[0037] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described in some examples.

[0038] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described in some examples.

[0039] Figure 1 A flow chart of face recognition in an embodiment of the present application is shown. A face recognition system usually needs to go through the following steps:

[0040] 1. Face detection, i.e. finding the location of faces in an image, if there are faces, return the bounding box containing each face.

[0041] 2. Face alignment, the goal of face alignment is to scale and crop the face image using a set of reference points located at fixed positions in the image.

[0042] 3. Face representation, in the face representation stage, the pixel values of the face image are converted into a compact and discriminative feature vector, ideally, all faces of the same subject should be mapped to similar feature vectors.

[0043] 4. Live detection, i.e. judging whether the captured face is a real face or a fake face attack, such as a face image printed using color paper, a mask, etc. non-real face image.

[0044] 5. Face quality detection, i.e. detecting face images with occlusions, blurring, improper lighting.

[0045] 6. Face recognition, identity recognition of face images, comparing the face to be recognized with known faces in the database, and obtaining related information.

[0046] From the above steps, it can be seen that face labeling runs through the entire process of face recognition and is of great significance to the process of face recognition. On the other hand, the aspects concerned in the above processes are also the problems encountered in face labeling, and therefore the above problems need to be fully considered when establishing a labeling model.

[0047] Figure 2 A flowchart of a face key point labeling method in an embodiment of the application is shown. The face key point labeling method comprises the following steps:

[0048] S11: Label five key points in the small sample data selected in the face image data set.

[0049] In this step, the five key points include two left and right corners of the mouth, the centers of two eyes and the nose; the five key points belong to the internal key points of the face, and according to them, the pose of the face can be calculated. Of course, there are also schemes of labeling four points and six points in the early stage, but the scheme of five key points is adopted in this embodiment. It should be noted that a key point needs to have multiple labeling points. In this embodiment, 106 labeling points are used to label the internal key points and contour key points of the face, the internal key points include eyebrows, eyes, nose and mouth, a total of 73 labeling points, and the contour key points include 33 labeling points. The face image data set includes front face photos and side face photos.

[0050] In some embodiments, for the side face photos, auxiliary key points, i.e., ears, are also added. In the side face photos, generally only one ear is present. For example, in the right face photos, the right ear is often present, and in the left face photos, the left ear is often present. However, since the ear is not always clearly present in the photo, it cannot be used as a main key point. In the side face photos, according to the different angles of the face, the eye and the corner of the mouth on one side are compressed to different degrees, which causes the size and angle of the image to change, increasing the difficulty of recognition. At the same time, the ear on the other side is more likely to be displayed and occupies more space in the image, and therefore can be used as an auxiliary key point for recognizing and judging the side face photos. In this embodiment, the ear is labeled by using an additional 26 labeling points, i.e., a total of 132 labeling points, so that comprehensive recognition and labeling of the face can be achieved.

[0051] S12: Use the inference of the multi-task model to perform pixel-level positioning and labeling on the small sample data of different sizes, and form a first labeling model.

[0052] In this step, the deep neural network directly learns a vector regression function to infer the shape of the entire face through the feature data of the labeled data in the previous step, and finally realizes the function of autonomous recognition. The multi-task learning includes face frame position detection function, face frame confidence classification function, and face key point regression function, wherein the face key point can be further used for face angle calculation. The face frame position detection function refers to that the algorithm outputs four parameters of a frame: upper left corner coordinates (x, y), frame width and height (w, h), which can locate the position of the face in a larger picture, so as to perform separate extraction, enlargement and other operations on the face. The face frame confidence classification refers to that the algorithm obtains a confidence value conf, which is greater than a pre-set threshold value for positive and less than the threshold value for negative, which can be used to judge whether it is a face. The face key point regression refers to the regression of the coordinates (x, y) of each key point, which can be used to judge the face area.

[0053] The first labeling model also identifies the face contour line, which is used for face angle calculation together with the key point information. When the lens is directly opposite the face, the key points and the face contour line both show a basic symmetrical feature, and when the lens has a certain angle with the face, the key points and the face contour line will both appear a certain deformation. For example, when the lens is located on the right side of the face, the contour line of the right face is relatively flat, while the contour line of the left face is relatively steep, and the horizontal distance between the left eye key point and the nose key point is less than the horizontal distance between the right eye key point and the nose key point. When the angle of the angle with the lens is greater than a pre-set value, the auxiliary key point is started to assist in judgment.

[0054] This step adopts a single-stage face detection framework and uses a multi-task loss of strong supervision and self-supervised signals, and adopts a dense face positioning method. This embodiment adopts a multi-task learning strategy to simultaneously predict a face score, a face frame, five face key points, and a three-dimensional position and corresponding relationship of each face pixel, and can process multiple face problems.

[0055] S13: Check the labeling effect of the small sample data, adjust the parameters in the previous step until the labeling effect reaches the expectation.

[0056] In this step, the effect of the first labeling model is tested, and when the effect does not reach the expectation, it returns to step S12 to modify the parameters and re-train. When the effect reaches the expectation, step S14 is performed. When judging the effect of the model, the overall labeling result can be used for judgment, or part of the data can be used for judgment, such as a side face.

[0057] S14: Training the first labeling model to obtain a second labeling model by using large sample data selected from the face image data set.

[0058] In this step, the large sample data is a sample set with a larger sample size than the small sample data. Through training with a larger amount of image data, the model can be better corrected, making it have better universality for the model.

[0059] S15: pre-labeling the face image data set by using the second labeling model;

[0060] In this step, the labeling efficiency is improved through pre-labeling technology. Pre-labeling technology refers to using automatic labeling tools to perform simple preprocessing on data, making it easier and more efficient to label, thereby improving the accuracy and speed of labeling. Specifically, when labeling images, only a few points are needed to generate automatic labeling of the entire surface, which will greatly improve the labeling efficiency.

[0061] The second labeling model also identifies the face contour line, which is used for face angle calculation together with the key point information. When the lens is directly opposite the face, the key points and the face contour line both exhibit a basic symmetry feature, while when the lens has a certain angle with the face, the key points and the face contour line will both appear a certain deformation. For example, when the lens is below the face, the lower part of the contour line is almost the same as when it is opposite, while the upper part of the contour line is shortened on the image, and the labeling points are more dense; the distance between the nose key point and the mouth key point is less than the distance between the two when the face is straight. When the angle of the lens is greater than a preset value, auxiliary key points are started to assist in judgment. In the judgment, the proportional relationship of different organs of the face can also be used for auxiliary judgment.

[0062] S16: supplementing the pre-labeling to improve the labeling effect.

[0063] In this step, the pre-labeling in the previous step is supplemented, making the labeling more complete and the labeling effect more explicit, so that the image labeling function can be better completed.

[0064] Figure 3This illustration shows a face annotation method according to an embodiment of the present invention. This embodiment employs a single-stage approach, combining Feature Pyramid Networks (FPNs) with independent context modeling, and calculates the loss for each positive and negative anchor box. Specifically, our algorithm incorporates a feature pyramid and adds independent context models to the five pyramid feature maps to improve modeling capabilities; for multiple positive samples, it uses the same loss head and different feature maps to calculate multiple losses; it uses a specific ratio of anchors at the feature pyramid levels from P2 to P6; and it uses OHEM to address the imbalance between positive and negative anchor boxes. Context Modeling enhances the model's context module inference capabilities to capture small faces; SSH and PyramidBox expand the receptive field of the Euclidean mesh on the feature pyramid using context modules.

[0065] In step S11, as Figure 4 As shown, in some embodiments, annotation points are manually selected for the displayed image, and these annotation points can be adjusted by dragging key points one by one; for example... Figure 5 As shown, in some embodiments, annotation points are manually selected for the displayed image, and multiple annotation points for the same key point can be dragged as a whole; for example... Figure 6 As shown, in some embodiments, annotation points are selected for the displayed image using an automated annotation method. During operation, a point (eye, nose, mouth) is first selected, and the program selects the corresponding point set (eye, nose, mouth) and adjusts the position of the entire point set according to the corresponding position given by the network.

[0066] Automated annotation requires the use of a multi-task loss function:

[0067] For any trained anchor i, minimize the multi-task loss function:

[0068]

[0069] (1) Face classification p here i Let anchor i be the predicted probability of a face. 1 represents a positive anchor, and 0 represents a negative anchor. (Classification) This refers to the softmax loss in the binary classification case (face / non-face). In some embodiments, p i The calculation function also includes the face angle parameter ang. i and ear parametersi .

[0070] (2) Face bounding box regression loss Here t i = {t x , t y , t w , t h}, represent the coordinates of the predicted bounding box and the ground-truth box associated with the positive anchor, respectively. The regression targets (center coordinates, width and height) are normalized using Here R is the Robust loss function (smooth-L1)

[0071] (3) Face landmark regression loss Here represent the predicted five face landmarks and the ground-truth. The regression of the five face landmarks also adopts anchor center based target normalization.

[0072] (4) Dense regression loss L pixel The loss adjustment parameters λ1-λ3 are set to 0.25, 0.1 and 0.01, which means that we increase the importance of the bounding box and the key point positioning in the supervision signal.

[0073]

[0074] In some embodiments, the images in the face image dataset are depth images. Depth images, also known as range images, are images in which the distance (depth) values of each point in the scene captured by the image collector are taken as pixel values. They directly reflect the geometric shape of the visible surface of the scene, so depth images have more data information than traditional RGB images, and their recognition is more accurate. Therefore, among the above-mentioned various parameters, those with position information need to increase their depth parameters, so that the data is converted from two-dimensional to three-dimensional. When labeling key points, each key point should be as close as possible to the actual point cloud of the labeled target, and the fixed key point labeling position should not have errors, and the key point position labeling qualified rate should be no less than 97%. The methods for obtaining depth images include laser radar depth imaging, computer stereo vision imaging, coordinate measuring machine method, Moire fringe method, and structured light method, etc. Figure 7 A kind of light spot used in the embodiment of the application is shown. The embodiment does not limit the method for obtaining depth images. Correspondingly, the first labeling model and the second labeling model are learned and obtained according to the depth images. The training method is the same as before, which will not be repeated here.

[0075] In some embodiments, the images in the face image dataset respectively store an RGB image and depth data corresponding to the RGBD image. When the depth camera acquires an image, the directly obtained data is an RGB image and depth data, and a depth image is generated by fusing the RGB image and the depth data after registration. In some application scenarios, a depth image does not need to be generated, and therefore it is necessary to train separately on the RGB image and the depth data. Correspondingly, the first annotation model and the second annotation model are respectively learned from the RGB image and the depth data, and the first annotation model and the second annotation model are fused according to the correspondence between the RGB image and the depth data to obtain the final first annotation model and the final second annotation model. Since the RGB image and the depth data are registered data, the pixel points have a one-to-one correspondence, and therefore the first annotation model and the second annotation model have a very good fusion basis, which can greatly improve the fusion efficiency of the model, and at the same time, the face marking can be performed before the depth image is acquired, and the amount of marking is greatly reduced, thereby greatly improving the efficiency of face marking.

[0076] The embodiment of the present application also provides a computer readable storage medium for storing a program, characterized in that the program is executed to realize the face key point annotation method steps described above. The computer readable storage medium includes a program product for realizing the method, which can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of the present application is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or apparatus.

[0077] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0078] The computer readable storage medium can include a computer-readable medium in baseband or propagated as a carrier wave in a propagated signal, wherein the propagated signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. Computer readable storage medium can be any media readable by a computer and / or a processor that causes a computer or processor to execute a program of instructions to perform a particular function. Computer readable storage medium can be any available media that can be accessed by a general purpose or special purpose computer including, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, and any suitable combination thereof. Computer readable storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer readable storage medium includes, but is not limited to, RAM, ROM, EEPROM, solid state drives (SSDs), flash memory, phase-change memory ("PCM"), and / or other technology.

[0079] Program code used by or in connection with the described embodiments can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0080] The labeling method of the present application can be used not only for labeling of face RGB images, but also for labeling of face depth images, and can be used for calibration of face depth data, i.e., can be used for labeling of all face images. For example, in common face labeling, the accuracy of the model is continuously improved through optimization of the model, but existing public face data sets are all planar images, and real scene is more complex, thus leading to a bottleneck that is difficult to break through in the prior art. For another example, in the prior art, a face nose tip point is recognized through a 3D image, and a face key point is determined according to a preset face model, but the labeling method of the present application labels according to the characteristics of the image data itself, can better reflect the characteristics of different faces, and has better discrimination.

[0081] Alternatively, in the depth image marking of the human face, both the depth image can be directly recognized, and the RGB image and the depth data can be respectively recognized, so that the method has a better application range. For example, in the scene where the depth camera needs to generate a depth image, such as face payment, face unlocking and other application scenarios, the human face can be directly marked and recognized according to the depth image. For another example, in monitoring, early warning and other places, it can not be necessary to directly display the depth image, or it can be necessary to respond more quickly to the monitoring situation, the RGB image and the depth data can be respectively recognized, the combined marking can be generated, and then the human face can be recognized, so as to improve the efficiency.

[0082] The marking method of the present application can train a human face marking model with stronger adaptability, and at the same time, due to the higher accuracy of artificial marking, the quality of human face marking is also significantly improved. At the same time, the marking method of the present application introduces the human face angle parameter, so that the marking of the human face is more comprehensive, the auxiliary key points are increased when the angle is large, and the accuracy of the marking is improved, so that the subsequent human face recognition process is more accurate.

[0083] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0084] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the above specific embodiments, and various modifications or changes can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application.

Claims

1. A face key point labeling method, characterized in that, The method comprises the following steps: S11: labeling five key points in small sample data selected from a face image dataset; wherein the five key points include two left and right corners of the mouth, the centers of two eyes, and a nose; the face image dataset includes front face photos and side face photos; S12: using inference of a multi-task model to perform pixel-level positioning and labeling on the small sample data of different sizes, and forming a first labeling model; wherein the multi-task model comprises a face frame position detection function, a face frame confidence classification function, and a face key point regression function, wherein the face key points can be further used for face angle calculation; S13: checking the labeling effect of the small sample data, adjusting the parameters in the previous step until the labeling effect reaches the expectation; S14: training the first labeling model using large sample data selected from the face image dataset to obtain a second labeling model; S15: pre-labeling the face image dataset using the second labeling model; S16: supplementing the pre-labeling to improve the labeling effect.

2. The method of claim 1, wherein, For the side face photos, auxiliary key points, i.e., ears, are added.

3. The method of claim 1, wherein, When labeling the five key points in the small sample data selected from the face image dataset, a manual selection method is adopted.

4. The method of claim 1, wherein, When labeling the five key points in the small sample data selected from the face image dataset, a method of first automatic selection and then manual adjustment is adopted.

5. The method of claim 1, wherein, The images in the face image dataset are depth images.

6. The method of claim 5, wherein, The RGB image and depth data corresponding to the RGBD image are respectively stored.

7. The method of claim 5, wherein, The first labeling model and the second labeling model are obtained by learning from depth images.

8. The method of claim 6, wherein, The first labeling model and the second labeling model are respectively obtained by learning from the RGB image and the depth data, and the final first labeling model and second labeling model are obtained by fusion according to the corresponding relationship between the RGB image and the depth data.

9. The method of claim 1, wherein, The first labeling model and the second labeling model also identify face contour lines for face angle calculation.

10. The face key point labeling method according to claim 1, characterized in that: The first labeling model and the second labeling model are obtained in a feature pyramid manner.

Images