Model training method, device, electronic device and storage medium

Abstract

The present application provides a model training method, including: obtaining a first face sample face image and a second sample face image; adjusting the key point prediction model according to the first predicted position information and key point position annotation information to obtain an adjusted face key point prediction model; adjusting the second sample face image according to the second predicted position information, and inputting the adjusted second sample face image into a face recognition model to obtain an identity recognition result corresponding to the second sample face image; adjusting the face recognition model according to the identity recognition result and the identity annotation information; looping through the above steps, using the adjusted face key point prediction model obtained in the previous loop as the face key point prediction model for the current loop in each loop, until the face recognition model meets the training requirements. The model training method provided by the present application can improve the accuracy of face recognition results.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to a model training method. This application also relates to a model training device, an electronic device, and a storage medium. This application also relates to a face recognition method, device, electronic device, and storage medium. Background Art

[0002] Face recognition is a biometric recognition technology that is widely used in various fields of life and has become the mainstream technology for identity recognition.

[0003] Face recognition tasks fall into two categories: 1:1 face verification and 1:N face retrieval. 1:1 face verification determines whether two input images are of the same person. 1:N face retrieval involves searching an input face image against a face database to determine whether the image is in the database and which person it is.

[0004] The process of face recognition is generally as follows: face image acquisition, face detection, key point location, face calibration, and face feature extraction and comparison. The specific process is: first, for an image, detect whether it contains a face. If a face is detected, perform key point detection, and then perform a similarity transformation on the face image based on the detected key points to obtain a calibrated image. The calibrated face image is then input into the face recognition model to extract features. Depending on the specific situation, if it is a 1:1 face verification, the similarity between the two extracted features is directly calculated. If it is a 1:N face retrieval, the extracted face features need to be compared with all the face features stored in the database, and the one with the highest similarity is found. Then, a pre-set threshold is used to determine whether it is the person.

[0005] Existing facial recognition technology has become very mature after years of development, but when faced with low-quality facial images, it is prone to misidentification and has a low accuracy rate. Summary of the Invention

[0006] The present application provides a model training, device, electronic device and storage medium to improve the accuracy of face recognition.

[0007] This application provides a model training method, including:

[0008] Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information;

[0009] Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information;

[0010] adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model;

[0011] The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0012] Optionally, obtaining the key point position prediction information of the sample facial image includes: respectively obtaining the key point position prediction information of each facial key point in the sample facial image, and obtaining an error evaluation value; the error evaluation value is a predicted value of a normalized average of errors between the key point position prediction information of each facial key point in the same sample facial image and the key point actual position information of the corresponding facial key point;

[0013] The training and adjustment of the key point prediction model based on the key point position prediction information and the key point position labeling information of the first sample face image includes: training and adjusting the key point prediction model based on the key point position prediction information of each facial key point in the first sample face image, the key point position labeling information of each facial key point in the first sample face image, and the error evaluation value of the first sample face image.

[0014] Optionally, the training and adjusting the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, and the error evaluation value of the first sample facial image includes:

[0015] obtaining, based on the key point position prediction information of each facial key point in the first sample facial image, a difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point as an actual error;

[0016] Adjust the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image and the actual error; the adjustment goals include making the key point position annotation information of each facial key point in the first sample facial image consistent with the key point position annotation information of the corresponding key point, and making the error evaluation value of the first sample facial image consistent with the normalized average of the actual error.

[0017] Optionally, the normalized average value of the actual error is calculated as follows:

[0018] Obtaining first position coordinates corresponding to each facial key point in the first sample facial image based on key point position prediction information of each facial key point in the first sample facial image, and obtaining second position coordinates corresponding to each facial key point in the first sample facial image based on key point position annotation information of each facial key point in the first sample facial image;

[0019] Obtaining the sum of distances between first position coordinates corresponding to each facial key point in the first sample facial image and second position coordinates of the corresponding facial key point;

[0020] Obtain the length and width of the annotation box corresponding to the key point position annotation information;

[0021] A normalized average value of the actual error is obtained according to the distance sum, the length value, and the width value.

[0022] Optionally, adjusting the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error includes:

[0023] Using key point position prediction information of each facial key point in the first sample facial image, key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error as input parameters of a first loss function, to obtain a loss function value of the first loss function;

[0024] If the loss function value of the first loss function is not less than a first specified value, the model parameters of the key point prediction model are adjusted.

[0025] Optionally, adjusting the second sample facial image based on the key point position prediction information of the second sample facial image includes: adjusting the second sample facial image by at least one of rotation, scaling, and translation based on the key point position prediction information of the second sample facial image, so that the second sample facial image conforms to the set facial image setting method.

[0026] Optionally, providing the adjusted second sample face image to a face recognition model to perform training and adjustment on the face recognition model includes:

[0027] obtaining, by the key point prediction model, a position confidence of each facial key point in the second sample facial image, wherein the position confidence is used to identify a degree of credibility of key point position prediction information of each facial key point in the second sample facial image;

[0028] Obtaining image features corresponding to the second sample face image, and obtaining image features corresponding to each facial key point in the second sample face image;

[0029] determining, based on the position confidence of each facial key point in the second sample facial image, a recognition weight of an image feature corresponding to each facial key point in the second sample facial image;

[0030] An identity recognition result corresponding to the second sample face image is obtained based on the image features corresponding to the second sample face image, the image features corresponding to each facial key point in the second sample face image, and the recognition weights of the image features corresponding to different facial key points in the second sample face image.

[0031] Optionally, determining the recognition weight of the image feature corresponding to each facial key point in the second sample facial image according to the position confidence of each facial key point in the second sample facial image includes:

[0032] Normalizing the position confidence of each facial key point in the second sample facial image to obtain a normalized position confidence of each facial key point in the second sample facial image;

[0033] The normalized position confidence of each facial key point in the second sample facial image is mapped to a recognition weight of an image feature corresponding to each facial key point in the second sample facial image.

[0034] Optionally, providing the adjusted second sample face image to a face recognition model to train and adjust the face recognition model further includes: comparing the identity recognition result corresponding to the second sample image with the identity annotation information to obtain a recognition accuracy rate of the face recognition model;

[0035] Determining whether the recognition accuracy of the face recognition model reaches a first accuracy threshold;

[0036] If not, adjust the model parameters of the face recognition model until the recognition accuracy of the face recognition model reaches a first accuracy threshold.

[0037] Optionally, obtaining the key point position prediction information of the sample facial image includes: respectively obtaining the key point position prediction information of each facial key point in the sample facial image, and obtaining an error evaluation value; the error evaluation value is a predicted value of a normalized average of errors between the key point position prediction information of each facial key point in the same sample facial image and the key point actual position information of the corresponding facial key point;

[0038] In the step of training and adjusting the face recognition model, the weight of the role played by the second sample face image in adjusting the model parameters of the face recognition model is determined based on the error evaluation value of the second sample face image.

[0039] Optionally, the method further includes: if the recognition accuracy of the face recognition model reaches the first accuracy threshold, ending the current cycle and entering the next cycle.

[0040] Optionally, the training requirements include at least: the recognition accuracy of the face recognition model reaches a second accuracy threshold, and the number of the sample face images reaches a predetermined number.

[0041] In another aspect, the present application further provides a model training device, comprising:

[0042] An image processing unit, configured to input a sample facial image into a key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotated information and a second sample facial image with identity annotated information;

[0043] A first model adjustment unit is used to train and adjust the key point prediction model according to the key point position prediction information and the key point position labeling information of the first sample face image;

[0044] a second model adjustment unit, configured to adjust the second sample facial image based on the key point position prediction information of the second sample facial image, provide the adjusted second sample facial image to the facial recognition model, perform training adjustment on the facial recognition model, and determine whether the facial recognition model meets the training requirements;

[0045] Among them, if the face recognition model does not meet the training requirements, the image processing unit, the first model adjustment unit and the second model adjustment unit are looped to execute the steps executed by each unit until the face recognition model meets the training requirements; each time the first model adjustment unit executes the corresponding step, it uses the key point prediction model that has undergone the latest training adjustment step.

[0046] In another aspect, the present application further provides an electronic device, characterized in that it includes:

[0047] processor; and

[0048] The memory is used to store a program of the model training method. After the device is powered on and the program of the model training method is run by the processor, the following steps are performed:

[0049] Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information;

[0050] Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information;

[0051] adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model;

[0052] The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0053] In another aspect, the present application further provides a storage medium storing a program of a model training method, wherein the program is executed by a processor to perform the following steps:

[0054] Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information;

[0055] Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information;

[0056] adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model;

[0057] The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0058] On the other hand, the present application also provides a face recognition method, comprising:

[0059] Obtaining a face image to be recognized;

[0060] Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point detection model and a target face recognition model;

[0061] The target face recognition model and the key point prediction model are obtained as follows:

[0062] A sample face image is input into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target face key point detection model is obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0063] Optionally, the step of obtaining an identity recognition result corresponding to the face image to be recognized by using a target key point detection model and a target face recognition model includes:

[0064] Inputting the face image to be recognized into a target key point prediction model to obtain key point position prediction information of the face image to be recognized;

[0065] According to the key point position prediction information of the face image to be recognized, the face image to be recognized is adjusted, and the adjusted face image to be recognized is input into the target face recognition model to obtain the face recognition result corresponding to the face image to be recognized.

[0066] In another aspect, the present application further provides a face recognition device, comprising:

[0067] A face image obtaining unit, used to obtain a face image to be identified;

[0068] A recognition result obtaining unit, configured to obtain an identity recognition result corresponding to the face image to be recognized by using a target key point detection model and a target face recognition model;

[0069] Among them, the target face recognition model and the key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0070] In another aspect, the present application further provides an electronic device, comprising:

[0071] processor; and

[0072] The memory is used to store a program of the face recognition method. After the device is powered on and the program of the face recognition method is run by the processor, the following steps are performed:

[0073] Obtaining a face image to be recognized;

[0074] Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point detection model and a target face recognition model;

[0075] Among them, the target face recognition model and the key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0076] In another aspect, the present application further provides a storage medium storing a program for a face recognition method, wherein the program is executed by a processor to perform the following steps:

[0077] Obtaining a face image to be recognized;

[0078] Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point detection model and a target face recognition model;

[0079] Among them, the target face recognition model and the key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0080] On the other hand, the present application further provides a face recognition system, characterized by comprising: a platform server and a user terminal;

[0081] The platform server is used to obtain the face image to be identified sent by the user terminal; obtain the identity recognition result corresponding to the face image to be identified using the target key point detection model and the target face recognition model; and provide the identity recognition result to the user terminal;

[0082] The user terminal is used to send the face image to be identified to the platform server; obtain the identity recognition result provided by the face image to be identified; and display the identity recognition result through a human-computer interaction page;

[0083] Among them, the target face recognition model and the key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0084] Compared with the prior art, this application has the following advantages:

[0085] The present application provides a model training method, which inputs a sample face image into a key point prediction model. First, the sample face image is input into the key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; then, the key point prediction model is trained and adjusted based on the key point position prediction information and the key point position annotation information of the first sample face image; finally, the second sample face image is adjusted based on the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to a face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used. The model training method provided in this application simultaneously adjusts the facial key point prediction model and the facial recognition model in the process of obtaining a face recognition model that meets the training requirements. This ensures that the facial key point prediction model can detect facial key points accurately while ensuring the face recognition accuracy of the face recognition model. Therefore, the model training method provided in this application can better improve the accuracy of facial recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is the first schematic diagram of the application scenario of the model training method in the embodiment of the present application.

[0087] Figure 1A This is a second schematic diagram of an application scenario of the model training method in an embodiment of the present application.

[0088] Figure 2 This is a second schematic diagram of an application scenario of the model training method in an embodiment of the present application.

[0089] Figure 3 A flowchart of a model training method is provided in the first embodiment of the present application.

[0090] Figure 4 This is a schematic diagram of a model training device provided in the second embodiment of the present application.

[0091] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application.

[0092] Figure 6 This is a flowchart of a face recognition method provided in the fifth embodiment of the present application.

[0093] Figure 7 This is a schematic diagram of a face recognition device provided in the sixth embodiment of the present application.

[0094] Figure 8 This is a schematic diagram of a face recognition system provided in the ninth embodiment of the present application. DETAILED DESCRIPTION

[0095] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0096] To more clearly demonstrate the model training method provided in the embodiments of the present application, we first introduce the application scenarios of the model training method provided in the embodiments of the present application. In practical applications, the technical solution provided in the present application is used to synchronously train the key point prediction model and the face recognition model embedded in the face recognition program. The face recognition program can be a face recognition program used for clocking in and out in office software, or a face recognition program used for users to log in to the software and confirm payments in payment software. In addition, the model training method provided in the embodiments of the present application can also be used to train face recognition models embedded in certain face recognition devices.

[0097] The model training method provided in the embodiments of the present application can be implemented through a client, a server, or through interaction between a client and a server. The so-called client is office software or payment software with a face recognition program installed on an electronic device. The so-called electronic device is generally a mobile phone, a tablet, and a computer. The so-called server is a computing device used to provide data processing and other services to the client, and the specific implementation method is generally a server or a server cluster.

[0098] Typically, the model training method provided in the embodiments of the present application can be implemented on the server side. When using the face recognition model for face recognition, a specific face image can be collected on the client side as the object to be recognized. During the model training process, the client side can obtain continuously provided sample face images, or it can directly read sample face images stored and annotated in the database.

[0099] Please refer to Figure 1 , which is the first schematic diagram of the application scenario of the model training method in the embodiment of the present application.

[0100] First, the server 101 obtains a first sample face image with key point position annotation information and a second sample face image with identity annotation information provided by the client 102 .

[0101] The first sample facial image and the second sample facial image comprise multiple facial images. The key point position annotation information is the annotation information generated by annotating the positions of the key points of each facial image in the first sample facial image. The identity annotation information is the annotation information generated by annotating the identity corresponding to the second sample facial image.

[0102] Secondly, the server 101 uses the first sample face image and the second sample face image to synchronously train the key point prediction model and the face recognition model.

[0103] Again, the server 101 provides the trained key point prediction model and face recognition model to the client 102 for use. The specific usage method can be to directly provide the key point prediction model and face recognition model to the client 102 and have the client 102 directly execute them, or to provide the client 102 with an interface to access the above-mentioned model, and the client 102 provides the face image to be recognized, and the server 101 executes the specific prediction and recognition process of the model, that is, adopting the SAAS (Software-as-a-Service) working mode.

[0104] Finally, the client 102 obtains the face image to be identified, inputs the face image to be identified into the key point prediction model, obtains the key point position prediction information of the face image to be identified, adjusts the face image to be identified based on the key point position prediction information of the face image to be identified, and inputs the adjusted face image to be identified into the target face recognition model to obtain the face recognition result corresponding to the face image to be identified.

[0105] In the application scenario of the model training method provided in the embodiment of the present application, the server 101 uses the first sample face image and the second sample face image to synchronously train the key point prediction model and the face recognition model in the following steps:

[0106] First, input the first and second sample facial images into a key point prediction model to obtain key point position prediction information for the first and second sample facial images. Obtaining key point position prediction information for the first and second sample facial images specifically involves obtaining key point position prediction information for each facial key point in the first and second sample facial images, and obtaining key point position prediction information for each facial key point in the second sample facial image. Facial key points are pre-set points that are prominent features on the face. They can be set to a fixed number and located at fixed positions relative to the face, such as the tip of the nose or the corners of the eyes.

[0107] At this time, the so-called key point prediction model is a pre-trained available key point prediction model, which is used to obtain a specified face image and identify each facial key point in the specified face image, obtain the key point position prediction information of the specified face image, the error evaluation value of the specified face image, and the position confidence of each facial key point in the specified face image.

[0108] The key point position prediction information for a specified facial image refers to the position prediction information of each facial key point in the specified facial image, as determined by the key point prediction model. Specifically, the position is typically the coordinates of the facial key points in a preset coordinate system.

[0109] The so-called error evaluation value of a specified facial image is the predicted value of the normalized average of the errors between the key point position prediction information of each facial key point in the specified facial image and the key point actual position information of the corresponding facial key point. In specific implementation, the so-called error evaluation value is generally the normalized average. That is, the error evaluation value of the possible error between each key point and the corresponding actual key point is first obtained, and then the error evaluation value is normalized and then averaged to obtain the normalized average. It should be noted that the error evaluation value here is an estimate of the prediction error based on the model's own characteristics, and does not compare the predicted key point position with the actual key point position, because the actual key point position cannot be known unless it has been marked; this error evaluation value is an estimate of the key point prediction model's own prediction accuracy.

[0110] The so-called position confidence of each facial key point in the specified face image is used to identify the credibility of the key point position prediction information of each facial key point in the specified face image.

[0111] The three outputs of the above-mentioned key point prediction model, namely the key point prediction position, error evaluation value and position confidence, are all obtained through model training. That is, the sample face image with key point annotation information is provided to the key point prediction model (the one that has not been trained is the key point prediction initial model). According to the prediction, the prediction result is compared with the annotation information and the information inferred from the annotation information. The parameters are adjusted, and the prediction and comparison are repeated in a continuous cycle to obtain a key point prediction model that can provide reasonable output.

[0112] In the application scenario of the model training method provided in the embodiment of the present application, the process of obtaining an available key point prediction model is as follows:

[0113] First, an initial key point prediction model is obtained. Then, a sample facial image with key point location annotated information is obtained. The sample facial image is provided to the initial key point prediction model to obtain key point location prediction information for the sample facial image. Finally, based on the key point location prediction information and key point location annotated information from the first sample facial image, the key point prediction model is continuously adjusted until a usable key point prediction model is obtained.

[0114] The specific implementation method of continuously adjusting the key point prediction model until a usable key point prediction model is obtained is as follows:

[0115] First, the initial keypoint prediction model is used to obtain the keypoint position prediction information and the corresponding error evaluation value for the sample face image. The so-called error evaluation value is the predicted value of the normalized average error between the keypoint position prediction information of each facial keypoint in the sample face image and the actual keypoint position information of the corresponding facial keypoint.

[0116] Then, according to the key point position prediction information of each facial key point in the sample face image, the gap between the key point position prediction information of each facial key point in the sample face image and the key point position annotation information of the corresponding key point is obtained as the actual error.

[0117] The so-called normalized mean of the actual error is calculated as follows:

[0118] First, based on the key point position prediction information of each facial key point in the sample face image, the first position coordinates corresponding to each facial key point in the sample face image are obtained, and based on the key point position annotation information of each facial key point in the sample face image, the second position coordinates corresponding to each facial key point in the sample face image are obtained.

[0119] Second, obtain the sum of the distances between the first position coordinates corresponding to each facial key point in the sample facial image and the second position coordinates of the corresponding facial key point. Among them, p k is the first position coordinate, g k is the second position coordinate, and k is the kth facial key point in the sample face image.

[0120] Third, obtain the length value width of the annotation box corresponding to the key point position annotation information bbox and width value width bbox height bbox .

[0121] Third, based on the distance, length, and width values, obtain the normalized average value of the actual error. For details, please refer to Formula 1: Among them, NME is the normalized mean of the actual error,

[0122] Please refer to Figure 1A , which is a second schematic diagram of an application scenario of the model training method in an embodiment of the present application, Figure 1A is the annotation information corresponding to the face image to be recognized, where k = (1, 2, 3...68).

[0123] Finally, please refer to Figure 1 , according to the key point position prediction information of each facial key point in the sample face image, the key point position annotation information of each facial key point in the sample face image and the error evaluation value, the key point prediction model is trained and adjusted.

[0124] The process of training and adjusting the key point prediction model is as follows:

[0125] First, the key point position prediction information of each facial key point in the sample face image, the key point position labeling information of each facial key point in the sample face image, the error evaluation value, and the actual error are used as input parameters of the first loss function to obtain the loss function value of the first loss function. The form of the so-called first loss function is shown in Formula 2: LOSS1 = (1 + γC) || p k -g k ||+α||B-NME||...Formula (2), where LOSS1 is the loss function value of the first loss function, B is the error evaluation value, and γ, C, and α are pre-set self-learning parameters. Third, if the loss function value of the first loss function is not less than the first specified value, the model parameters of the key point prediction model are adjusted. The key point prediction model is continuously adjusted until the loss function value of the first loss function is less than the first specified value, thereby obtaining a usable key point prediction model.

[0126] In the application scenario of the model training method provided in the embodiment of this application, please refer to the specific process of model training. Figure 2 , which is a second schematic diagram of an application scenario of the model training method in the embodiment of the present application. In the application scenario of the model training method in the embodiment of the present application, the specific process of model training is implemented by a model training system, which includes three modules, namely, a key point generation module 201, a key point quality assessment module 202, and a face understanding module 203.

[0127] The key point prediction model generation module 201 is used to obtain a first sample face image and a second sample face image; input the first sample face image and the second sample face image into the key point prediction model to obtain the position prediction information of each facial key point in the first sample face image and the error evaluation value of the first sample face image; and obtain the position prediction information of each facial key point in the second sample face image, the position confidence of each facial key point in the second sample face image and the error evaluation value of the second sample face image.

[0128] In addition, the key point prediction model generation module 201 is further configured to provide the above information to the key point quality assessment module 202 .

[0129] After obtaining the above information, the key point quality assessment module 202 will simultaneously perform the following two different processes:

[0130] The first process involves training and adjusting the key point prediction model based on the key point position prediction information and key point position annotation information of the first sample facial image. Specifically, the error evaluation value and actual error of the first sample facial image are obtained based on the key point position prediction information and key point position annotation information of each facial key point in the first sample facial image. Secondly, the error evaluation value and actual error of the first sample facial image are visualized via the key point perception module 202-1. Thirdly, the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value, and the actual error of the first sample facial image are used as input parameters of a first loss function to obtain a loss function value of the first loss function. Finally, if the loss function value of the first loss function is not less than a first specified value, the model parameters of the key point prediction model are adjusted.

[0131] The second process is: first, according to the key point position prediction information of the second sample face image, the second sample face image is adjusted. That is, according to the key point position prediction information of the second sample face image, the face position in the second sample face image is adjusted to meet the preset requirements for the image when performing face image recognition. It can be figuratively considered that the face image is rotated, moved, scaled, etc. according to the key point position prediction information. In terms of specific implementation, some mathematical transformation methods can be used. For example, a similarity transformation matrix is ​​set, and the coordinates of the predicted positions of each face key point in the second sample face image are obtained through the similarity transformation matrix and the coordinates of the predicted positions of each face key point in the second sample face image after the similarity transformation. For details, please refer to the formula (3): Y = TX... In formula (3), T is the similarity transformation matrix, Y is the coordinates of each key point in the second sample face image after the similarity transformation, and X is the position coordinates corresponding to different face key points in the second sample face image, where,

[0132] The key point perception module 202-1 is a module for visualizing information and adjusting the second sample face images. The purpose of adjusting the second sample face images is to perform similarity transformation on different second sample face images, and transform different second sample face images into images with uniform size and angle.

[0133] In addition, the key point quality assessment module 202 can also obtain the normalized position confidence of each facial key point in the second sample facial image by normalizing the position confidence of each facial key point in the second sample facial image; and map the normalized position confidence of each facial key point in the second sample facial image to the recognition weight of the image feature corresponding to each facial key point in the second sample facial image.

[0134] Keypoint quality assessment module 202 inputs the adjusted second sample facial image and the recognition weights of the image features corresponding to each facial key point in the second sample facial image into face understanding module 203. Face understanding module 203 then sequentially performs the following steps: First, image features corresponding to the second sample facial image and image features corresponding to each facial key point in the second sample facial image are obtained using image feature extraction model 203-1. Then, based on the image features corresponding to the second sample facial image, the recognition weights of the image features corresponding to each facial key point in the second sample facial image, and the recognition weights of the image features corresponding to each facial key point in the second sample facial image, an identity recognition result corresponding to the second sample image is obtained.

[0135] The face understanding module 203 is also used to adjust the model parameters of the face recognition model based on the identity recognition results and identity annotation information. That is, first, the identity recognition results are compared with the identity annotation information to obtain the recognition accuracy of the face recognition model. If the number of second sample face images obtained in a cycle is 100, if the identity recognition results are compared with the identity annotation information and 90 of the identity recognition results are consistent with the identity annotation information, then the recognition accuracy of the face recognition model is 90%. Then, it is determined whether the recognition accuracy of the face recognition model reaches the first accuracy threshold. If not, the model parameters of the face recognition model are adjusted until the recognition accuracy of the face recognition model reaches the first accuracy threshold. If so, the current cycle ends and the next cycle begins until the face recognition model meets the predetermined training requirements. That is, the first sample face image and the second sample face image are re-obtained and input into the key point prediction model. In each cycle, the most recently adjusted face key point prediction model is used as the face key point prediction model for the current cycle. That is, the above steps are executed in a loop, and in each loop, the adjusted facial key point prediction model obtained in the previous loop is used as the facial key point prediction model of the current loop until the face recognition model meets the predetermined training requirements.

[0136] The so-called specific implementation method of adjusting the face recognition model is: first, according to the error evaluation value of each sample face image, determine the weight of the role of the sample face image in the parameter adjustment of the face recognition model; then, according to the weight, adjust the face recognition model; specifically, the error evaluation value reflects the recognition level of the sample face image, and the larger the error evaluation value, the more difficult the sample face image is to recognize; for sample face images that are easy to recognize, they should have a higher weight in training to ensure that the images that are easy to recognize are accurately recognized, and for sample face images with larger error evaluation values, they can be given a relatively low weight in training to avoid introducing unreasonable parameter adjustment directions to the face recognition model.

[0137] The so-called predetermined training requirements include at least: the face recognition model's recognition accuracy in each cycle reaches a second accuracy threshold, and the number of sample face images reaches a predetermined number. If a face recognition model is not trained with sufficient samples, it will be undertrained. Even if it can accurately recognize all labeled samples, it may still have a low recognition rate when facing actual recognition tasks.

[0138] The embodiments of this application do not specifically limit the application scenarios of the model training method provided in the embodiments of this application. The embodiments corresponding to the application scenarios of the above-mentioned model training method are provided to facilitate understanding of the model training method provided in this application, and are not used to limit the model training method provided in this application.

[0139] First embodiment

[0140] In the first embodiment of the present application, a model training method is provided. The specific process is as follows: Figure 3 As shown, it is a flowchart of the model training method provided in the first embodiment of the present application. Figure 3 The model training method shown includes: steps S301 to S303.

[0141] In step S301, the sample face image is input into the key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information.

[0142] The first sample facial image and the second sample facial image can be facial images obtained from an existing facial recognition image database, or can be facial images captured by an image acquisition device. The image acquisition device includes but is not limited to a camera, a camcorder, and a mobile phone. The images can also be video frames.

[0143] In the first embodiment of the present application, the key point prediction model is used to obtain a specified facial image and identify each facial key point in the specified facial image, obtain the key point position prediction information of the specified facial image, the error evaluation value of the specified facial image, and the position confidence of each facial key point in the specified facial image.

[0144] The key point position prediction information for a specified facial image refers to the position prediction information of each facial key point in the specified facial image, as determined by the key point prediction model. Specifically, the position is typically the coordinates of the facial key points in a preset coordinate system.

[0145] The so-called error evaluation value of a specified facial image is the predicted value of the normalized average of the errors between the key point position prediction information of each facial key point in the specified facial image and the key point actual position information of the corresponding facial key point. In specific implementation, the so-called error evaluation value is generally the normalized average. That is, the error evaluation value of the possible error between each key point and the corresponding actual key point is first obtained, and then the error evaluation value is normalized and then averaged to obtain the normalized average. It should be noted that the error evaluation value here is an estimate of the prediction error based on the model's own characteristics, and does not compare the predicted key point position with the actual key point position, because the actual key point position cannot be known unless it has been marked; this error evaluation value is an estimate of the key point prediction model's own prediction accuracy.

[0146] The so-called position confidence of each facial key point in the specified face image is used to identify the credibility of the key point position prediction information of each facial key point in the specified face image.

[0147] In the first embodiment of the present application, the specific steps of obtaining the key point position prediction information of the sample face image are: obtaining the key point position prediction information of each facial key point in the sample face image respectively, and obtaining the error evaluation value.

[0148] The so-called error evaluation value is the predicted value of the normalized average of the errors between the key point position prediction information of each facial key point in the same sample face image and the key point actual position information of the corresponding facial key point.

[0149] In step S302, the key point prediction model is trained and adjusted based on the key point position prediction information and the key point position labeling information of the first sample face image.

[0150] In the first embodiment of the present application, the process of training and adjusting the key point prediction model based on the key point position prediction information and the key point position annotation information of the first sample face image is: training and adjusting the key point prediction model based on the key point position prediction information of each facial key point in the first sample face image, the key point position annotation information of each facial key point in the first sample face image, and the error evaluation value of the first sample face image.

[0151] The specific implementation method for training and adjusting the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, and the error evaluation value of the first sample facial image is as follows: first, based on the key point position prediction information of each facial key point in the first sample facial image, the difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point is obtained as the actual error. Then, based on the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error, the key point prediction model is adjusted.

[0152] In the first embodiment of the present application, the adjustment target includes making the key point position annotation information of each facial key point in the first sample facial image consistent with the key point position annotation information of the corresponding key point, and the error evaluation value of the first sample facial image consistent with the normalized average value of the actual error.

[0153] In addition, the normalized mean of the actual error is calculated as follows:

[0154] First, based on the key point position prediction information of each facial key point in the first sample face image, the first position coordinates corresponding to each facial key point in the first sample face image are obtained, and based on the key point position annotation information of each facial key point in the first sample face image, the second position coordinates corresponding to each facial key point in the first sample face image are obtained.

[0155] Second, obtain the sum of the distances between the first position coordinates corresponding to each facial key point in the first sample facial image and the second position coordinates of the corresponding facial key point.

[0156] Third, obtain the length and width of the annotation box corresponding to the key point position annotation information.

[0157] Fourth, according to the distance, length and width values, the normalized average value of the actual error is obtained.

[0158] In the first embodiment of the present application, the execution steps of adjusting the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error are as follows: first, the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error are used as input parameters of the first loss function to obtain the loss function value of the first loss function; second, if the loss function value of the first loss function is not less than the first specified value, the model parameters of the key point prediction model are adjusted.

[0159] In step S303, the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to perform training adjustment on the face recognition model.

[0160] In the first embodiment of the present application, the specific implementation method of adjusting the second sample facial image based on the key point position prediction information of the second sample facial image is: based on the key point position prediction information of the second sample facial image, at least one of rotation, scaling, and translation is adopted to adjust the second sample facial image so that the second sample facial image conforms to the set facial image setting method.

[0161] In the first embodiment of the present application, after providing the adjusted second sample face image to the face recognition model, the face recognition model performs the following steps:

[0162] First, the position confidence of each facial key point in the second sample face image is obtained through the key point prediction model.

[0163] The so-called position confidence is used to identify the credibility of the key point position prediction information of each facial key point in the second sample face image.

[0164] Next, image features corresponding to the second sample face image are obtained, and image features corresponding to each facial key point in the second sample face image are obtained.

[0165] Thirdly, determining the recognition weight of the image feature corresponding to each facial key point in the second sample facial image according to the position confidence of each facial key point in the second sample facial image;

[0166] The specific implementation method of determining the recognition weight of the image feature corresponding to each facial key point in the second sample facial image based on the position confidence of each facial key point in the second sample facial image is: normalizing the position confidence of each facial key point in the second sample facial image to obtain the normalized position confidence of each facial key point in the second sample facial image; and mapping the normalized position confidence of each facial key point in the second sample facial image to the recognition weight of the image feature corresponding to each facial key point in the second sample facial image.

[0167] Finally, the identity recognition result corresponding to the second sample face image is obtained based on the image features corresponding to the second sample face image, the image features corresponding to each facial key point in the second sample face image, and the recognition weights of the image features corresponding to different facial key points in the second sample face image.

[0168] In the first embodiment of the present application, the steps for training and adjusting the face recognition model are as follows: first, the identity recognition result corresponding to the second sample image is compared with the identity annotation information to obtain the recognition accuracy of the face recognition model. Then, it is determined whether the recognition accuracy of the face recognition model reaches the first accuracy threshold. Finally, if not, the model parameters of the face recognition model are adjusted until the recognition accuracy of the face recognition model reaches the first accuracy threshold; if the recognition accuracy of the face recognition model reaches the first accuracy threshold, the current cycle is ended and the next cycle is entered. That is, the above steps S301-S303 are executed cyclically until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0169] In the step of training and adjusting the face recognition model, the weight of the role played by the second sample face image in adjusting the model parameters of the face recognition model is determined based on the error evaluation value of the second sample face image.

[0170] In the first embodiment of the present application, the training requirements include at least: the recognition accuracy of the face recognition model reaches a second accuracy threshold, and the number of sample face images reaches a predetermined number.

[0171] A first embodiment of the present application provides a model training method, which inputs a sample facial image into a key point prediction model. First, the sample facial image is input into the key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotation information, and a second sample facial image with identity annotation information; then, the key point prediction model is trained and adjusted based on the key point position prediction information and the key point position annotation information of the first sample facial image; finally, the second sample facial image is adjusted based on the key point position prediction information of the second sample facial image, and the adjusted second sample facial image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the facial key point prediction position information of the sample facial image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used. The model training method provided by the present application adjusts the facial key point prediction model and the facial recognition model simultaneously in the process of obtaining a facial recognition model that meets the training requirements, so as to ensure the facial key point detection accuracy of the facial key point prediction model while ensuring the facial recognition accuracy of the facial recognition model. In the above-mentioned joint training process, the improvement of the key point prediction model can effectively improve the recognition ability of the facial recognition model. At the same time, the facial recognition ability, as an important feedback, can also promote the optimization of the key point prediction model; that is, if the recognition ability of the facial recognition model cannot reach the target, it will put forward optimization requirements for the key point prediction model, which will promote the optimization of the key point prediction model. Therefore, the model training method provided by the present application can better improve the accuracy of the face.

[0172] In addition, in the model training method provided in the first embodiment of the present application, in the step of training and adjusting the face recognition model, the weight of the role played by the second sample face image in adjusting the model parameters of the face recognition model is determined based on the error evaluation value of the second sample face image. Therefore, the influence of sample face images with larger error evaluation values ​​on the face recognition model can be reduced, while the influence of sample face images with smaller error evaluation values ​​on the face recognition model can be enhanced. Therefore, the face recognition performance of the face recognition model can be improved.

[0173] Second embodiment

[0174] Corresponding to the embodiment corresponding to the application scenario of the model training method provided in this application and the model training method provided in the first embodiment, the second embodiment of this application also provides a model training device. Since the device embodiment is basically similar to the embodiment corresponding to the application scenario and the first embodiment, the description is relatively simple. For relevant details, please refer to the embodiment corresponding to the application scenario and the partial description of the first embodiment. The device embodiment described below is only illustrative.

[0175] Please refer to Figure 4 , which is a schematic diagram of a model training device provided in the second embodiment of the present application.

[0176] The model training device provided in the second embodiment of the present application includes:

[0177] An image processing unit 401 is configured to input a sample facial image into a key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotated information and a second sample facial image with identity annotated information;

[0178] A first model adjustment unit 402 is configured to train and adjust the key point prediction model based on the key point position prediction information and the key point position labeling information of the first sample face image;

[0179] a second model adjustment unit 403 configured to adjust the second sample facial image based on the key point position prediction information of the second sample facial image, provide the adjusted second sample facial image to the face recognition model, perform training adjustment on the face recognition model, and determine whether the face recognition model meets the training requirements;

[0180] Among them, if the face recognition model does not meet the training requirements, the image processing unit, the first model adjustment unit and the second model adjustment unit are looped to execute the steps executed by each unit until the face recognition model meets the training requirements; each time the first model adjustment unit executes the corresponding step, it uses the key point prediction model that has undergone the latest training adjustment step.

[0181] Optionally, the image processing unit 401 is specifically configured to respectively obtain key point position prediction information of each facial key point in the sample facial image, and obtain an error evaluation value; the error evaluation value is a predicted value of a normalized average of errors between the key point position prediction information of each facial key point in the same sample facial image and the key point actual position information of the corresponding facial key point;

[0182] The first model adjustment unit 402 is specifically used to train and adjust the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, and the error evaluation value of the first sample facial image.

[0183] Optionally, the training and adjusting the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, and the error evaluation value of the first sample facial image includes:

[0184] obtaining, based on the key point position prediction information of each facial key point in the first sample facial image, a difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point as an actual error;

[0185] Adjust the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image and the actual error; the adjustment goals include making the key point position annotation information of each facial key point in the first sample facial image consistent with the key point position annotation information of the corresponding key point, and making the error evaluation value of the first sample facial image consistent with the normalized average of the actual error.

[0186] Optionally, the normalized average value of the actual error is calculated as follows:

[0187] Obtaining first position coordinates corresponding to each facial key point in the first sample facial image based on key point position prediction information of each facial key point in the first sample facial image, and obtaining second position coordinates corresponding to each facial key point in the first sample facial image based on key point position annotation information of each facial key point in the first sample facial image;

[0188] Obtaining the sum of distances between first position coordinates corresponding to each facial key point in the first sample facial image and second position coordinates of the corresponding facial key point;

[0189] Obtain the length and width of the annotation box corresponding to the key point position annotation information;

[0190] A normalized average value of the actual error is obtained according to the distance sum, the length value, and the width value.

[0191] Optionally, adjusting the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error includes:

[0192] Using key point position prediction information of each facial key point in the first sample facial image, key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error as input parameters of a first loss function, to obtain a loss function value of the first loss function;

[0193] If the loss function value of the first loss function is not less than a first specified value, the model parameters of the key point prediction model are adjusted.

[0194] Optionally, the second model adjustment unit 403 is specifically used to adjust the second sample face image by at least one of rotation, scaling, and translation based on the key point position prediction information of the second sample face image, so that the second sample face image conforms to the set face image setting method.

[0195] Optionally, the second model adjustment unit 403 is specifically used to obtain the position confidence of each facial key point in the second sample facial image through the key point prediction model, and the position confidence is used to identify the credibility of the key point position prediction information of each facial key point in the second sample facial image; obtain the image features corresponding to the second sample facial image, and obtain the image features corresponding to each facial key point in the second sample facial image; determine the recognition weight of the image features corresponding to each facial key point in the second sample facial image according to the position confidence of each facial key point in the second sample facial image; obtain the identity recognition result corresponding to the second sample image according to the image features corresponding to the second sample facial image, the image features corresponding to each facial key point in the second sample facial image, and the recognition weights of the image features corresponding to different facial key points in the second sample facial image.

[0196] Optionally, determining the recognition weight of the image feature corresponding to each facial key point in the second sample facial image according to the position confidence of each facial key point in the second sample facial image includes:

[0197] Normalizing the position confidence of each facial key point in the second sample facial image to obtain a normalized position confidence of each facial key point in the second sample facial image;

[0198] The normalized position confidence of each facial key point in the second sample facial image is mapped to a recognition weight of an image feature corresponding to each facial key point in the second sample facial image.

[0199] Optionally, the second model adjustment unit 403 is specifically used to compare the identity recognition result corresponding to the second sample image with the identity annotation information to obtain the recognition accuracy of the face recognition model; determine whether the recognition accuracy of the face recognition model reaches a first accuracy threshold; if not, adjust the model parameters of the face recognition model until the recognition accuracy of the face recognition model reaches the first accuracy threshold.

[0200] Optionally, the image processing unit 401 is specifically configured to respectively obtain key point position prediction information of each facial key point in the sample facial image, and obtain an error evaluation value; the error evaluation value is a predicted value of a normalized average of errors between the key point position prediction information of each facial key point in the same sample facial image and the key point actual position information of the corresponding facial key point;

[0201] In the step of training and adjusting the face recognition model, the weight of the role played by each sample face image in adjusting the model parameters of the face recognition model is determined based on the error evaluation value of the sample face image.

[0202] Optionally, the method further includes: if the recognition accuracy of the face recognition model reaches the first accuracy threshold, ending the current cycle and entering the next cycle.

[0203] Optionally, the training requirements include at least: the recognition accuracy of the face recognition model reaches a second accuracy threshold, and the number of the sample face images reaches a predetermined number.

[0204] Third embodiment

[0205] Corresponding to the model training method provided in the first embodiment of this application, the third embodiment of this application also provides an electronic device. Since the third embodiment is basically similar to the first embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the first embodiment. The device embodiment described below is merely illustrative.

[0206] Please refer to Figure 5 , which is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0207] The electronic device includes: a processor 501;

[0208] The memory 502 is used to store a program of the model training method. After the device is powered on and the program of the model training method is run by the processor, the following steps are performed:

[0209] Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information;

[0210] Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information;

[0211] adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model;

[0212] The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0213] It should be noted that the detailed description of the electronic device provided in the third embodiment of the present application can refer to the application scenario embodiment of the model training method provided in the present application and the relevant description of the model training method provided in the first embodiment, which will not be repeated here.

[0214] Fourth embodiment

[0215] Corresponding to the model training method provided in the first embodiment of this application, the fourth embodiment of this application also provides a storage medium. Since the fourth embodiment is basically similar to the first embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the first embodiment. The device embodiment described below is merely illustrative.

[0216] The storage medium stores a computer program, which is executed by a processor to perform the following steps:

[0217] Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information;

[0218] Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information;

[0219] adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model;

[0220] The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0221] It should be noted that the detailed description of the storage medium provided in the fourth embodiment of the present application can refer to the application scenario embodiment of the model training method provided in the present application and the relevant description of the model training method provided in the first embodiment, and will not be repeated here.

[0222] Fifth embodiment

[0223] Corresponding to the application scenario embodiment of the model training method provided in this application and the model training method provided in the first embodiment, the fifth embodiment of this application also provides a face recognition method. For relevant details, please refer to the partial description of the first embodiment. The following description of the fifth embodiment is merely illustrative.

[0224] Please refer to Figure 6 , which is a flowchart of a face recognition method provided in the fifth embodiment of the present application. Figure 6 The face recognition method shown includes: steps S601 to S602.

[0225] Step S601: Obtain a face image to be recognized.

[0226] Step S602: using the target key point detection model and the target face recognition model to obtain the identity recognition result corresponding to the face image to be recognized.

[0227] In the fifth embodiment of the present application, the target face recognition model and the key point prediction model are obtained in the following manner: the sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information and the key point position annotation information of the first sample face image; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model is obtained, and the target face key point detection model is obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0228] The specific implementation method for using the target keypoint detection model and the target face recognition model to obtain the identity recognition result corresponding to the face image to be recognized is as follows: First, the face image to be recognized is input into the target keypoint prediction model to obtain the predicted keypoint positions of the face image to be recognized. Then, based on the predicted keypoint positions of the face image to be recognized, the face image to be recognized is adjusted and the adjusted face image to be recognized is input into the target face recognition model to obtain the face recognition result corresponding to the face image to be recognized.

[0229] The fifth embodiment of the present application provides a face recognition method, which first obtains a face image to be recognized; then, uses a target key point detection model and a target face recognition model to obtain an identity recognition result corresponding to the face image to be recognized; wherein, the sample face image is input into the key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; according to the key point position prediction information and the key point position annotation information of the first sample face image, the key point prediction model is trained and adjusted; according to the key point position prediction information of the second sample face image, the second sample face image is adjusted, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used. The face recognition method provided in the fifth embodiment of the present application simultaneously adjusts the facial key point prediction model and the face recognition model in the process of obtaining a face recognition model that meets the training requirements. This ensures that the facial key point prediction model can detect facial key points accurately while ensuring the face recognition accuracy of the face recognition model. Therefore, the face recognition method provided in the fifth embodiment of the present application can better improve the accuracy of face recognition results.

[0230] Sixth embodiment

[0231] Corresponding to the application scenario embodiment of the model training method provided in this application and the face recognition method provided in the fifth embodiment, the sixth embodiment of this application also provides a face recognition device. Since the device embodiment is basically similar to the embodiment corresponding to the application scenario and the fifth embodiment, the description is relatively simple. For relevant details, please refer to the embodiment corresponding to the application scenario and the partial description of the fifth embodiment. The device embodiment described below is merely illustrative.

[0232] Please refer to Figure 7 , which is a schematic diagram of a face recognition device provided in the sixth embodiment of the present application.

[0233] The face recognition device provided in the sixth embodiment of the present application includes:

[0234] The face image obtaining unit 702 is used to obtain a face image to be recognized;

[0235] The recognition result obtaining unit 702 is used to obtain the identity recognition result corresponding to the face image to be recognized by using the target key point detection model and the target face recognition model;

[0236] Among them, the target face recognition model and the key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, and the target face recognition model and the target face key point detection model are obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0237] Optionally, the recognition result obtaining unit 702 is specifically used to input the face image to be identified into the target key point prediction model to obtain key point position prediction information of the face image to be identified; adjust the face image to be identified according to the key point position prediction information of the face image to be identified, and input the adjusted face image to be identified into the target face recognition model to obtain the face recognition result corresponding to the face image to be identified.

[0238] Seventh embodiment

[0239] Corresponding to the face recognition method provided in the fifth embodiment of this application, the seventh embodiment of this application also provides an electronic device. Since the seventh embodiment is substantially similar to the fifth embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the fifth embodiment. The device embodiment described below is merely illustrative.

[0240] Please refer to Figure 5 , which is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0241] The electronic device includes: a processor 501;

[0242] and a memory 502 for storing a program of a face recognition method. After the device is powered on and the program of the face recognition method is run by the processor, the following steps are performed:

[0243] Obtaining a face image to be recognized;

[0244] Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point detection model and a target face recognition model;

[0245] The target face recognition model and the key point prediction model are obtained as follows:

[0246] A sample face image is input into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target face key point detection model is obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0247] It should be noted that the detailed description of the electronic device provided in the seventh embodiment of the present application can refer to the application scenarios of the face recognition method provided in the present application and the relevant description of the face recognition method provided in the fifth embodiment, which will not be repeated here.

[0248] Eighth embodiment

[0249] Corresponding to the face recognition method provided in the fifth embodiment of this application, the eighth embodiment of this application further provides a storage medium. Since the eighth embodiment is substantially similar to the fifth embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the fifth embodiment. The device embodiment described below is merely illustrative.

[0250] The storage medium stores a computer program, which is executed by a processor to perform the following steps:

[0251] Obtaining a face image to be recognized;

[0252] Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point detection model and a target face recognition model;

[0253] The target face recognition model and the key point prediction model are obtained as follows:

[0254] A sample face image is input into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target face key point detection model is obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0255] It should be noted that the detailed description of the storage medium provided in the eighth embodiment of the present application can refer to the relevant description of the face recognition method provided in the fifth embodiment of the present application, and will not be repeated here.

[0256] Ninth embodiment

[0257] Corresponding to the application scenario embodiment of the model training method provided in this application and the face recognition method provided in the fifth embodiment, the ninth embodiment of this application also provides a face recognition device. Since the device embodiment is basically similar to the embodiment corresponding to the application scenario and the fifth embodiment, the description is relatively simple. For relevant details, please refer to the embodiment corresponding to the application scenario and the partial description of the fifth embodiment. The device embodiment described below is merely illustrative.

[0258] Please refer to Figure 8 , which is a schematic diagram of a face recognition system provided in the ninth embodiment of the present application.

[0259] The face recognition system provided in the ninth embodiment of the present application includes: a platform server 801 and a user terminal 802.

[0260] In the ninth embodiment of the present application, the platform server 801 refers to a computing device that provides services for the software platform or application platform installed on the user terminal 802 for executing the face recognition method provided in this application, and in specific implementations is generally a server or server cluster. The user terminal 802 refers to a computing device installed with the software platform or application platform for executing the face recognition method provided in this application, and in specific implementations is generally a smartphone, tablet computer, personal computer, etc.

[0261] The platform server 801 is used to obtain the face image to be identified sent by the user terminal 802; use the target key point detection model and the target face recognition model to obtain the identity recognition result corresponding to the face image to be identified; and provide the identity recognition result to the user terminal 802.

[0262] The user terminal 802 is used to send the face image to be identified to the platform server 801; obtain the identity recognition result provided by the face image to be identified; and display the identity recognition result through a human-computer interaction page.

[0263] The target face recognition model and key point prediction model are obtained as follows:

[0264] Obtaining a face image to be recognized;

[0265] Using the target key point detection model and the target face recognition model, the identity recognition result corresponding to the face image to be recognized is obtained;

[0266] The target face recognition model and key point prediction model are obtained as follows:

[0267] The sample face image is input into the key point prediction model to obtain the key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point prediction model is trained and adjusted according to the key point position prediction information and the key point position annotation information of the first sample face image; the second sample face image is adjusted according to the key point position prediction information of the second sample face image, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, and the target face recognition model is obtained, and the target face key point detection model is obtained; each time the step of obtaining the face key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

[0268] Although the present application is disclosed as above with reference to preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0269] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0270] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0271] 1. Computer-readable media, including permanent and non-permanent, removable and non-removable media, can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmitting media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0272] 2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A model training method, characterized in that: include: Inputting a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotated information and a second sample face image with identity annotated information; Training and adjusting the key point prediction model based on the key point position prediction information of the first sample face image and the key point position annotation information; adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model; The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used; The method also includes: obtaining key point position prediction information and error evaluation values ​​of each facial key point in the sample facial image through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; training and adjusting the key point prediction model based on the key point position prediction information of the first sample facial image, the key point position annotation information of the first sample facial image, the error evaluation value of the first sample facial image, and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample facial image and the key point position annotation information of the corresponding key point.

2. The method according to claim 1, characterized in that The training and adjusting of the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error includes: According to the key point position prediction information of each facial key point in the first sample facial image, obtaining the difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point as the actual error; Adjust the key point prediction model based on the key point position prediction information of each facial key point in the first sample facial image, the key point position annotation information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image and the actual error; the adjustment goals include making the key point position annotation information of each facial key point in the first sample facial image consistent with the key point position annotation information of the corresponding key point, and making the error evaluation value of the first sample facial image consistent with the normalized average of the actual error.

3. The method according to claim 2, characterized in that The normalized mean value of the actual error is calculated as follows: Obtaining first position coordinates corresponding to each facial key point in the first sample facial image based on key point position prediction information of each facial key point in the first sample facial image, and obtaining second position coordinates corresponding to each facial key point in the first sample facial image based on key point position annotation information of each facial key point in the first sample facial image; Obtaining the sum of distances between first position coordinates corresponding to each facial key point in the first sample facial image and second position coordinates of the corresponding facial key point; Obtain the length and width of the annotation box corresponding to the key point position annotation information; A normalized average value of the actual error is obtained according to the distance sum, the length value, and the width value.

4. The method according to claim 2, characterized in that The adjusting the key point prediction model according to the key point position prediction information of each facial key point in the first sample facial image, the key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error includes: Using key point position prediction information of each facial key point in the first sample facial image, key point position labeling information of each facial key point in the first sample facial image, the error evaluation value of the first sample facial image, and the actual error as input parameters of a first loss function, to obtain a loss function value of the first loss function; If the loss function value of the first loss function is not less than a first specified value, the model parameters of the key point prediction model are adjusted.

5. The method according to claim 1, wherein The adjusting of the second sample facial image according to the key point position prediction information of the second sample facial image includes: adjusting the second sample facial image by at least one of rotation, scaling, and translation according to the key point position prediction information of the second sample facial image, so that the second sample facial image conforms to the set facial image setting method.

6. The method according to claim 1, wherein Providing the adjusted second sample face image to the face recognition model to train and adjust the face recognition model includes: obtaining, by the key point prediction model, a position confidence of each facial key point in the second sample facial image, wherein the position confidence is used to identify a degree of credibility of key point position prediction information of each facial key point in the second sample facial image; Obtaining image features corresponding to the second sample face image, and obtaining image features corresponding to each facial key point in the second sample face image; determining, based on the position confidence of each facial key point in the second sample facial image, a recognition weight of an image feature corresponding to each facial key point in the second sample facial image; An identity recognition result corresponding to the second sample facial image is obtained based on the image features corresponding to the second sample facial image, the image features corresponding to each facial key point in the second sample facial image, and the recognition weights of the image features corresponding to different facial key points in the second sample facial image.

7. The method according to claim 6, characterized in that The determining, based on the position confidence of each facial key point in the second sample facial image, the recognition weight of the image feature corresponding to each facial key point in the second sample facial image includes: Normalizing the position confidence of each facial key point in the second sample facial image to obtain a normalized position confidence of each facial key point in the second sample facial image; The normalized position confidence of each facial key point in the second sample facial image is mapped to a recognition weight of an image feature corresponding to each facial key point in the second sample facial image.

8. The method according to claim 6, characterized in that Providing the adjusted second sample face image to the face recognition model to train and adjust the face recognition model further includes: comparing the identity recognition result corresponding to the second sample face image with the identity tag information to obtain the recognition accuracy of the face recognition model; Determining whether the recognition accuracy of the face recognition model reaches a first accuracy threshold; If not, adjust the model parameters of the face recognition model until the recognition accuracy of the face recognition model reaches a first accuracy threshold.

9. The method according to claim 8, characterized in that The obtaining of the key point position prediction information of the sample facial image comprises: respectively obtaining the key point position prediction information of each facial key point in the sample facial image, and obtaining an error evaluation value, wherein the error evaluation value is used to represent an estimation of the prediction accuracy of the key point prediction model itself; In the step of training and adjusting the face recognition model, the weight of the role played by the second sample face image in adjusting the model parameters of the face recognition model is determined based on the error evaluation value of the second sample face image.

10. The method according to claim 8, characterized in that Also includes: If the recognition accuracy of the face recognition model reaches the first accuracy threshold, the current cycle ends and the next cycle begins.

11. The method according to claim 10, characterized in that The training requirements include at least: the recognition accuracy of the face recognition model reaches a second accuracy threshold, and the number of the sample face images reaches a predetermined number.

12. A model training device, characterized in that: include: An image processing unit is configured to input a sample facial image into a key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotated information and a second sample facial image with identity annotated information; obtaining the key point position prediction information of the sample facial image includes: obtaining key point position prediction information and an error evaluation value of each facial key point in the sample facial image through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; A first model adjustment unit is configured to perform training and adjustment on the key point prediction model based on the key point position prediction information and the key point position labeling information of the first sample facial image, including: performing training and adjustment on the key point prediction model based on the key point position prediction information of the first sample facial image, the key point position labeling information of the first sample facial image, an error evaluation value of the first sample facial image, and an actual error, wherein the actual error is a difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point; a second model adjustment unit, configured to adjust the second sample facial image based on the key point position prediction information of the second sample facial image, provide the adjusted second sample facial image to the facial recognition model, perform training adjustment on the facial recognition model, and determine whether the facial recognition model meets the training requirements; Among them, if the face recognition model does not meet the training requirements, the image processing unit, the first model adjustment unit and the second model adjustment unit are looped to execute the steps executed by each unit until the face recognition model meets the training requirements; each time the first model adjustment unit executes the corresponding step, it uses the key point prediction model that has undergone the latest training adjustment step.

13. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program of the model training method. After the device is powered on and the program of the model training method is run by the processor, the following steps are performed: Inputting a sample facial image into a key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotated information and a second sample facial image with identity annotated information; obtaining the key point position prediction information of the sample facial image includes: obtaining key point position prediction information and an error evaluation value of each facial key point in the sample facial image through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; Training and adjusting the key point prediction model based on the key point position prediction information and the key point position labeling information of the first sample facial image, including: training and adjusting the key point prediction model based on the key point position prediction information of the first sample facial image, the key point position labeling information of the first sample facial image, an error evaluation value of the first sample facial image, and an actual error, wherein the actual error is a difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point; adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model; The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

14. A storage medium, characterized in that A program storing a model training method is executed by a processor to perform the following steps: Inputting a sample facial image into a key point prediction model to obtain key point position prediction information of the sample facial image; the sample facial image includes a first sample facial image with key point position annotated information and a second sample facial image with identity annotated information; obtaining the key point position prediction information of the sample facial image includes: obtaining key point position prediction information and an error evaluation value of each facial key point in the sample facial image through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; Training and adjusting the key point prediction model based on the key point position prediction information and the key point position labeling information of the first sample facial image, including: training and adjusting the key point prediction model based on the key point position prediction information of the first sample facial image, the key point position labeling information of the first sample facial image, an error evaluation value of the first sample facial image, and an actual error, wherein the actual error is a difference between the key point position prediction information of each facial key point in the first sample facial image and the key point position labeling information of the corresponding key point; adjusting the second sample facial image according to the key point position prediction information of the second sample facial image, and providing the adjusted second sample facial image to a face recognition model to perform training and adjustment on the face recognition model; The above steps are executed in a loop until the training requirements of the face recognition model are met; each time the step of obtaining the predicted position information of the facial key points of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment steps is used.

15. A face recognition method, characterized in that: include: Obtaining a face image to be recognized; Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point prediction model and a target face recognition model; The target face recognition model and the target key point prediction model are obtained as follows: Input a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the obtaining of the key point position prediction information of the sample face image includes: obtaining the key point position prediction information and error evaluation value of each facial key point in the sample face image respectively through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image and the key point position annotation information, including: training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image and the key point position annotation information of the first sample face image The key point prediction model is trained and adjusted based on the set annotation information, the error evaluation value of the first sample face image and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample face image and the key point position annotation information of the corresponding key point; according to the key point position prediction information of the second sample face image, the second sample face image is adjusted, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target key point prediction model is obtained; each time the step of obtaining the facial key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment step is used.

16. The face recognition method according to claim 15, characterized in that: The method of obtaining an identity recognition result corresponding to the face image to be recognized by using the target key point prediction model and the target face recognition model includes: Inputting the face image to be recognized into a target key point prediction model to obtain key point position prediction information of the face image to be recognized; According to the key point position prediction information of the face image to be recognized, the face image to be recognized is adjusted, and the adjusted face image to be recognized is input into the target face recognition model to obtain the face recognition result corresponding to the face image to be recognized.

17. A face recognition device, characterized in that: include: A face image obtaining unit, used to obtain a face image to be identified; A recognition result obtaining unit, configured to obtain an identity recognition result corresponding to the face image to be recognized by using a target key point prediction model and a target face recognition model; Wherein, the target face recognition model and the target key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point position prediction information of the sample face image is obtained, including: obtaining the key point position prediction information and error evaluation value of each face key point in the sample face image respectively through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information, including: training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image The key point prediction model is trained and adjusted based on the key point position prediction information of the first sample face image, the key point position annotation information of the first sample face image, the error evaluation value of the first sample face image and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample face image and the key point position annotation information of the corresponding key point; according to the key point position prediction information of the second sample face image, the second sample face image is adjusted, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target key point prediction model is obtained; each time the step of obtaining the facial key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment step is used.

18. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program of the face recognition method. After the device is powered on and the program of the face recognition method is run by the processor, the following steps are performed: Obtaining a face image to be recognized; Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point prediction model and a target face recognition model; Among them, the target face recognition model and the target key point prediction model are obtained in the following manner: obtaining a first sample face image carrying key point position annotation information and a second sample face image carrying identity annotation information; inputting the first sample face image and the second sample face image into the key point prediction model, obtaining first predicted position information corresponding to the face key points in the first sample face image, and obtaining second predicted position information corresponding to the face key points in the second sample face image; adjusting the key point prediction model according to the first predicted position information and the key point position annotation information to obtain the adjusted face key point prediction model; adjusting the key point prediction model according to the second predicted position information. a second sample face image, and input the adjusted second sample face image into the face recognition model to obtain an identity recognition result corresponding to the second sample face image; adjust the face recognition model according to the identity recognition result and the identity labeling information; loop through the above steps, using the adjusted face key point prediction model obtained in the previous loop as the face key point prediction model of the current loop in each loop, until the face recognition model meets the training requirements, obtaining the target face recognition model, and obtaining the target key point prediction model; each time the step of obtaining the predicted face key point position information of the sample face image is performed, the key point prediction model that has undergone the latest training and adjustment steps is used; It also includes: obtaining key point position prediction information and error evaluation values ​​of each facial key point in the sample facial image through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; training and adjusting the key point prediction model according to the key point position prediction information of the first sample facial image, the key point position annotation information of the first sample facial image, the error evaluation value of the first sample facial image, and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample facial image and the key point position annotation information of the corresponding key point.

19. A storage medium, characterized in that: A program for face recognition method is stored and executed by a processor to perform the following steps: Obtaining a face image to be recognized; Obtaining an identity recognition result corresponding to the face image to be recognized using a target key point prediction model and a target face recognition model; The target face recognition model and the target key point prediction model are obtained as follows: Input a sample face image into a key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the obtaining of the key point position prediction information of the sample face image includes: obtaining the key point position prediction information and error evaluation value of each facial key point in the sample face image respectively through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image and the key point position annotation information, including: training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image and the key point position annotation information of the first sample face image The key point prediction model is trained and adjusted based on the set annotation information, the error evaluation value of the first sample face image and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample face image and the key point position annotation information of the corresponding key point; according to the key point position prediction information of the second sample face image, the second sample face image is adjusted, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed repeatedly until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target key point prediction model is obtained; each time the step of obtaining the facial key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment step is used.

20. A face recognition system, characterized in that: include: Platform server and user terminal; The platform server is used to obtain the face image to be recognized sent by the user terminal; Obtaining an identity recognition result corresponding to the face image to be recognized by using a target key point prediction model and a target face recognition model; and providing the identity recognition result to the user terminal; The user terminal is used to send the face image to be identified to the platform server; obtain the identity recognition result provided by the face image to be identified; and display the identity recognition result through a human-computer interaction page; Wherein, the target face recognition model and the target key point prediction model are obtained in the following manner: a sample face image is input into the key point prediction model to obtain key point position prediction information of the sample face image; the sample face image includes a first sample face image with key point position annotation information, and a second sample face image with identity annotation information; the key point position prediction information of the sample face image is obtained, including: obtaining the key point position prediction information and error evaluation value of each face key point in the sample face image respectively through the key point prediction model, wherein the error evaluation value is used to represent the key point prediction model's estimation of its own prediction accuracy; the key point prediction model is trained and adjusted according to the key point position prediction information of the first sample face image and the key point position annotation information, including: training and adjusting the key point prediction model according to the key point position prediction information of the first sample face image The key point prediction model is trained and adjusted based on the key point position prediction information of the first sample face image, the key point position annotation information of the first sample face image, the error evaluation value of the first sample face image and the actual error, wherein the actual error is the gap between the key point position prediction information of each facial key point in the first sample face image and the key point position annotation information of the corresponding key point; according to the key point position prediction information of the second sample face image, the second sample face image is adjusted, and the adjusted second sample face image is provided to the face recognition model to train and adjust the face recognition model; the above steps are executed in a loop until the training requirements of the face recognition model are met, the target face recognition model is obtained, and the target key point prediction model is obtained; each time the step of obtaining the facial key point prediction position information of the sample face image is executed, the key point prediction model that has undergone the latest training and adjustment step is used.

Images