Model training method, face key point detection method, electronic device and medium

Abstract

The embodiment of the application relates to the technical field of face detection, and discloses a model training method, a face key point detection method, electronic equipment and a medium. The model training method comprises the following steps: inputting a face image into a key point detection network to obtain a pose parameter, a 3D key point and a scale factor corresponding to the face in the face image; converting a preset standard 3D key point and a predicted 3D key point according to a camera internal parameter, a pre-pose parameter and the scale factor to obtain corresponding first 2D key points and second 2D key points; taking the distance between the first 2D key points and the real 2D key points of the face image as a first loss, the distance between the second 2D key points and the real 2D key points as a second loss, and the distance between the depth information of the predicted 3D key points and the depth information of the standard 3D key points as a third loss; and training the key point detection network based on the first loss, the second loss and the third loss.

Description

Technical Field

[0001] This application relates to the field of face detection technology, and in particular to a model training method, a facial landmark detection method, an electronic device, and a medium. Background Technology

[0002] When a face is occluded in an image or has a large pose angle, key point location detection is easily negatively affected. For example, when the mouth is occluded in a face image, it is impossible to directly obtain the location information of the mouth key points from the face image; when a face in a face image turns its head to the right, only the left half of the face is visible in the face image, and it is impossible to directly obtain the key point information of the right half of the face.

[0003] Currently, there are two main methods for keypoint detection based on deep learning: direct regression and heatmap prediction. Direct regression involves inputting a face image and pre-annotated facial keypoint information into a deep learning network, which then directly outputs the predicted keypoint information after training. Heatmap prediction, on the other hand, involves the network predicting a Gaussian heatmap for each keypoint on its corresponding feature map, using the center of the Gaussian heatmap within the feature map as the predicted keypoint location.

[0004] However, neither of these two methods fundamentally solves the problem of losing key point information in some facial regions due to large pose angles or occlusions, thus failing to obtain accurate facial key point location information. Summary of the Invention

[0005] The purpose of this application is to provide a model training method, a face key point detection method, an electronic device and a storage medium, which obtains accurate 2D key points based on the 3D key points, pose parameters and scale factors output by the key point detection network.

[0006] To address the aforementioned technical problems, this application provides a model training method, comprising: inputting a face image into a keypoint detection network to predict the pose parameters, 3D keypoints, and scale factor corresponding to the face in the face image; converting preset standard 3D keypoints and predicted 3D keypoints into corresponding first 2D keypoints and second 2D keypoints based on preset camera intrinsics, predicted pose parameters, and predicted scale factor; using the distance between the first 2D keypoint and the real 2D keypoint of the face image as a first loss, the distance between the second 2D keypoint and the real 2D keypoint as a second loss, and the distance between the depth information of the predicted 3D keypoint and the depth information of the standard 3D keypoint as a third loss; training the keypoint detection network based on the first loss, the second loss, and the third loss to obtain a trained keypoint detection network.

[0007] The embodiments of this application also provide a method for detecting facial landmarks, comprising: inputting a face image to be tested into a pre-trained landmark detection network to predict pose parameters, 3D landmarks, and scale factors corresponding to the face in the face image to be tested; converting the 3D landmarks into 2D landmarks of the face image to be tested based on the predicted pose parameters, the predicted scale factors, and preset camera intrinsic parameters; wherein, the landmark detection network is obtained by the model training method described in the above embodiments.

[0008] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the model training method mentioned in the above embodiments, or to perform the facial landmark detection method mentioned in the above embodiments.

[0009] The embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the model training method mentioned in the above embodiments, or is capable of executing the facial landmark detection method mentioned in the above embodiments.

[0010] The model training method provided in this application predicts the pose parameters, 3D keypoints, and scale factor of a face in a face image through a keypoint detection network. Based on preset camera intrinsics, predicted pose parameters, and predicted scale factor, standard 3D keypoints are converted into first 2D keypoints, and predicted 3D keypoints are converted into second 2D keypoints. Then, a first loss representing the distance between the first 2D keypoints and the real 2D keypoints is used to constrain the network to train accurate pose parameters and scale factor. A second loss representing the distance between the second 2D keypoints and the real 2D keypoints is used to further constrain the network to train accurate pose parameters, scale factor, and 3D keypoints. Finally, a third loss is used to constrain the network to train accurate depth information. Since 2D keypoint information of a face in an image is correlated with the face pose angle, and the shape of a face is symmetrical and has a specific spatial structure, the keypoint detection network of this application can predict 3D keypoint information and pose parameters from 2D planar face images. It also fully utilizes standard 3D keypoints to ensure accurate 3D keypoints even when dealing with face images with large pose angles. Furthermore, after predicting the pose parameters, this application can obtain the keypoint position information of a face in a 2D image at any pose angle by changing the pose parameters. Attached Figure Description

[0011] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0012] Figure 1 This is a flowchart of the model training method provided in the embodiments of this application;

[0013] Figure 2 This is a flowchart of the facial landmark detection method provided in the embodiments of this application;

[0014] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable the reader to better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0016] The following provides an example illustrating the implementation details of model training in this embodiment. The following implementation details are provided for ease of understanding only and are not essential for implementing this solution.

[0017] The embodiments of this application relate to a model training method, such as... Figure 1 As shown, it includes:

[0018] Step 101: Input the face image into the key point detection network to predict the pose parameters, 3D key points and scale factor of the face in the face image.

[0019] In this embodiment, when training the key point detection network, the face images in the training set can be obtained by taking pictures with different cameras, or by taking pictures of different faces with different cameras at different shooting distances and pose angles.

[0020] A keypoint detection network can obtain the pose parameters, 3D keypoints, and scale factor for each face in multiple face images. The pose parameters include rotation and translation matrices. The predicted 3D keypoints include their x-coordinates, y-coordinates, and depth values; the x-coordinates and y-coordinates are generally obtained using the world coordinate system. The scale factor of a face image is related to the camera that captured it; generally, face images captured by the same camera have the same scale factor.

[0021] Step 102: Based on the preset camera intrinsic parameters, predicted pose parameters and predicted scale factor, convert the preset standard 3D key points and predicted 3D key points to obtain the corresponding first 2D key points and second 2D key points.

[0022] Specifically, the preset camera intrinsic parameters include the camera focal length f. x f y Optical center coordinates c x c y Camera intrinsic parameters can be selected based on experience using standardized empirical statistical values, or actual parameters can be used directly, such as the camera intrinsic parameters corresponding to the camera used when acquiring standard 3D keypoints. The first and second 2D keypoints are keypoints in the image coordinate system, including the x and y coordinates of the 2D keypoints.

[0023] In one embodiment, standard 3D key points are obtained through the following steps: acquiring multiple face depth maps generated by the same depth camera capturing different faces at the same shooting distance and pose angle; acquiring face key points in the multiple face depth maps according to a preset key point detection algorithm; and averaging the information of the same key point in the different face depth maps to obtain standard 3D key points.

[0024] Specifically, the faces in multiple face depth maps can be faces corresponding to face images in the training set, or faces not corresponding to face images in the training set. Facial keypoints (such as eyebrow keypoints, nose keypoints, eye keypoints, mouth keypoints, etc.) are detected in the face depth maps. The information (x-coordinate, y-coordinate, and depth value) of the same keypoint in multiple face depth maps is averaged to obtain the standard 3D keypoint. This involves summing and averaging the x-coordinates of the same keypoint in different face depth maps to obtain the x-coordinate of the standard 3D keypoint; summing and averaging the y-coordinates of the same keypoint in different face depth maps to obtain the y-coordinate of the standard 3D keypoint; and summing and averaging the depth values ​​of the same keypoint in different face depth maps to obtain the depth value of the standard 3D keypoint.

[0025] In other words, the standard 3D key point in this application is a 3D key point of a standard face (or average face, general face).

[0026] It's important to note that for each face image in the training set, there is a corresponding predicted pose parameter, a predicted 3D keypoint, and a predicted scale factor. For each face image, the predicted 3D keypoint is transformed into a second 2D keypoint using the corresponding pose parameter, scale factor, and preset camera intrinsics. Similarly, for each face image in the training set, the standard 3D keypoint is transformed into a first 2D keypoint using the corresponding pose parameter and scale factor. In other words, each face image corresponds to a different 3D keypoint, pose parameter, and scale factor, but all correspond to a unique standard 3D keypoint.

[0027] In one embodiment, the first 2D keypoint and the second 2D keypoint are obtained using the following formula:

[0028]

[0029]

[0030]

[0031]

[0032] in, This refers to the 3D information of the nth keypoint in the b-th face image obtained through standard 3D keypoint conversion. for The first two dimensions of the information represent the first 2D key points of the b-th face image. To obtain the 3D information of the nth keypoint in the b-th face image through the predicted 3D keypoint transformation, for The first two dimensions of the information represent the second 2D keypoint of the b-th face image, cut(*) is the cropping function, f is the predicted scale factor, C is the preset camera intrinsic parameter, R is the rotation matrix in the pose parameters predicted by the keypoint detection network, T is the translation matrix in the pose parameters predicted by the keypoint detection network, Q is the 3D keypoint predicted by the keypoint detection network, and S is the standard 3D keypoint.

[0033] During calculation, the camera intrinsic C is a 3×3 matrix, the translation matrix T is a 3×1 matrix, the rotation matrix R is a 3×3 matrix, the predicted 3D keypoints Q are a 3×N matrix, and the scale factor f is a scalar.

[0034] It is understandable that the key point location information of a face in a face image is essentially formed by the face in the real world being rotated and translated in 3D space and then projected onto a planar image. Therefore, the 2D key points of a face image are obtained by rotating the predicted 3D key points through the rotation matrix R and translating them through the translation matrix T, and then by projecting them through the camera intrinsic parameters C and scaling them through the predicted scale factor.

[0035] Step 103: The distance between the first 2D key point and the real 2D key point of the face image is used as the first loss, the distance between the second 2D key point and the real 2D key point is used as the second loss, and the distance between the depth information of the predicted 3D key point and the depth information of the standard 3D key point is used as the third loss.

[0036] In this embodiment, since the first 2D keypoint is calculated from standard 3D keypoints using predicted pose parameters and predicted scale factors, a first loss characterizing the distance between the first 2D keypoint and the true 2D keypoint allows the keypoint detection network to learn the transformation process from standard 3D keypoints to the first 2D keypoint, thereby training and constraining the predicted pose parameters and predicted scale factors. Simultaneously, since the second 2D keypoint is calculated from predicted 3D keypoints using predicted pose parameters and predicted scale factors, a second loss characterizing the distance between the second 2D keypoint and the true 2D keypoint allows the network to be trained to constrain accurate 3D keypoints, pose parameters, and scale factors. The depth information of the predicted 3D keypoints is used to constrain the training through a third loss.

[0037] It is understandable that different faces have different depth information such as eye depth and nose height, but they will not be very different from standard faces. Therefore, the key point detection network can learn the depth information of predicted 3D key points by using the depth information of standard 3D key points.

[0038] It's important to note that the location of facial keypoints in an image is interrelated with the facial pose angle, and the facial pose angle, in turn, affects the depth values ​​of the keypoints. For example, the depth value of the nose keypoint for a frontal face is always less than the depth value of the nose keypoint for the same face in a profile view. In other words, 2D keypoints, pose parameters, and 3D keypoints are interconnected and mutually influential. Therefore, the first loss (constraining pose parameters and scale factor), the second loss (constraining 3D keypoints, pose parameters, and scale factor), and the third loss (depth information of 3D keypoints) compete with each other during training.

[0039] For example, when the first loss reaches its minimum, the optimal pose parameters and scale factor are obtained. However, these optimal pose parameters and scale factor do not necessarily minimize the second loss. Similarly, when the third loss reaches its minimum, the optimal depth value is obtained. However, the pose parameters and keypoints corresponding to this optimal depth value do not necessarily minimize the losses of both the second and first losses. Therefore, this application continuously optimizes and adjusts the training parameters of the keypoint detection network to make the first loss L1, the second loss L2, and the third loss L3 relatively optimal, while minimizing the total loss (L = L1 + L2 + L3).

[0040] In addition, it is worth mentioning that since the shape of the human face is symmetrical and has a specific spatial structure, this application can predict or correct missing key points by using the pose parameters of the face and standard 3D key points when facing the adverse effects of large pose angles or occlusion.

[0041] In one embodiment, the loss function for the first loss is constructed using the following formula:

[0042]

[0043] Where α1 is a hyperparameter greater than 0, and B is the number of face images. Let n be the nth keypoint among the first 2D keypoints in the b-th face image. Let A be the nth keypoint among the real 2D keypoints of the b-th face image, and let dist(A, B) be the distance between A and B.

[0044] The loss function for the second loss is constructed using the following formula:

[0045]

[0046] Where α2 is a hyperparameter greater than 0, and B is the number of face images. Let n be the nth keypoint in the second 2D keypoints of the b-th face image. Let A be the nth keypoint among the real 2D keypoints of the b-th face image, and let dist(A, B) be the distance between A and B.

[0047] The loss function for the third loss is constructed using the following formula:

[0048]

[0049] Where α3 is a hyperparameter greater than 0, and B is the number of face images. Let n be the depth value of the nth keypoint among the predicted 3D keypoints in the b-th face image. Let A be the depth value of the nth keypoint in the standard 3D keypoints of the b-th face image, and let dist(A, B) be the distance between A and B.

[0050] In this embodiment, the distance represented by dist(A,B) in the first loss, second loss and third loss can be Euclidean distance, Mahalanobis distance, cosine distance, etc., and is not limited here.

[0051] Step 104: Train the keypoint detection network based on the first loss, the second loss, and the third loss to obtain the trained keypoint detection network.

[0052] Specifically, by continuously optimizing and adjusting α1, α2, and α3 to minimize L = L1 + L2 + L3, the keypoint detection network training converges. During training, conventional deep learning gradient descent optimization methods can be used, such as determining whether the training has reached a specified number of iterations or whether the total loss function no longer decreases significantly.

[0053] Understandably, before training, a keypoint detection network does not know how to obtain the corresponding 3D keypoints, pose parameters, and scale factors from a face image. Therefore, this application uses a first loss function to allow the network to learn the transformation process from standard 3D keypoints to first 2D keypoints, i.e., to initially train and constrain the pose parameters and scale factors; at the same time, it uses a second loss function to allow the network to learn the transformation process from predicted 3D keypoints to second 2D keypoints, i.e., to initially train and constrain the 3D keypoints and further train and constrain the pose parameters and scale factors; and at the same time, it uses a third loss function to further train and constrain the 3D keypoints.

[0054] The key point detection network provided in this embodiment can be any one of the existing deep learning networks, such as convolutional networks, transformer structures, teacher-student models, or combinations thereof. This embodiment does not specifically limit the model.

[0055] The model training method provided in this application predicts the pose parameters, 3D keypoints, and scale factor of a face in a face image through a keypoint detection network. Based on preset camera intrinsics, predicted pose parameters, and predicted scale factor, standard 3D keypoints are converted into first 2D keypoints, and predicted 3D keypoints are converted into second 2D keypoints. Then, a first loss representing the distance between the first 2D keypoints and the real 2D keypoints is used to constrain the network to train accurate pose parameters and scale factor. A second loss representing the distance between the second 2D keypoints and the real 2D keypoints is used to further constrain the network to train accurate pose parameters, scale factor, and 3D keypoints. Finally, a third loss is used to constrain the network to train accurate depth information. Since 2D keypoint information of a face in an image is correlated with the face pose angle, and the shape of a face is symmetrical and has a specific spatial structure, the keypoint detection network of this application can predict 3D keypoint information and pose parameters from 2D planar face images. It also fully utilizes standard 3D keypoints to ensure accurate 3D keypoints even when dealing with face images with large pose angles. Furthermore, after predicting the pose parameters, this application can obtain the keypoint position information of a face in a 2D image at any pose angle by changing the pose parameters.

[0056] The embodiments of this application relate to a facial landmark detection method, such as... Figure 2 As shown, it includes:

[0057] Step 201: Input the face image to be tested into the pre-trained keypoint detection network to predict the pose parameters, 3D keypoints and scale factor of the face in the face image to be tested.

[0058] Step 202: Based on the predicted pose parameters, predicted scale factor and preset camera intrinsic parameters, the 3D key points are converted to obtain the 2D key points of the face image to be tested.

[0059] In this embodiment, the key point detection network is obtained through the model training method described in the above implementation. The preset camera intrinsic parameters are the same as those used in the model training stage. The calculation process of the 2D key points of the face image to be tested is the same as the calculation process of the second 2D key points, and will not be described in detail here.

[0060] In addition, after converting the 3D key points to obtain the 2D key points of the face image to be tested, the method further includes: adjusting the pose parameters according to the preset target pose angle to obtain the pose parameters corresponding to the target pose angle; and converting the 3D key points to obtain the 2D key points of the face image corresponding to the target pose angle according to the predicted scale factor, the preset camera intrinsic parameters and the pose parameters corresponding to the target pose angle.

[0061] In other words, this application can predict the key point location information of a face at any pose angle in a 2D face image from any 2D planar face image. That is, it calculates the pose parameters corresponding to the target pose angle, and then transforms the 3D key points according to the pose parameters corresponding to the target pose angle to obtain the 2D key points corresponding to the target pose angle.

[0062] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0063] The embodiments of this application relate to an electronic device, such as... Figure 3 As shown, it includes:

[0064] At least one processor 301; and a memory 302 communicatively connected to at least one processor 301; wherein the memory 302 stores instructions executable by at least one processor 301, the instructions being executed by at least one processor 301 to enable at least one processor 301 to perform the model training method mentioned in the above embodiments, or to perform the face key point detection method mentioned in the above embodiments.

[0065] The electronic device includes: one or more processors 301 and a memory 302. Figure 3 Taking a processor 301 as an example, the processor 301 and the memory 302 can be connected via a bus or other means. Figure 3 Taking a bus connection as an example, memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 301 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 302, thereby implementing the aforementioned model training method or facial landmark detection method.

[0066] Memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 302 may optionally include memory remotely located relative to processor 301, and these remote memories can be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] One or more modules are stored in memory 302. When executed by one or more processors 301, they perform the model training method in any of the above embodiments, or can perform the facial landmark detection method mentioned in the above embodiments.

[0068] The above-mentioned products can perform the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects of performing the methods. For technical details not described in detail in this embodiment, please refer to the methods provided in the embodiments of this application.

[0069] Embodiments of this application relate to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the above-described method embodiments.

[0070] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0071] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0072] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application.

Claims

1. A model training method, characterized in that, include: A face image is input into a keypoint detection network to predict the pose parameters, 3D keypoints, and scale factor of the face in the face image; wherein, the pose parameters include rotation and translation matrices, the predicted 3D keypoints include the x-coordinate, y-coordinate, and depth value of the 3D keypoints, and the scale factor is a scalar. Based on the preset camera intrinsics, predicted pose parameters, and predicted scale factor, the preset standard 3D key points and predicted 3D key points are converted to obtain the corresponding first 2D key points and second 2D key points. The standard 3D key points are obtained through the following steps: acquiring multiple face depth maps generated by the same depth camera capturing different faces at the same shooting distance and pose angle; acquiring face key points in the multiple face depth maps according to a preset key point detection algorithm; and averaging the information of the same key point in the different face depth maps to obtain the standard 3D key points. The first 2D keypoint and the second 2D keypoint are obtained using the following formula: ; ; ; ; in, This refers to the 3D information of the nth keypoint in the b-th face image obtained through standard 3D keypoint conversion. for The first two dimensions of information in the image, namely the first 2D key points of the b-th face image; This refers to the 3D information of the nth keypoint in the b-th face image obtained from the predicted 3D keypoint transformation. for The first two dimensions of information in the b-th face image are the second 2D key points; cut(*) is the cutting function for extracting the first two dimensions of information, f is the predicted scale factor, C is the preset camera intrinsic parameter, R is the rotation matrix, T is the translation matrix, Q is the predicted 3D key point, and S is the standard 3D key point. The distance between the first 2D keypoint and the real 2D keypoint of the face image is used as the first loss, the distance between the second 2D keypoint and the real 2D keypoint is used as the second loss, and the distance between the depth information of the predicted 3D keypoint and the depth information of the standard 3D keypoint is used as the third loss. The first loss is used to constrain the accuracy of the pose parameters and scale factor output by the keypoint detection network, the second loss is used to simultaneously constrain the accuracy of the pose parameters, scale factor, and 3D keypoints output by the keypoint detection network, and the third loss is used to constrain the depth information of the predicted 3D keypoints. The keypoint detection network is trained based on the first loss, the second loss, and the third loss to obtain a trained keypoint detection network; a total loss function is constructed based on the first loss, the second loss, and the third loss, and the total loss function is: L = L1 + L2 + L3; where L is the total loss, L1 is the first loss, L2 is the second loss, and L3 is the third loss; during training, the hyperparameters corresponding to the first loss, the second loss, and the third loss are adjusted to make the total loss function converge.

2. The model training method according to claim 1, characterized in that, The loss function for the first loss is constructed using the following formula: ; Where α1 is a hyperparameter greater than 0, and B is the number of face images. Let n be the nth keypoint among the first 2D keypoints in the b-th face image. Let dist(A, B) be the nth keypoint among the real 2D keypoints of the bth face image, and let dist(A, B) be the Euclidean distance, Mahalanobis distance, or cosine distance between A and B.

3. The model training method according to claim 1, characterized in that, The loss function for the second loss is constructed using the following formula: ; Where α2 is a hyperparameter greater than 0, and B is the number of face images. Let n be the nth keypoint in the second 2D keypoints of the b-th face image. Let dist(A, B) be the nth keypoint among the real 2D keypoints of the bth face image, and let dist(A, B) be the Euclidean distance, Mahalanobis distance, or cosine distance between A and B.

4. The model training method according to claim 1, characterized in that, The loss function for the third loss is constructed using the following formula: ; Where α3 is a hyperparameter greater than 0, and B is the number of face images. Let n be the depth value of the nth keypoint among the predicted 3D keypoints in the b-th face image. Let be the depth value of the nth keypoint in the standard 3D keypoints of the b-th face image, and dist(A, B) be the Euclidean distance, Mahalanobis distance, or cosine distance between A and B.

5. A method for detecting facial landmarks, characterized in that, include: The face image to be tested is input into a pre-trained key point detection network to predict the pose parameters, 3D key points and scale factor of the face in the face image to be tested. Based on the predicted pose parameters, the predicted scale factor, and the preset camera intrinsic parameters, the 3D key points are converted to obtain the 2D key points of the face image to be tested. The keypoint detection network is obtained by the model training method as described in any one of claims 1-4.

6. The key point detection method according to claim 5, characterized in that, The 3D key points are converted. After obtaining the 2D key points of the face image to be tested, the method further includes: The pose parameters are adjusted according to the preset target pose angle to obtain the pose parameters corresponding to the target pose angle; the 3D key points are transformed according to the predicted scale factor, the preset camera intrinsic parameters and the pose parameters corresponding to the target pose angle to obtain the 2D key points of the face image corresponding to the target pose angle.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the model training method as described in any one of claims 1 to 4, or to perform the facial landmark detection method as described in any one of claims 5 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model training method of any one of claims 1 to 4, or the facial landmark detection method of any one of claims 5 to 6.

Images