A face pose estimation method, device and medium

Abstract

The present application relates to the technical field of face recognition, and specifically provides a face pose estimation method and device and medium, aiming to solve the technical problem of how to make the face pose estimation more accurate. To this end, the present application obtains face picture data to be measured, predicts the face picture data to be measured through a preset face pose estimation model, and obtains face pose estimation data, wherein the face pose estimation model includes a regression branch and a classification branch, the regression branch is trained with the aid of the classification branch, so that the trained regression branch can predict face picture data. Through the above configuration mode, the regression branch is optimized to predict face picture data, which can improve the accuracy of face pose estimation.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, specifically providing a facial pose estimation method, device, and medium. Background Technology

[0002] Face pose estimation is a key technology in face recognition and analysis, forming the foundation for face recognition, facial expression analysis, and 3D face reconstruction. Face pose estimation helps reconstruct face models, understand face pose attributes, evaluate face image quality, and estimate eye gaze. In recent years, the application of face recognition and understanding technologies has become relatively mature, and face pose estimation technology has been widely used.

[0003] Currently, face pose estimation is essentially the prediction of 3D Euler angles. Predicting 3D spatial angles from 2D images is an underconstrained problem. Therefore, inferring 3D spatial angles from 2D information is challenging. Due to the insufficiency of 2D information and the inaccuracy of angle annotation, designing a reasonable model to achieve more accurate face pose estimation is a technical problem that needs to be solved.

[0004] Accordingly, there is a need in this field for a face pose estimation scheme to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, the present invention is proposed to provide a solution, or at least a partial solution, to the technical problem of how to estimate face pose more accurately.

[0006] In a first aspect, the present invention provides a face pose estimation method, the method comprising:

[0007] Obtain the facial image data of the person to be tested;

[0008] The face image data to be tested is predicted by a preset face pose estimation model to obtain face pose estimation data.

[0009] The face pose estimation model includes a regression branch and a classification branch. The classification branch assists the regression branch in training so that the trained regression branch can predict the face image data.

[0010] In one technical solution of the aforementioned face pose estimation method, the step of training the regression branch by assisting the classification branch includes:

[0011] Based on the face image data used for training, obtain the facial spatial semantic features of the face image data;

[0012] For each iteration of the training, based on the facial spatial semantic features, the classification branch is applied to perform facial pose classification prediction to obtain the classification prediction result;

[0013] Based on the facial spatial semantic features, the regression branch is applied to perform facial pose regression prediction to obtain the regression prediction result;

[0014] Based on the classification prediction results and the regression prediction results, the first regression loss of the regression branch is obtained;

[0015] Based on the first regression loss, backpropagation is performed to update the model parameters of the regression branch, thereby achieving the training of the regression branch.

[0016] In one technical solution of the aforementioned face pose estimation method, obtaining the first regression loss of the regression branch based on the classification prediction result and the regression prediction result includes:

[0017] Based on the classification prediction results and the ground truth facial pose data of the facial image data, the classification loss of the classification branch is obtained;

[0018] Based on the regression prediction results and the true face pose data, the second regression loss of the regression branch is obtained;

[0019] The first regression loss is obtained based on the classification loss and the second regression loss.

[0020] In one technical solution of the above-mentioned face pose estimation method, the classification branch includes multiple classification sub-branches with different quantization precisions; the classification prediction result includes multiple sub-branch classification prediction results.

[0021] The application of the classification branch to classify and predict face pose, and the resulting classification prediction, includes:

[0022] For each sub-branch, the sub-branch is applied to perform face pose classification prediction, and the sub-branch classification prediction result of the sub-branch is obtained.

[0023] The soft-argmax algorithm is applied to obtain the classification prediction result based on the classification prediction results of multiple sub-branches.

[0024] In one technical solution of the aforementioned face pose estimation method, obtaining the first regression loss based on the classification loss and the second regression loss includes:

[0025] The first regression loss is obtained by weighting the classification loss and the second regression loss.

[0026] In one technical solution of the above-mentioned face pose estimation method, the step of training the regression branch by assisting the classification branch further includes:

[0027] Based on the classification prediction result of each sub-branch and the ground truth data of the face pose, obtain the sub-branch classification loss of the corresponding classification sub-branch;

[0028] Backpropagation is performed based on the classification loss of each sub-branch to update the model parameters of the corresponding classification sub-branch.

[0029] In one technical solution of the aforementioned face pose estimation method, the step of predicting the face image data to be tested using a preset face pose estimation model to obtain face pose estimation data includes:

[0030] Based on the face image data to be tested, obtain the facial spatial semantic features of the face image data;

[0031] The facial spatial semantic features are input into the facial pose estimation model to obtain the facial pose estimation data.

[0032] In one technical solution of the aforementioned face pose estimation method, the step of acquiring the face spatial semantic features of the face image data includes:

[0033] ROI region extraction is performed on the face image data to obtain the face ROI region in the face image data;

[0034] When the face image data is used for training, the face ROI region is augmented to obtain the augmented face ROI region; based on the augmented face ROI region, feature extraction is performed to obtain the face spatial semantic features.

[0035] When the face image data is the face image data to be tested, feature extraction is performed based on the face ROI region to obtain the face spatial semantic features.

[0036] In a second aspect, a control device is provided, comprising at least one processor and at least one storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the face pose estimation method described in any of the above-described technical solutions.

[0037] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the face pose estimation method described in any of the above-described technical solutions.

[0038] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0039] In implementing the technical solution of this invention, the present invention acquires face image data to be tested, and uses a preset face pose estimation model to predict the face image data to obtain face pose estimation data. The face pose estimation model includes a regression branch and a classification branch. The classification branch assists the regression branch in training, enabling the trained regression branch to predict the face image data. Through this configuration, the regression branch, used for predicting continuous values, has higher precision, while the classification branch, used for predicting discrete values, has relatively lower precision but higher resolution accuracy. By fusing the regression and classification branches in the model results, and using the classification branch to assist the regression branch in training, the accuracy of the regression branch and the robustness of the classification branch can be combined, thereby optimizing the process of the regression branch predicting the face image data and effectively improving the accuracy of face pose estimation. Attached Figure Description

[0040] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0041] Figure 1 This is a schematic diagram of the main steps of a face pose estimation method according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of three angles of facial pose data according to one embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the main steps in training a classification branch-assisted regression branch according to one embodiment of the present invention. Detailed Implementation

[0044] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0045] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0046] Here we will first explain some of the terms involved in this invention.

[0047] The face ROI (Region of Interest) is the bounding box of the area where the face is located, usually represented by a rectangle. The coordinates of the top left corner of the rectangle, combined with the width and height of the rectangle, can determine the face ROI.

[0048] Face Euler angles: relative to the image coordinate system (right-handed coordinate system), consisting of three angles, each limited to the range of -90° to +90°.

[0049] 1. Yaw: The angle of the left and right sides of a person's face.

[0050] 2. Pitch: The angle at which a person's face is tilted up or down.

[0051] 3. Roll (roll angle): The angle at which a person's head tilts to the left or right.

[0052] Regression branch is suitable for predicting non-fixed continuous values, while classification branch is suitable for predicting fixed discrete values.

[0053] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a face pose estimation method according to an embodiment of the present invention. Figure 1 As shown, the face pose estimation method in this embodiment of the invention mainly includes the following steps S101-S102.

[0054] Step S101: Obtain the face image data of the person to be tested.

[0055] In this embodiment, face image data can be acquired for face pose estimation.

[0056] In one implementation, the facial image data to be tested can be collected by electronic devices such as cameras, video cameras, or mobile phones.

[0057] In one implementation, the face image data to be tested includes more than just face data; therefore, other data besides the face must be removed before predicting the face image data to be tested.

[0058] Step S102: Predict the face image data to be tested using a preset face pose estimation model to obtain face pose estimation data. The face pose estimation model includes a regression branch and a classification branch. The classification branch is used to assist the regression branch in training so that the trained regression branch can predict the face image data.

[0059] In this embodiment, the face pose estimation model includes a classification branch and a regression branch. The classification branch is used for training, and the trained regression branch is used to predict face image data to obtain face pose estimation data.

[0060] In one implementation, see Appendix Figure 2 , Figure 2 This is a schematic diagram of three angles representing face pose estimation data according to one embodiment of the present invention. Figure 2 As shown, the facial pose estimation data includes facial data from three angles. The first is Yaw: the angle of the face to the left and right. The second is Pitch: the angle of the face tilting up and down. The third is Roll: the angle of the face tilting left and right.

[0061] Based on steps S101-S102 above, this embodiment of the invention acquires the face image data to be tested, and uses a preset face pose estimation model to predict the face image data to obtain face pose estimation data. The face pose estimation model includes a regression branch and a classification branch. The classification branch assists the regression branch in training, enabling the trained regression branch to predict the face image data. Through this configuration, the regression branch, used for predicting continuous values, has higher precision, while the classification branch, used for predicting discrete values, has relatively lower precision but higher resolution accuracy. By fusing the regression and classification branches in the model results, and using the classification branch to assist the regression branch in training, the accuracy of the regression branch and the robustness of the classification branch can be combined, thereby optimizing the process of the regression branch predicting the face image data and effectively improving the accuracy of face pose estimation.

[0062] The following is a further explanation of step S102.

[0063] In one embodiment of the present invention, step S102 may include steps S1021 to S1022:

[0064] Step S1021: Based on the face image data to be tested, obtain the face spatial semantic features of the face image data.

[0065] Step S1022: Input the face spatial semantic features into the face pose estimation model to obtain face pose estimation data.

[0066] In this embodiment, a deep model is used to extract the spatial semantic features of the face image data to be tested, which are then used for face pose prediction. The deep models used in this method include, but are not limited to, ResNet, MobileNet, GhostNet, and ShuffleNet.

[0067] In one embodiment, step S1021 can be further configured to: extract the ROI region from the face image data to obtain the face ROI region in the face image data.

[0068] In this embodiment, the ROI region of the face image data can be extracted by cropping, and then the ROI region of the extracted face image data can be adjusted by resizing, so that the size of the ROI region of the face image data is converted into the model input size.

[0069] In one embodiment of the present invention, training by using a classification branch to assist a regression branch can be achieved through the following steps S201 to S205:

[0070] Step S201: Obtain the facial spatial semantic features of the facial image data based on the facial image data used for training.

[0071] Step S202: For each iteration of training, based on the semantic features of the face space, apply the classification branch to predict the face pose and obtain the classification prediction result.

[0072] Step S203: Based on the semantic features of the face space, apply the regression branch to perform regression prediction of the face pose and obtain the regression prediction result.

[0073] Step S204: Based on the classification prediction results and the regression prediction results, obtain the first regression loss of the regression branch.

[0074] Step S205: Based on the first regression loss, perform backpropagation to update the model parameters of the regression branch, thereby achieving the training of the regression branch.

[0075] In this embodiment, refer to the appendix. Figure 3 , Figure 3 This is a schematic diagram illustrating the main steps of training a classification branch-assisted regression branch according to one embodiment of the present invention. Figure 3 As shown, during the training process, the face image data used for training is input into the regression branch and the classification branch respectively. The classification branch obtains the classification prediction result, and the regression branch obtains the regression prediction result. Based on the classification prediction result and the regression prediction result, the first regression loss of the regression branch is obtained. The first regression loss is backpropagated to update the model parameters of the regression branch, so as to realize the training of the regression branch.

[0076] In this embodiment, step S201 may include steps S2011 to S2012:

[0077] Step S2011: Extract the ROI region from the face image data used for training to obtain the face ROI region in the face image data.

[0078] Step S2012: Perform data augmentation on the face ROI region to obtain the augmented face ROI region; based on the augmented face ROI region, perform feature extraction to obtain the face spatial semantic features.

[0079] In this embodiment, the face image data used for training is augmented to increase the learning probability of hard samples and optimize complex face scenes such as blurry, dimly lit, and faces wearing masks or glasses. This embodiment employs data augmentation methods including, but not limited to, Gaussian blur, motion blur, extreme brightness, and random erasure for targeted optimization. For blurry images, Gaussian blur and motion blur are used. For dimly lit images, extreme brightness simulation is used. For faces wearing masks or glasses, random erasure and mask simulation are used. The augmented face ROI regions are then used to extract features, obtaining facial spatial semantic features for training the face pose estimation model.

[0080] In one implementation, the face image data used for training is a single-target face image, with each image containing one face, i.e., a face ROI region. The face ROI region contains three Euler angle values ​​labeled, i.e., ground truth face pose data (including rotation angles in the corresponding directions of yaw, pitch, and roll), with each angle limited to the range of -90 to +90 degrees.

[0081] In this embodiment, step S204 may include steps S2041 to S2043:

[0082] Step S2041: Based on the classification prediction results and the ground truth facial pose data of the face image data, obtain the classification loss of the classification branch.

[0083] Step S2042: Based on the regression prediction results and the ground truth face pose data, obtain the second regression loss of the regression branch.

[0084] Step S2043: Obtain the first regression loss based on the classification loss and the second regression loss.

[0085] In this embodiment, such as Figure 3 As shown, the classification loss can be obtained based on the classification prediction results and the ground truth face pose data; the second regression loss can be obtained based on the regression prediction results and the ground truth face pose data; and the first regression loss can be obtained based on the classification loss and the second regression loss.

[0086] In one implementation, the least squares error (MSE) loss can be used to calculate the regression prediction results and the ground truth face pose data to obtain the second regression loss of the regression branch.

[0087] In one implementation, the classification loss of the classification branch can be obtained by calculating the classification prediction results and the ground truth face pose data of the face image data using cross entropy loss.

[0088] In one implementation, step S2043 may be further configured as follows:

[0089] The classification loss and the second regression loss are weighted and calculated to obtain the first regression loss.

[0090] In this embodiment, weights A and B (A+B=1, A>0, B>0) can be set for the classification loss and the second regression loss respectively. The classification loss and the second regression loss can be multiplied by their corresponding weights A and B and then added together to obtain the first regression loss.

[0091] In this embodiment, the classification branch includes multiple classification sub-branches with different quantization precisions; the classification prediction result includes the classification prediction results of multiple sub-branches, and step S202 may include steps S2021 to S2022:

[0092] Step S2021: For each sub-branch, apply the sub-branch classification to predict the face pose and obtain the sub-branch classification prediction result of the sub-branch.

[0093] Step S2022: Apply the soft-argmax algorithm to obtain the classification prediction result based on the classification prediction results of multiple sub-branches.

[0094] In this embodiment, the classification prediction results of multiple sub-branches can be aggregated to obtain a classification prediction result. Since the classification prediction results of each sub-branch are discontinuous, the soft-argmax algorithm is applied to calculate the expected probability of prediction results with different quantization precisions to obtain the classification prediction result. Furthermore, since the soft-argmax algorithm is differentiable, the obtained classification prediction result can be differentiated, transforming discrete values ​​into continuous values. The transformed continuous values ​​can then be used with ground truth face pose data to obtain the classification loss. The classification loss and the second regression loss are then weighted to obtain the first regression loss, thereby enabling backpropagation using the first regression loss to update the model parameters of the regression branch.

[0095] like Figure 3 As shown, the classification branch can include three sub-branches with different quantization accuracies: 1°, 3°, and 9°. For a 1-degree quantization accuracies, continuous angle values ​​in the range of -90° to 90° are divided into 180 classes (-180, -179, -178), i.e., 180 sub-branches. For a 3-degree quantization accuracies, they are divided into 60 classes (-180, -177, -174...), i.e., 60 sub-branches. For a 9-degree quantization accuracies, they are divided into 20 classes (-180, -171, -162...), i.e., 20 sub-branches. By inputting the face image data used for training into the three sub-branches with different quantization accuracies, the classification prediction results of the three sub-branches are obtained.

[0096] In one implementation, the classification prediction results of the three sub-branches can be obtained by the soft-argmax algorithm, and the classification prediction results are the expectations of the classification prediction results of the three sub-branches.

[0097] It should be noted that for different quantization progress levels, as the quantization accuracy decreases, the number of quantization categories decreases, making the quantization sub-branches easier to predict and more robust. Those skilled in the art can set the number of quantization sub-branches and the quantization accuracy of each quantization sub-branches according to the needs of actual applications.

[0098] In one implementation, the step of training by using the classification branch to assist the regression branch may further include the following steps S206 to S207:

[0099] Step S206: Based on the classification prediction results of each sub-branch and the ground truth data of face pose, obtain the sub-branch classification loss of the corresponding classification sub-branch.

[0100] Step S207: Perform backpropagation based on the classification loss of each sub-branch, and update the model parameters of the corresponding classification sub-branch.

[0101] In this embodiment, the sub-branch classification loss of the corresponding sub-branch can be obtained by using cross entropy loss to calculate the classification prediction results of each sub-branch and the ground truth data of face pose.

[0102] In one implementation, the classification loss of each sub-branch can be backpropagated on the model of the corresponding classification sub-branch to update the model parameters of the corresponding classification sub-branch, thereby optimizing the model of the branch.

[0103] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0104] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0105] Furthermore, the present invention also provides a control device. In one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the face pose estimation method of the above-described method embodiments, and the processor can be configured to execute the program in the storage device. The program includes, but is not limited to, a program for executing the face pose estimation method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This control device can be a control device device comprising various electronic devices.

[0106] In embodiments of the present invention, the control device may be a control device device comprising various electronic devices. In some possible implementations, the control device may include multiple storage devices and multiple processors. The program executing the face pose estimation method of the above method embodiments can be divided into multiple subroutines, each subroutine can be loaded and run by a processor to execute different steps of the face pose estimation method of the above method embodiments. Specifically, each subroutine can be stored in different storage devices, and each processor can be configured to execute programs in one or more storage devices to jointly implement the face pose estimation method of the above method embodiments, that is, each processor executes different steps of the face pose estimation method of the above method embodiments to jointly implement the face pose estimation method of the above method embodiments.

[0107] The aforementioned multiple processors can be processors deployed on the same device. For example, the aforementioned control device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors can also be processors deployed on different devices. For example, the aforementioned control device can be a server cluster, and the aforementioned multiple processors can be processors on different servers within the server cluster.

[0108] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program that performs the face pose estimation method of the above-described method embodiments. This program can be loaded and run by a processor to implement the face pose estimation method described above. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0109] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, a part of its hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0110] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0111] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A face pose estimation method, characterized in that, The method includes: Obtain the facial image data of the person to be tested; The face image data to be tested is predicted by a preset face pose estimation model to obtain face pose estimation data; The face pose estimation model includes a regression branch and a classification branch. The classification branch assists the regression branch in training so that the trained regression branch can predict the face image data. The step of training the regression branch by assisting the classification branch includes: Based on the face image data used for training, obtain the facial spatial semantic features of the face image data; For each iteration of the training, based on the facial spatial semantic features, the classification branch is applied to perform facial pose classification prediction to obtain the classification prediction result; Based on the facial spatial semantic features, the regression branch is applied to perform facial pose regression prediction to obtain the regression prediction result; Based on the classification prediction results and the regression prediction results, the first regression loss of the regression branch is obtained; Based on the first regression loss, backpropagation is performed to update the model parameters of the regression branch, thereby achieving the training of the regression branch; The classification branch includes multiple classification sub-branches with different quantization precisions, and the classification prediction result includes classification prediction results of multiple sub-branches; The application of the classification branch to classify and predict face pose, and the resulting classification prediction, includes: For each sub-branch, the sub-branch is applied to perform face pose classification prediction, and the sub-branch classification prediction result of the sub-branch is obtained. The soft-argmax algorithm is applied to obtain the classification prediction result based on the classification prediction results of multiple sub-branches.

2. The face pose estimation method according to claim 1, characterized in that, The step of obtaining the first regression loss of the regression branch based on the classification prediction result and the regression prediction result includes: Based on the classification prediction results and the ground truth facial pose data of the facial image data, the classification loss of the classification branch is obtained; Based on the regression prediction results and the true face pose data, the second regression loss of the regression branch is obtained; The first regression loss is obtained based on the classification loss and the second regression loss.

3. The face pose estimation method according to claim 2, characterized in that, The step of obtaining the first regression loss based on the classification loss and the second regression loss includes: The first regression loss is obtained by weighting the classification loss and the second regression loss.

4. The face pose estimation method according to claim 1, characterized in that, The step of training the regression branch by assisting the classification branch also includes: Based on the classification prediction results of each sub-branch and the ground truth data of face pose, obtain the sub-branch classification loss of the corresponding classification sub-branch; Backpropagation is performed based on the classification loss of each sub-branch to update the model parameters of the corresponding classification sub-branch.

5. The face pose estimation method according to any one of claims 1 to 4, characterized in that, The step of predicting the face image data to be tested using a preset face pose estimation model to obtain face pose estimation data includes: Based on the face image data to be tested, obtain the facial spatial semantic features of the face image data; The facial spatial semantic features are input into the facial pose estimation model to obtain the facial pose estimation data.

6. The face pose estimation method according to claim 5, characterized in that, The facial spatial semantic features obtained from the facial image data include: ROI region extraction is performed on the face image data to obtain the face ROI region in the face image data; When the face image data is used for training, the face ROI region is augmented to obtain the augmented face ROI region; based on the augmented face ROI region, feature extraction is performed to obtain the face spatial semantic features. When the face image data is the face image data to be tested, feature extraction is performed based on the face ROI region to obtain the face spatial semantic features.

7. A control device comprising at least one processor and at least one storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the face pose estimation method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the face pose estimation method according to any one of claims 1 to 6.

Images