Face pose estimation method and device, electronic equipment and storage medium
By extracting the spatial structural characteristics of face key points and integrating the shallow fusion in the pose estimation network, the problem of insufficient robustness and generalization in face pose estimation is solved, and higher detection accuracy and adaptability are achieved.
Patent Information
- Application Number
- CN202411381078.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art has problems of insufficient robustness and generalization in facial pose estimation. There is an error between the 3D standard template and the actual face. The method based on a specific neural network is insufficient generalization of the original image's direct pose estimation.
After face recognition, spatial structure features are extracted from the face key point images, and the features are fused in the shallow layer of the pose estimation network to improve the network's spatial structure learning ability of the face image, combining face key points and pose estimation The network realizes facial posture estimation.
On the basis of ensuring the robustness of the pose estimation network, its generalization ability is improved so that the face pose estimation can take into account both robustness and generalization, and improve the accuracy and adaptability of detection.
Smart Images
Figure CN119314214A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular relates to a face posture estimation method, a face posture estimation device, an electronic device, and a computer-readable storage medium. Background Art
[0002] In various application scenarios such as human-computer interaction, it is often necessary to detect the collected facial postures in order to filter out faces that do not meet the requirements of the scene. For facial posture estimation methods based on 3D standard templates, there is still a large robustness problem due to the large error between the 3D standard template and the actual 3D face; for facial posture estimation methods based on specific neural networks, since the posture of the face in the original image is directly estimated, there is still a problem of insufficient generalization. Based on this, how to balance robustness and generalization when estimating facial posture has become a problem to be solved. Summary of the invention
[0003] The present application provides a face pose estimation method, a face pose estimation device, an electronic device and a computer-readable storage medium, which can take into account both robustness and generalization when estimating face pose.
[0004] In a first aspect, the present application provides a method for estimating a face posture, comprising:
[0005] Perform face recognition on the target image to obtain a face image and a face key point image;
[0006] Extract spatial structural features from facial key point images;
[0007] The face image is input into the pose estimation network, and the spatial structure features are fused at the shallow layer of the pose estimation network to obtain the face pose result output by the pose estimation network for the face image.
[0008] In a second aspect, the present application provides a face pose estimation device, comprising:
[0009] The recognition module is used to perform face recognition on the target image to obtain the face image and the face key point image;
[0010] An extraction module is used to extract spatial structural features from facial key point images;
[0011] The acquisition module is used to input the face image into the posture estimation network and fuse the spatial structure features at the shallow layer of the posture estimation network to obtain the face posture result output by the posture estimation network for the face image.
[0012] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.
[0013] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.
[0014] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it implements the steps of the method of the first aspect.
[0015] Compared with the prior art, the present application has the following beneficial effects: the present application combines the facial key points and the posture estimation network to realize facial posture estimation. Specifically, after face recognition, the electronic device can extract the spatial structure features from the facial key point image; the posture estimation network will fuse the spatial structure features at its shallow layer, so as to better learn the spatial structure of the face image. Through the above fusion operation, the generalization ability of the posture estimation network can be improved on the basis of ensuring its robustness, so that both robustness and generalization can be taken into account when estimating facial posture.
[0016] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 It is a flowchart of a face posture estimation method provided in an embodiment of the present application;
[0019] Figure 2 Schematic diagram of the implementation of the face posture estimation method provided in the embodiment of the present application;
[0020] Figure 3 is another schematic diagram of implementing the face posture estimation method provided in an embodiment of the present application;
[0021] Figure 4 is a structural block diagram of a face posture estimation device provided in an embodiment of the present application;
[0022] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0023] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0024] The face posture estimation method proposed in the embodiment of the present application can be applied to electronic devices with data processing functions. In some examples, the electronic device can be a robot, a personal computer, a smart phone, a tablet computer, a server, etc., which is not limited here. Figure 1 , the face posture estimation method provided in the embodiment of the present application includes:
[0025] Step 101, performing face recognition on the target image to obtain a face image and a face key point image;
[0026] When the electronic device is equipped with a camera, it can take pictures through the camera, and each frame of image taken can be used as a target image; when the electronic device is not equipped with a camera, it can receive images sent by other devices through a communication connection with other devices, and the images can be used as the target image.
[0027] After obtaining the target image, the electronic device can identify and locate the face frame containing the face information in the target image and the face key points therein through a preset face recognition algorithm. Specifically, for the face frame, the electronic device can crop the face part in the target image through the face frame to obtain a separate face image, that is, to segment the part containing only the face, thereby obtaining a face image; this operation can help the subsequent face posture estimation or other face-based tasks to reduce background interference. For the face key points, the electronic device can perform Gaussian smoothing based on a preset radius (e.g., 5 pixel length), specifically: using a Gaussian distribution function to expand the value of each key point into a local area, that is, to expand each key point outward into a circular area based on the preset radius, so as to smoothly transition the pixel values closer to the key point to form a heatmap, which is the face key point image. It can be understood that through Gaussian smoothing, the pixels around the key point are no longer just isolated points, but form a region with spatial distribution, which can help provide smoother and continuous features.
[0028] It should be noted that the facial key point image should be aligned with the facial image. That is, the facial key point image and the facial image are in the same coordinate system, thereby ensuring that the position of the facial key points is consistent with the actual position in the facial image. Through alignment, the key point information and image features can be extracted from the two images in the same coordinate system during subsequent processing, helping the subsequent posture estimation model to correctly understand the geometric relationship between the facial key points and the entire face.
[0029] It should be noted that the facial key points that the embodiments of the present application focus on are specifically five basic points, namely: the left eye center key point, the right eye center key point, the nose tip key point, the left mouth corner key point and the right mouth corner key point.
[0030] In some examples, the face recognition algorithm can be specifically implemented based on a convolutional neural network, which is not limited in the embodiments of the present application.
[0031] Step 102: extracting spatial structural features from the facial key point image.
[0032] The distance between the left and right eyes is defined as d1, which can be determined by the Euclidean distance between the left eye key point and the right eye key point; the distance from the eyes to the mouth is defined as d2, which can be determined by the average of the Euclidean distance between the left eye key point and the left corner of the mouth key point and the Euclidean distance between the right eye key point and the right corner of the mouth key point. In the process of rotating the face from the front face to the side face, it can be found that d2 remains basically unchanged, but d1 decreases significantly. Based on this, the ratio d1 / d2 can be used to quantify the degree of frontal and side face of the face, as shown in Table 1 below:
[0033] Face angle a(°) <10 10<a<30 30<a<60 60<a<90 d1 / d2 1.2-1.5 0.95-1.2 0.55-0.95 <0.55
[0034] Table 1
[0035] It can be seen that facial key points are helpful for predicting facial posture. Based on this, electronic devices can extract spatial structural features from facial key point images for subsequent facial posture estimation. Since spatial structural features are facial geometric information extracted based on the position and distribution of key points, they are related to facial posture to a certain extent and can help understand the posture of the face in three-dimensional space.
[0036] Step 103, inputting the face image into a posture estimation network, and integrating the spatial structure features at a shallow layer of the posture estimation network to obtain a face posture result output by the posture estimation network for the face image.
[0037] The posture estimation network can be used to predict the three-dimensional posture of the face, including yaw, pitch, and roll. In some examples, the posture estimation network may include a backbone network and prediction branches corresponding to the above three-dimensional postures; wherein the backbone network may include: a first convolutional layer, a first-stage convolutional layer block connected to the first convolutional layer, a second-stage convolutional layer block connected to the first-stage convolutional layer block, a second convolutional layer, a pooling layer, and a fully connected layer; the prediction branches corresponding to each dimensional posture may include: a fully connected layer and a classifier (such as softmax, etc.). In the embodiment of the present application, the structure of the posture estimation network is not limited.
[0038] For the posture estimation network, its input is specifically the face image obtained by step 101. On this basis, in order to improve generalization and accuracy, the embodiment of the present application also integrates the spatial structure features obtained by step 102 in the shallow layer of the posture estimation network. Through the above fusion operation, the external spatial structure features are introduced in the early stage of the posture estimation network, so that the geometric information extracted based on the key points of the face can be combined with the image features extracted by the backbone network of the posture estimation network to form a more complete feature representation. After integrating the spatial structure features, the final output of the posture estimation network is: face posture results, including yaw angle, pitch angle and roll angle.
[0039] The face pose results obtained by the above face pose estimation method can be used in various application scenarios, including but not limited to augmented reality (AR), human-computer interaction and driver monitoring system, which will not be elaborated here.
[0040] In some embodiments, the spatial structure feature may include a first spatial structure feature. The electronic device may obtain the first spatial structure feature by downsampling the facial key point image. In one application scenario, the facial key point image may be downsampled by a pooling layer to obtain the first spatial structure feature. In another application scenario, the facial key point image may also be downsampled by a convolution layer to obtain the first spatial structure feature, specifically: first, the facial key point image is processed by a preset first convolution kernel to obtain a downsampling result; then, the downsampling result is continued to be processed by a preset second convolution kernel to obtain the first spatial structure feature. In order to make the structure as simple as possible, in some examples, the size of the first convolution kernel may be 3*3, and the step size may be 2; the size of the second convolution kernel may be determined according to the preset radius used in the Gaussian smoothing process, for example, when the preset radius is 5, the size of the second convolution kernel may be 5*5, and the step size may be 1, so as to better cover the facial key points.
[0041] The electronic device can introduce the first spatial structure feature after the first convolution layer of the posture estimation network, so as to realize the fusion of the first spatial structure feature in the posture estimation network, which can be specifically manifested as: splicing the output of the first convolution layer with the first spatial structure feature to obtain a first splicing result. On this basis, the input of the convolution layer block of the first stage is no longer the output of the first convolution layer, but the first splicing result. That is, the electronic device will input the first splicing result into the convolution layer block of the first stage.
[0042] It should be noted that the first spatial structure feature should be aligned with the output of the first convolutional layer to ensure that the two can be spliced together.
[0043] See also Figure 2 , Figure 2 An illustration is given of a method for realizing face pose estimation by fusing the first spatial structure feature in a shallow layer of a pose estimation network.
[0044] In some embodiments, the spatial structure feature may further include a second spatial structure feature. The electronic device may further process the first spatial structure feature to obtain the second spatial structure feature, which may be specifically performed as follows: the first spatial structure feature is processed by a preset third convolution kernel to obtain the second spatial structure feature. The third convolution kernel has the same properties as the first convolution kernel; that is, the two have the same size and step length.
[0045] Since the processing performed by the convolution layer block in the first stage may destroy the spatial structure information originally introduced based on the first spatial structure feature, the second spatial structure feature can be introduced again after the convolution layer block in the first stage to compensate for the spatial structure information, which can be specifically manifested as: splicing the output of the convolution layer block in the first stage with the second spatial structure feature to obtain a second splicing result. On this basis, the input of the convolution layer block in the second stage is no longer the output of the convolution layer block in the first stage, but the second splicing result. That is, the electronic device will input the second splicing result into the convolution layer block in the second stage.
[0046] Through the fusion method proposed above, the fusion of spatial structural features is performed after the first convolutional layer of the posture estimation network and the convolutional layer block of the first stage, which can avoid affecting the deep semantic feature extraction of the convolutional layer block of the second stage of the posture estimation network.
[0047] It should be noted that the second spatial structure feature should be aligned with the output of the convolutional layer block in the first stage to ensure that the two can be spliced together.
[0048] See also Figure 3 , Figure 3 An illustration is given of a method for realizing face pose estimation by fusing a first spatial structure feature and a second spatial structure feature in a shallow layer of a pose estimation network.
[0049] As can be seen from the above, the embodiment of the present application combines the face key point and the posture estimation network to realize face posture estimation. Specifically, after face recognition, the electronic device can extract the spatial structure features from the face key point image; the posture estimation network will fuse the spatial structure features at its shallow layer, so as to better learn the spatial structure of the face image. Through the above fusion operation, the generalization ability of the posture estimation network can be improved on the basis of ensuring the robustness of the posture estimation network, so that both robustness and generalization can be taken into account when estimating face posture.
[0050] Corresponding to the face posture estimation method provided above, the embodiment of the present application also provides a face posture estimation device. Figure 4 As shown, the face posture estimation device 4 includes: a recognition module 401, an extraction module 402 and an acquisition module 403.
[0051] The recognition module 401 is used to perform face recognition on the target image to obtain a face image and a face key point image;
[0052] Extraction module 402, used to extract spatial structural features from the facial key point image;
[0053] The acquisition module 403 is used to input the face image into the posture estimation network and fuse the spatial structure features at the shallow layer of the posture estimation network to obtain the face posture result output by the posture estimation network for the face image.
[0054] In some embodiments, the spatial structure feature includes a first spatial structure feature; the extraction module 402 includes:
[0055] A first processing unit, used for processing the facial key point image by using a preset first convolution kernel to obtain a downsampling result;
[0056] The second processing unit is used to process the downsampling result through a preset second convolution kernel to obtain a first spatial structure feature.
[0057] In some embodiments, the backbone network of the posture estimation network includes: a first convolutional layer, a first-stage convolutional layer block connected to the first convolutional layer, a second-stage convolutional layer block connected to the first-stage convolutional layer block, a second convolutional layer, a pooling layer, and a fully connected layer; an acquisition module 403 includes:
[0058] A first splicing unit, used for splicing the output of the first convolutional layer with the first spatial structure feature to obtain a first splicing result;
[0059] The first input unit is used to input the first concatenation result into the convolution layer block of the first stage.
[0060] In some embodiments, the spatial structure feature further includes a second spatial structure feature; and the extraction module 402 further includes:
[0061] The third processing unit is used to process the first spatial structure feature through a preset third convolution kernel to obtain a second spatial structure feature, wherein the third convolution kernel has the same properties as the first convolution kernel.
[0062] In some embodiments, the acquisition module 403 further includes:
[0063] A second splicing unit is used to splice the output of the convolutional layer block in the first stage with the second spatial structure feature to obtain a second splicing result;
[0064] The second input unit is used to input the second concatenation result into the convolution layer block of the second stage.
[0065] In some embodiments, the identification module 401 includes:
[0066] A recognition unit, used for performing face recognition on a target image based on a preset face recognition algorithm;
[0067] A segmentation unit, for segmenting a face image from a target image according to the face frame when the face recognition result includes a face frame and face key points;
[0068] The smoothing processing unit is used to perform Gaussian smoothing on the facial key points based on a preset radius to obtain a facial key point image, wherein the facial key point image is aligned with the facial image.
[0069] As can be seen from the above, the embodiment of the present application combines the face key point and the posture estimation network to realize face posture estimation. Specifically, after face recognition, the electronic device can extract the spatial structure features from the face key point image; the posture estimation network will fuse the spatial structure features at its shallow layer, so as to better learn the spatial structure of the face image. Through the above fusion operation, the generalization ability of the posture estimation network can be improved on the basis of ensuring the robustness of the posture estimation network, so that both robustness and generalization can be taken into account when estimating face posture.
[0070] Corresponding to the face posture estimation method provided above, the present application embodiment also provides an electronic device. Figure 5 , Figure 5 The electronic device 5 includes a memory 501, one or more processors 502 ( Figure 5Only one is shown) and a computer program stored in the memory 501 and executable on the processor. The memory 501 is used to store software programs and units, and the processor 502 executes various functional applications and data processing by running the software programs and units stored in the memory 501 to obtain the resources corresponding to the above-mentioned preset events. Specifically, the processor 502 implements the following steps when running the above-mentioned computer program stored in the memory 501:
[0071] Perform face recognition on the target image to obtain a face image and a face key point image;
[0072] Extract spatial structural features from facial key point images;
[0073] The face image is input into the pose estimation network, and the spatial structure features are fused at the shallow layer of the pose estimation network to obtain the face pose result output by the pose estimation network for the face image.
[0074] Assuming that the above is the first possible implementation, in a second possible implementation provided on the basis of the first possible implementation, the spatial structure feature includes a first spatial structure feature; and extracting the spatial structure feature from the facial key point image includes:
[0075] The facial key point image is processed by a preset first convolution kernel to obtain a downsampling result;
[0076] The downsampling result is processed by a preset second convolution kernel to obtain a first spatial structure feature.
[0077] In a third possible implementation provided as a basis for the above second possible implementation, the backbone network of the posture estimation network includes: a first convolutional layer, a first-stage convolutional layer block connected to the first convolutional layer, a second-stage convolutional layer block connected to the first-stage convolutional layer block, a second convolutional layer, a pooling layer, and a fully connected layer; the shallow fusion spatial structure features of the posture estimation network include:
[0078] Concatenate the output of the first convolutional layer with the first spatial structure feature to obtain a first concatenation result;
[0079] The first concatenation result is input into the convolutional layer block of the first stage.
[0080] In a fourth possible implementation provided on the basis of the third possible implementation, the spatial structure feature further includes a second spatial structure feature; and extracting the spatial structure feature from the facial key point image further includes:
[0081] The first spatial structure feature is processed by a preset third convolution kernel to obtain a second spatial structure feature, wherein the third convolution kernel has the same property as the first convolution kernel.
[0082] In a fifth possible implementation provided on the basis of the fourth possible implementation, the shallow fusion of spatial structure features in the posture estimation network further includes:
[0083] The output of the convolutional layer block in the first stage is concatenated with the second spatial structure feature to obtain a second concatenation result;
[0084] The second concatenation result is input into the convolutional layer block of the second stage.
[0085] In a sixth possible implementation provided on the basis of the first possible implementation, or the second possible implementation, or the third possible implementation, or the fourth possible implementation, or the fifth possible implementation, face recognition is performed on the target image to obtain a face image and a face key point image, including:
[0086] Based on the preset face recognition algorithm, perform face recognition on the target image;
[0087] When the face recognition result includes a face frame and face key points, a face image is segmented from the target image according to the face frame, and the face key points are Gaussian smoothed based on a preset radius to obtain a face key point image, wherein the face key point image is aligned with the face image.
[0088] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0089] The memory 501 may include a read-only memory and a random access memory, and provide instructions and data to the processor 502. A part or all of the memory 501 may also include a non-volatile random access memory. For example, the memory 501 may also store information of a device category.
[0090] As can be seen from the above, the embodiment of the present application combines the face key point and the posture estimation network to realize face posture estimation. Specifically, after face recognition, the electronic device can extract the spatial structure features from the face key point image; the posture estimation network will fuse the spatial structure features at its shallow layer, so as to better learn the spatial structure of the face image. Through the above fusion operation, the generalization ability of the posture estimation network can be improved on the basis of ensuring the robustness of the posture estimation network, so that both robustness and generalization can be taken into account when estimating face posture.
[0091] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0092] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0093] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of external device software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0094] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0095] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0096] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the associated hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The above-mentioned computer-readable storage medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer-readable memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media does not include electrical carrier signals and telecommunication signals.
[0097] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A method for estimating a face posture, characterized in that: include: Perform face recognition on the target image to obtain a face image and a face key point image; Extracting spatial structural features from the facial key point image; The face image is input into a posture estimation network, and the spatial structure features are fused at a shallow layer of the posture estimation network to obtain a face posture result output by the posture estimation network for the face image.
2. The method for estimating facial posture according to claim 1, wherein: The spatial structure feature includes a first spatial structure feature; The step of extracting spatial structural features from the facial key point image comprises: Processing the facial key point image by using a preset first convolution kernel to obtain a downsampling result; The downsampling result is processed by a preset second convolution kernel to obtain a first spatial structure feature.
3. The method for estimating facial posture as claimed in claim 2, characterized in that: The backbone network of the posture estimation network includes: a first convolutional layer, a first-stage convolutional layer block connected to the first convolutional layer, a second-stage convolutional layer block connected to the first-stage convolutional layer block, a second convolutional layer, a pooling layer, and a fully connected layer; the fusion of the spatial structure features in the shallow layer of the posture estimation network includes: Splicing the output of the first convolutional layer with the first spatial structure feature to obtain a first splicing result; The first concatenation result is input into the convolutional layer block of the first stage.
4. The method for estimating facial posture as claimed in claim 3, characterized in that: The spatial structural feature also includes a second spatial structural feature; The step of extracting spatial structural features from the facial key point image further comprises: The first spatial structure feature is processed by a preset third convolution kernel to obtain a second spatial structure feature, wherein the third convolution kernel has the same properties as the first convolution kernel.
5. The method for estimating facial posture as claimed in claim 4, characterized in that: The fusion of the spatial structure features at the shallow layer of the posture estimation network also includes: Splicing the output of the convolutional layer block in the first stage with the second spatial structure feature to obtain a second splicing result; The second concatenation result is input into the convolutional layer block of the second stage.
6. The method for estimating facial posture according to any one of claims 1 to 5, characterized in that: The step of performing face recognition on the target image to obtain a face image and a face key point image includes: Based on a preset face recognition algorithm, performing face recognition on the target image; In the case where the result of face recognition includes a face frame and face key points, a face image is segmented from the target image according to the face frame, and Gaussian smoothing is performed on the face key points based on a preset radius to obtain a face key point image, wherein the face key point image is aligned with the face image.
7. A face pose estimation device, characterized in that: include: The recognition module is used to perform face recognition on the target image to obtain the face image and the face key point image; An extraction module, used to extract spatial structural features from the facial key point image; The acquisition module is used to input the face image into a posture estimation network and fuse the spatial structure features at a shallow level of the posture estimation network to obtain a face posture result output by the posture estimation network for the face image.
8. The face pose estimation device according to claim 7, characterized in that: The spatial structure feature includes a first spatial structure feature; The extraction module comprises: A first processing unit, used for processing the facial key point image by using a preset first convolution kernel to obtain a downsampling result; The second processing unit is used to process the downsampling result through a preset second convolution kernel to obtain a first spatial structure feature.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.