A large-angle face recognition preprocessing method based on an improved DOPE network

Abstract

The application belongs to the technical field of face recognition, and proposes a large-angle face recognition preprocessing method based on an improved DOPE network, generates a heat map of each key point in a face image through a Face-DOPE network, then obtains 2D coordinates of each key point through the heat map, and generates a rotation matrix through the 2D coordinate set and a preset 3D coordinate set; finally, the face image is frontally processed through the rotation matrix, the correction of a non-ideal pose face image is realized without deviating from the original face features, and the accuracy and robustness of the face recognition result of the face image under a non-ideal pose are improved through the recognition of the corrected face image.

Description

Technical Field

[0001] This invention relates to the field of face recognition technology, and in particular to a large-angle face recognition preprocessing method based on an improved DOPE network. Background Technology

[0002] Facial recognition technology, as an advanced biometric identification method, has been widely used in identity authentication scenarios such as access control systems, payment verification, and hotel check-in management. In current mainstream applications, the recognition process typically relies on the user's active cooperation, meaning that recognition is completed under ideal conditions: sufficient lighting, appropriate shooting distance, and the face facing the acquisition device. In such application scenarios, the system can obtain high-quality facial images, thereby ensuring a high recognition accuracy rate.

[0003] However, in practical applications in uncontrolled environments, due to the fixed locations and limited viewing angles of surveillance cameras, and the varied postures of the subjects, severe facial distortion, tilting, or leaning at large angles often occur, leading to the loss or deformation of key facial feature information. Furthermore, uneven lighting and excessive distance further degrade the quality of facial images.

[0004] Although existing technologies have improved brightness distribution through illumination normalization algorithms or mitigated long-distance recognition problems by combining high-resolution imaging equipment with super-resolution reconstruction techniques, face recognition still faces challenges beyond low quality caused by uneven lighting and excessive distance. These include facial deformation, occlusion, and feature distortion due to large-angle poses. Existing methods for addressing large-angle face recognition include:

[0005] 3D Face Modeling and Frontalization: This method reconstructs the three-dimensional structure of a face using a 3D model and performs pose normalization. While highly accurate, it suffers from high computational complexity, making it difficult to meet real-time requirements, and it is also sensitive to initialization.

[0006] Generative Adversarial Network (GAN) image transformation: This method uses techniques such as TP-GAN and CapsuleFace to synthesize frontal views from profile images. However, this approach carries the risk of identity information distortion; the generated image may deviate from the original identity features, affecting the reliability of recognition.

[0007] Multi-view fusion recognition: This method uses multiple cameras to capture facial information from different angles and then fuses it. However, it is costly, complex to deploy, relies on precise calibration, and is difficult to implement in existing surveillance systems.

[0008] Keypoint alignment and local feature matching: Features are extracted after aligning detected 2D keypoints through affine transformation. However, keypoint detection itself is unstable at large angles and is prone to error accumulation.

[0009] However, the existing methods mentioned above still struggle to effectively improve the accuracy and robustness of large-angle face recognition. Therefore, insufficient robustness of face recognition at large angles has become a key technical bottleneck restricting the widespread application of face recognition technology in complex surveillance scenarios.

[0010] In conclusion, there is an urgent need for a facial recognition technology solution that can maintain high accuracy and stability even under non-ideal posture conditions, in order to improve its practicality and reliability in real-world application scenarios such as non-contact recognition and passive monitoring. Summary of the Invention

[0011] To address the issue of low accuracy and robustness of existing face recognition technologies under non-ideal pose conditions, a large-angle face recognition preprocessing method based on an improved DOPE network is proposed, which solves the aforementioned problem.

[0012] The technical solution of this invention is as follows:

[0013] A preprocessing method for large-angle face recognition based on an improved DOPE network, comprising the following steps:

[0014] S1: Face detection: Identify the face region in the input image and perform cropping and preprocessing to obtain the face image;

[0015] S2: Generate heatmaps of facial key points: Construct a Face-DOPE network, input the face image into the Face-DOPE network to obtain heatmaps of each key point in the face image; the Face-DOPE network extracts shallow feature maps of different scales through an encoder, and uses skip connections to concatenate the shallow feature maps with the deep feature maps output by the downsampling module in the decoder, which are used as inputs to the next-level downsampling module. The result of the end downsampling module is output to the mapping module to obtain a complete face feature heatmap, and the complete face feature heatmap is input to the output head to obtain heatmaps of each key point in the face image;

[0016] S3: Calculate the 6D pose of the face: Calculate the 6D pose of the face image relative to the camera based on the heat map of each key point in the face image, the EPnP algorithm and the preset 3D key point coordinate set;

[0017] S4: Frontalization of face image: The face image is transformed by back projection through the 6D pose to obtain a frontal face image;

[0018] S5: Face recognition: Input the frontal face image into the face recognition network for face recognition and obtain the face recognition result.

[0019] Preferably, the Face-DOPE network includes: an encoder, a skip connection, a decoder, a mapping module, and an output header;

[0020] Encoder: Extracts features from face images step by step using a lightweight convolutional neural network, outputs face feature maps, and saves shallow feature maps of different scales generated at each level of feature extraction.

[0021] Skip connection: The shallow feature map is concatenated with a deep feature map of the same scale output by the upsampling module in the decoder to obtain a concatenated feature; the concatenated feature is used as the input to the next-level upsampling module in the encoder;

[0022] Decoder: Composed of multiple upsampling modules connected in series; the spliced ​​features are upsampled by the multiple upsampling modules to generate deep feature maps corresponding to each upsampling module; wherein, the input of the first upsampling module is the face feature map output by the encoder end;

[0023] Mapping module: Maps the deep feature map generated by the last-level upsampling module to obtain a complete facial feature heatmap;

[0024] Output Header: Input the complete facial feature heatmap into the multi-channel convolutional layer and output a heatmap of N key points in the facial image.

[0025] Preferably, the Face-DOPE network is trained under supervised supervision using mean squared error loss; the mean squared error loss is:

[0026] (1)

[0027] in, This represents the total number of key points in a face image; This represents the predicted heatmap corresponding to the i-th keypoint output by the Face-DOPE network; The true heatmap representing the i-th key point in a face image; This represents the Frobenius norm.

[0028] Preferably, the method for generating the true heatmap of the i-th key point in the face image is as follows:

[0029] A1: Set the true 2D projection position of the i-th key point in the face image as... The actual two-dimensional projection position is normalized to obtain the standard two-dimensional projection position.

[0030] A2: Based on the standard two-dimensional projection position, the pixel values ​​of each pixel in the face image on the heatmap are determined as follows:

[0031] (2)

[0032] in, The pixel value at coordinates (x, y) in the real heatmap of the i-th key point in the face image; The x-coordinate in the standard two-dimensional projected coordinates of the i-th key point; x represents the ordinate of the i-th key point in the standard two-dimensional projected coordinates; x represents the abscissa of the pixel in the face image; y represents the ordinate of the pixel in the face image. Indicates the Gaussian kernel standard deviation;

[0033] A3: Traverse all pixels in the face image, substitute them one by one into formula (2) to calculate the pixel value, and obtain the real heat map of the i-th key point.

[0034] Preferably, in step S3,

[0035] S31: Decode the heatmap of each key point by means of heatmap peak detection or sub-pixel interpolation to obtain the 2D coordinates of each key point, and combine them into a 2D key point coordinate set.

[0036] S32: Based on the EPnP algorithm, the preset 3D key point coordinate set and the 2D key point coordinate set, the 6D pose of the face image relative to the camera is calculated.

[0037] Preferably, the method for obtaining the 2D coordinates of each key point through sub-pixel interpolation is as follows:

[0038] B1: Select the point corresponding to the maximum pixel value in the heatmap of the key point, i.e., the maximum value point. ; Calculate the neighborhood value of the maximum value point:

[0039] (3)

[0040] (4)

[0041] (5)

[0042] (6)

[0043] (7)

[0044] in, Represents the pixel values ​​at different locations in the predicted heatmap of the i-th keypoint; This represents the pixel value at the maximum value point; This represents the pixel values ​​in the left neighborhood of the maximum value point; This represents the pixel value in the right-hand neighborhood of the maximum value point; This represents the pixel value in the neighborhood above the maximum value point; This represents the pixel value in the neighborhood below the maximum value point;

[0045] B2: Calculate the offset in the x-direction and the offset in the y-direction based on the neighborhood values ​​of the maximum value point;

[0046] (8)

[0047] (9)

[0048] B3: Calculate the final coordinates of the keypoints, i.e., the 2D coordinates of the keypoints, based on the offsets in the x and y directions.

[0049] Preferably, the method for calculating the final coordinates of the key points based on the offsets in the x and y directions is as follows:

[0050] (10)

[0051] (11)

[0052] in, This represents the x-coordinate in the 2D coordinate system of a key point; The ordinate of the key point in 2D coordinates.

[0053] Preferably, the 6D pose includes a rotation matrix. Translation vector The rotation matrix is:

[0054] (12)

[0055] in,

[0056] (13)

[0057] (14)

[0058] (15)

[0059] The angle is solved by inversely using the matrix elements in the rotation matrix of formula (12):

[0060] (16)

[0061] in, Indicates the yaw angle; Indicates the pitch angle; It is the roll angle; It is the arcsine function; It is a two-parameter four-quadrant arctangent function; This represents the element in the third row and first column of the rotation matrix; This represents the element in the third row and second column of the rotation matrix; This represents the element in the third row and third column of the rotation matrix; This represents the element in the second row and first column of the rotation matrix; This represents the element in the first row and first column of the rotation matrix.

[0062] Preferably, the method of transforming a face image by backprojection is as follows:

[0063] S41: Calculate the inverse matrix of the rotation matrix in the 6D pose, which is:

[0064] (17)

[0065] in, , indicating that R is a rotation matrix in three-dimensional space;

[0066] S42: Input the face image into the DOPE network to obtain the 3D coordinates of each two-dimensional point in the face image in three-dimensional space;

[0067] S43: By using the inverse matrix of the rotation matrix and camera parameters, the 3D coordinates of each 2D point in the face image are frontalized in 3D space and projected into 2D coordinates in the frontal face image, as follows:

[0068] (18)

[0069] in, Indicates camera intrinsic parameters. This indicates finding the inverse of the rotation matrix; This represents the 3D coordinates of each two-dimensional point in a face image in three-dimensional space. Represents the 2D coordinates in a frontal human face image;

[0070] S44: Traverse the two-dimensional points in the face image and repeat the operation of step S43 to obtain a frontal face image.

[0071] Beneficial effects:

[0072] This invention proposes a large-angle face recognition preprocessing method based on an improved DOPE network. The method generates heatmaps of key points in a face image using the Face-DOPE network, then obtains 2D coordinate estimates for each key point from the heatmaps. A rotation matrix is ​​then generated using the 2D coordinate set and a preset 3D coordinate set. Finally, the rotation matrix is ​​used to frontalize the face image, achieving correction of face images with non-ideal poses without deviating from the original face features. Recognition of the corrected face image improves the accuracy and robustness of face recognition results under non-ideal poses.

[0073] Compared to the traditional DOPE model, the Face-DOPE network uses skip connections to concatenate shallow feature maps of different scales in the encoder with deep feature maps output by the downsampling module in the decoder, serving as input for the next-level downsampling module. This achieves the transfer of pixel-level spatial details from the shallow layers of the encoder to the corresponding scale downsampling module, avoiding detail loss caused by multiple downsampling operations in the decoder. Subsequently, the heatmap generated by the Face-DOPE network can accurately locate key points in the face image, improving the accuracy of the subsequently calculated 6D pose. Furthermore, the 6D pose can accurately correct face images under non-ideal poses, improving the accuracy and robustness of face recognition results under non-ideal poses.

[0074] Compared to the traditional DOPE model, which cannot effectively generate accurate rotation matrices for highly structured objects like faces, this paper proposes a Face-DOPE network to generate heatmaps of key points in face images. Subsequently, a 6D pose estimation mechanism is performed based on the heatmaps of key points generated by the Face-DOPE network, achieving pose perception under non-ideal poses. This significantly improves the accuracy, stability, and real-time performance of pose recognition under non-ideal poses, and solves the problem that direct regression of the traditional DOPE model is prone to failure in complex scenes. It also significantly improves the robustness and localization accuracy of key point recognition in face images. Attached Figure Description

[0075] Figure 1 This is a flowchart of a large-angle face recognition preprocessing method based on an improved DOPE network.

[0076] Figure 2 This is a flowchart for preprocessing large-angle facial images. Detailed Implementation

[0077] Example 1

[0078] like Figure 1 As shown, the steps of a large-angle face recognition preprocessing method based on an improved DOPE network are as follows:

[0079] S1: Face Image Detection: The input image is subjected to face detection using a face detection algorithm to obtain the face region in the input image; the input image is then cropped and preprocessed based on the face region to obtain the face image;

[0080] S2: Generate heatmaps of facial key points: Construct a Face-DOPE network; input the face image into the Face-DOPE network to obtain heatmaps of each key point in the face image; in the Face-DOPE network, shallow feature maps of different scales are extracted by the encoder, and the shallow feature maps are concatenated with the deep feature maps output by the downsampling module in the decoder using skip connections, which are used as the input of the next-level downsampling module. The result of the end downsampling module is output to the mapping module to obtain a complete face feature heatmap, and the complete face feature heatmap is input to the output head to obtain heatmaps of each key point in the face image;

[0081] S3: Calculate the 6D pose of the face: Solve the 2D coordinates of each key point based on the heat map of each key point, and construct a 2D coordinate set based on the 2D coordinates of all key points; The EPnP algorithm calculates the 6D pose of the face image relative to the camera based on the preset 3D key point coordinate set and the 2D coordinate set.

[0082] S4: Frontalization of face image: The face image is transformed by back projection through the 6D pose to obtain a frontal face image;

[0083] S5: Face recognition: Input the frontal face image into the face recognition network for face recognition and obtain the face recognition result.

[0084] Example 2

[0085] like Figure 2 As shown, a large-angle face recognition preprocessing method based on an improved DOPE network mainly includes the following steps:

[0086] Step S1: Input the original image into the face detection module to obtain the face region; crop and preprocess the original input image through the face region to obtain the face image.

[0087] Specifically, the preprocessing refers to changing the size of the cropped image. .

[0088] Step S2: Construct a face-specific DOPE network (Face-DOPE network)

[0089] 1. Based on the original DOPE network architecture, a lightweight convolutional neural network (including but not limited to the following networks such as MobileNetV3 and ShuffleNet V2) is used as the encoder, and a U-Net structure is used as the decoder;

[0090] 2. The input is a single RGB face image (suggested size 256×256); the network output is multi-channel heatmaps, each channel corresponding to a predefined probability distribution of the 2D projection position of the 3D face key points on the image plane;

[0091] 3. The 3D key points are pre-modeled based on a general 3D face template (such as BFM or FLAME model), and contain 68 to 120 key points with clear geometric meaning (such as the tip of the nose, the corner of the eye, the corner of the mouth, the edge of the ear, etc.). They are registered in the standard 3D coordinate system, and the number of channels can be adjusted according to actual needs to balance hardware computing power and model accuracy.

[0092] 4. During the training phase, the heatmap generation is supervised using the cross-entropy loss function.

[0093] Specifically, the Face-DOPE network aims to directly generate heatmaps of key points in a face image from a single image, without relying on external key point input or guidance from specific key points. The advantage of Face-DOPE generating heatmaps and then calculating the 2D coordinates of key points based on them is that the heatmaps present the positions of key points in the face in the form of a probability distribution. This transforms key point detection into probability prediction of each pixel's position, better capturing neighborhood information and reducing quantization errors. This not only lowers the learning difficulty of the model but also significantly improves the longitude of the 2D coordinates of subsequent facial key points, leading to more accurate 6D pose and improved face recognition accuracy under non-ideal poses.

[0094] 1.1 Overall Network Architecture

[0095] (1) Encoder: A lightweight convolutional neural network is used as the backbone of the entire network for feature extraction, outputting face feature maps and shallow feature maps of different scales; the encoder preferably uses the MobileNetV3-small model or the ShuffleNet V2 model, which balances accuracy and inference speed and is suitable for edge deployment.

[0096] (2) Decoder: This part gradually restores the spatial resolution of the face feature map through transposed convolution (or upsampling module).

[0097] (3) Skip Connections: Skip connections concatenate shallow feature maps of the same scale in the encoder with the current deep feature maps in the decoder and use them as input to the next level of transposed convolution (or upsampling module).

[0098] (4) Mapping module: Maps the deep feature map generated by the last-level upsampling module to obtain a complete face feature heatmap, i.e., a high-resolution heatmap; the complete face feature heatmap refers to a heatmap that includes all key points;

[0099] (5) Output Head: Multi-channel convolutional layer, output Each channel's heatmap represents the 2D location probability distribution of a 3D keypoint.

[0100] The input image size of the Face-DOPE network is [size missing]. The RGB image output image has a size of [size missing]. heatmap ( (Number of key points).

[0101] Specifically, the DOPE network is applied to face recognition tasks, expanding its application boundaries in the field of biometrics. Furthermore, the DOPE network is improved specifically for face recognition tasks, resulting in the proposed Face-DOPE network. The Face-DOPE network does not require key points in the face as display input and does not rely on external key point information during inference. The Face-DOPE network is optimized specifically for 3D face key point prediction, improving pose estimation accuracy.

[0102] 1.2 3D Keypoint Template Definition

[0103] (1) In order to realize the 3D-2D correspondence, a standard 3D face key point template needs to be established in advance.

[0104] Based on a general 3D face model, select There are 1 (the number and location of which can be adjusted according to computing power and actual needs, depending on the requirements) key points with clear geometric semantics, but they should cover the main facial structures, such as the tip of the nose, the corner of the eye, the corner of the mouth, the brow ridge, the edge of the ear, the jaw contour, etc.

[0105] Specifically, general 3D face models are constructed using two widely used parametric 3D face modeling techniques: BFM (Basel Face Model) or FLAME.

[0106] Specifically, 3D key points are key points on a general 3D face model.

[0107] (2) Coordinates of all 3D key points Stored in a standard 3D coordinate system (unit: millimeters), denoted as:

[0108] (19)

[0109] in Indicates the first The three-dimensional coordinates of the key points; express The dimension is .

[0110] 1.3 Heatmap Generation and Supervised Learning

[0111] (1) Definition of heat map

[0112] For the The key points and their corresponding heatmaps (like Let represent the probability distribution of the i-th keypoint appearing in the image. Ideally, the heatmap will generate a Gaussian peak at the true 2D projection location of the keypoint: the peak value corresponds to the true 2D projection location of the keypoint. .

[0113] Let the first The true 2D projection positions of the key points are: Normalized to the output feature map size (e.g., from) Mapped to ):

[0114] (20)

[0115] The heatmap is then defined as:

[0116] (2)

[0117] in The standard deviation of the Gaussian kernel is usually set to 1~2 (pixel units) to control the smoothness of the heatmap.

[0118] Specifically, the first The true 2D projection location of each key point can be obtained through pre-annotated datasets such as AFLW, 300W, and COFW; or manually annotated using the LabelMe tool.

[0119] (2) Network output and loss function

[0120] Predicted heatmaps are output using the Face-DOPE network. The Face-DOPE network is trained under supervision using mean squared error loss (MSE Loss); the mean squared error loss is:

[0121] (1)

[0122] in, This represents the total number of key points in a face image; This represents the heatmap corresponding to the i-th key point output by the Face-DOPE network. The true heatmap representing the i-th key point in a face image; It is the Frobenius norm.

[0123] Specifically, during the training phase, the Face-DOPE network autonomously learns key points in a face and generates corresponding heatmaps. The entire process does not require additional input of key feature descriptions or location data as guidance.

[0124] During the inference phase, the Face-DOPE network's input is also only a face image, requiring no external keypoint assistance.

[0125] Step S3: Calculate the 6D pose of the face

[0126] 1. Calculate the heatmap output of the Face-DOPE network. Middle 2D coordinate estimation of key points The 2D coordinates of each keypoint will be obtained and merged into a single 2D coordinate set. ;

[0127] Specifically, the 2D coordinate estimation of the i-th keypoint can be calculated using either the heatmap peaking algorithm or the subpixel interpolation algorithm. The heatmap peaking algorithm divides the heatmap into small regions and takes the maximum value within each region; the subpixel interpolation method calculates a larger heatmap from the original heatmap, and this heatmap is then divided into smaller regions.

[0128] The expression for heatmap peak detection is:

[0129] (twenty one)

[0130] To improve the accuracy of 2D coordinate estimation of key points, sub-pixel interpolation (such as quadratic function fitting) can be used. The specific steps are as follows:

[0131] set up The neighborhood value of the location with the maximum value is:

[0132] (3)

[0133] (4)

[0134] (5)

[0135] (6)

[0136] (7)

[0137] in, Represents the response value (pixel value) at different locations in the predicted heatmap of the i-th key point; This represents the center point response value, i.e., the location of the maximum value currently detected in the heatmap. The pixel values ​​are output; This represents the response value of the left neighborhood, which is the pixel value at the position one pixel to the left of the maximum value in the heatmap; This represents the response value of the right-hand neighborhood, which is the pixel value at the point one pixel to the right of the maximum value in the heatmap;

[0138] Then in The directional offset is:

[0139] (8)

[0140] The offset in the y-direction is:

[0141] (9)

[0142] Final coordinates:

[0143] (10)

[0144] (11)

[0145] Get all An estimated set of 2D keypoints:

[0146] (twenty two)

[0147] Specifically, 2D keypoints represent keypoints on a face image.

[0148] 2. Save the pre-stored set of 3D keypoint coordinates (From a 3D face template) and the estimated set of 2D key points corresponding to the face image. The input is used as the Efficient Perspective-n-Point (EPnP) algorithm; then the EPnP algorithm is used to solve for the 6D pose of the face relative to the camera, including: rotation matrix. Translation vector ;

[0149] Specifically, when solving 6D pose, the PNP algorithm or its variants can be used. The PNP algorithm or its variants are widely encapsulated in relevant function libraries, and the corresponding functions can be directly called from the relevant function libraries.

[0150] Specifically, because the 6D pose is calculated by calling the EPnP function, the output includes translation variables, but these translation variables are not used during the subsequent virtual frontalization.

[0151] 3. Further decompose the rotation matrix The Euler angles are obtained: yaw, pitch, and roll, which are used to quantify the degree of attitude deflection.

[0152] Specifically, the rotation matrix Decomposed into Euler angles (ZYX order, i.e., yaw-pitch-roll):

[0153] (12)

[0154] in,

[0155] (13)

[0156] (14)

[0157] (15)

[0158] The angle is solved by inversely using the matrix elements in the rotation matrix of formula (12):

[0159] (16)

[0160] Step S4: Virtual Fronting

[0161] Using the estimated rotation matrix That is, by using the yaw, pitch, and roll angles in the rotation matrix to perform inverse warping on the original face image (the face image obtained in step S1), the original face image is simulated to be "aligned" into a frontal view.

[0162] 1. Assume the original rotation matrix is Then the reverse rotation matrix is The formula is expressed as follows:

[0163] (17)

[0164] 2. For the coordinates of each 3D point Applying the aforementioned reverse rotation matrix, and then based on the camera intrinsic parameters... (Including focal length, principal point coordinates, etc.), project it back onto the 2D plane to obtain the coordinates of the fronted 2D image. .

[0165] Specifically, the projection formula can be expressed as:

[0166] (18)

[0167] Specifically, each 3D point The method for obtaining the data is as follows: input the face image into the DOPE network to obtain the 3D coordinates of each point in the face image;

[0168] The DOPE network used is the DOPE proposed by NVIDIA. During its use, the default 3D geometric prior of industrial objects in the DOPE network is replaced with a standard 3D face model, and the network output head is fine-tuned to match the number of facial key points (the number of key points is adjusted according to accuracy requirements and computing power). This allows a face image to be input into the DOPE model to obtain the 3D coordinates of each point in the face image, i.e., the coordinates of the 3D points. .

[0169] Specifically, the standard 3D face model can be constructed using either Basel Face Model (BFM) or FLAME, two widely used parametric 3D face modeling techniques.

[0170] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention patent.

Claims

1. A preprocessing method for large-angle face recognition based on an improved DOPE network, characterized in that, S1: Face detection: Identify the face region in the input image and perform cropping and preprocessing to obtain the face image; S2: Generate heatmaps of facial key points: Construct a Face-DOPE network, input the face image into the Face-DOPE network to obtain heatmaps of each key point in the face image; the Face-DOPE network extracts shallow feature maps of different scales through an encoder, and uses skip connections to concatenate the shallow feature maps with the deep feature maps output by the downsampling module in the decoder, which are used as inputs to the next-level downsampling module. The result of the end downsampling module is output to the mapping module to obtain a complete face feature heatmap, and the complete face feature heatmap is input to the output head to obtain heatmaps of each key point in the face image; S3: Calculate the 6D pose of the face: Calculate the 6D pose of the face image relative to the camera based on the heat map of each key point in the face image, the EPnP algorithm and the preset 3D key point coordinate set; S4: Frontalization of face image: The face image is transformed by back projection through the 6D pose to obtain a frontal face image; S5: Face recognition: Input the frontal face image into the face recognition network for face recognition and obtain the face recognition result.

2. A large-angle face recognition preprocessing method based on an improved DOPE network as described in claim 1, characterized in that, The Face-DOPE network includes: an encoder, a skip connection, a decoder, a mapping module, and an output header; Encoder: Extracts features from face images step by step using a lightweight convolutional neural network, outputs face feature maps, and saves shallow feature maps of different scales generated at each level of feature extraction. Skip connection: The shallow feature map is concatenated with a deep feature map of the same scale output by the upsampling module in the decoder to obtain a concatenated feature; the concatenated feature is used as the input to the next-level upsampling module in the encoder; Decoder: Composed of multiple upsampling modules connected in series; the spliced ​​features are upsampled by the multiple upsampling modules to generate deep feature maps corresponding to each upsampling module; wherein, the input of the first upsampling module is the face feature map output by the encoder end; Mapping module: Maps the deep feature map generated by the last-level upsampling module to obtain a complete facial feature heatmap; Output head: Input the complete facial feature heatmap into the multi-channel convolutional layer and output the heatmap of N key points in the facial image.

3. A large-angle face recognition preprocessing method based on an improved DOPE network as described in claim 2, characterized in that, The Face-DOPE network is trained under supervised supervision using mean squared error loss; the mean squared error loss is: （1） in, This represents the total number of key points in a face image; This represents the predicted heatmap corresponding to the i-th keypoint output by the Face-DOPE network; The true heatmap representing the i-th key point in a face image; This represents the Frobenius norm.

4. A large-angle face recognition preprocessing method based on an improved DOPE network according to claim 3, characterized in that, The method for generating the true heatmap of the i-th key point in the face image is as follows: A1: Set the true 2D projection position of the i-th key point in the face image as... The actual two-dimensional projection position is normalized to obtain the standard two-dimensional projection position. A2: Based on the standard two-dimensional projection position, the pixel values ​​of each pixel in the face image on the heatmap are determined as follows: （2） in, The pixel value at coordinates (x, y) in the real heatmap of the i-th key point in the face image; The x-coordinate in the standard two-dimensional projected coordinates of the i-th key point; x represents the ordinate of the i-th key point in the standard two-dimensional projected coordinates; x represents the abscissa of the pixel in the face image; y represents the ordinate of the pixel in the face image. Indicates the Gaussian kernel standard deviation; A3: Traverse all pixels in the face image, substitute them one by one into formula (2) to calculate the pixel value, and obtain the real heat map of the i-th key point.

5. A large-angle face recognition preprocessing method based on an improved DOPE network according to claim 1, characterized in that, In step S3 S31: Decode the heatmap of each key point by means of heatmap peak detection or sub-pixel interpolation to obtain the 2D coordinates of each key point, and combine them into a 2D key point coordinate set. S32: Based on the EPnP algorithm, the preset 3D key point coordinate set and the 2D key point coordinate set, the 6D pose of the face image relative to the camera is calculated.

6. A large-angle face recognition preprocessing method based on an improved DOPE network according to claim 1, characterized in that, The method for obtaining the 2D coordinates of each key point through sub-pixel interpolation is as follows: B1: Select the point corresponding to the maximum pixel value in the heatmap of the key point, i.e., the maximum value point. ; The neighborhood value of the maximum point is calculated as follows: （3） （4） （5） （6） （7） in, Represents the pixel values ​​at different locations in the predicted heatmap of the i-th keypoint; This represents the pixel value at the maximum value point; This represents the pixel value in the left neighborhood of the maximum value point; This represents the pixel value in the right-hand neighborhood of the maximum value point; This represents the pixel value in the neighborhood above the maximum value point; This represents the pixel value in the neighborhood below the maximum value point; B2: Calculate the offset in the x-direction and the offset in the y-direction based on the neighborhood values ​​of the maximum value point; （8） （9） B3: Calculate the final coordinates of the keypoints, i.e., the 2D coordinates of the keypoints, based on the offsets in the x and y directions.

7. A large-angle face recognition preprocessing method based on an improved DOPE network as described in claim 6, characterized in that, The method for calculating the final coordinates of key points based on the offsets in the x and y directions is as follows: （10） （11） in, This represents the x-coordinate in the 2D coordinate system of a key point; The ordinate of the key point in 2D coordinates.

8. A large-angle face recognition preprocessing method based on an improved DOPE network according to claim 1, characterized in that, The 6D pose includes a rotation matrix. Translation vector The rotation matrix is: （12） in, （13） （14） （15） The angle is solved by inversely using the matrix elements in the rotation matrix of formula (12): （16） in, Indicates the yaw angle; Indicates the pitch angle; It is the roll angle; It is the arcsine function; It is a two-parameter, four-quadrant arctangent function.

9. A large-angle face recognition preprocessing method based on an improved DOPE network according to claim 1, characterized in that, The method of transforming a face image by back projection is as follows: S41: Calculate the inverse matrix of the rotation matrix in the 6D pose, which is: （17） in, , indicating that R is a rotation matrix in three-dimensional space; S42: Input the face image into the DOPE network to obtain the 3D coordinates of each two-dimensional point in the face image in three-dimensional space; S43: Using the inverse matrix of the rotation matrix and camera parameters, the 3D coordinates of each 2D point in the face image are frontalized and projected into 2D coordinates in the frontal face image, as follows: （18） in, Indicates camera intrinsic parameters. This indicates finding the inverse of a rotation matrix; This represents the 3D coordinates of each two-dimensional point in a face image in three-dimensional space. Represents the 2D coordinates in a frontal human face image; S44: Traverse the two-dimensional points in the face image and repeat the operation of step S43 to obtain a frontal face image.

Images