3D dense face alignment model construction method, system and readable storage medium

Abstract

The application discloses a 3D dense face alignment model construction method, a system and a readable storage medium, wherein the method comprises the following steps: performing preprocessing on face images in a face data set to obtain face deformation information; a multi-scale double attention basic block and a similar information enhancement module are established to establish a 3D face regression model by using the face deformation information, wherein the 3D face regression model at least comprises the multi-scale double attention basic block and the similar information enhancement module; and a preset loss function is used to supervise training of the 3D face regression model to obtain the 3D dense face alignment model. Based on the deep learning technology with strong fitting capability, the application uses the face deformation information enhancement technology to provide an end-to-end training efficient model for further improving the 3D dense face alignment task precision, and can use the face deformation information to enhance the 3D face corresponding to the face in the input image, so that a more accurate 3D dense face alignment model framework is obtained.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method, system, and readable storage medium for constructing a 3D dense face alignment model. Background Technology

[0002] Initially, face alignment generally referred to the process of locating 2D facial landmarks in an input face image. However, 2D face alignment methods lacked the ability to detect invisible landmarks in the input image, leading to increased attention from researchers to 3D face alignment. With the advent of 3D face alignment, researchers went beyond simply detecting landmarks on the face; they went further to detect all feature points on the entire face, achieving dense point alignment relationships on the 3D face. 3D dense face alignment plays a significant role in numerous fields such as face recognition, face tracking, face animation, and human-computer interaction.

[0003] Existing 3D dense face alignment methods can be broadly categorized into two types: The first type involves fitting pre-registered 3D faces, such as 3DMMs, to obtain corresponding geometric information; these methods are also known as model-based methods. 3DMMs use PCA dimensionality reduction to summarize the entire face attribute as a linear combination of face shape and texture, thus obtaining a 3D face template. Initially, traditional methods used synthetic analysis to calculate the loss between the 3D face template and the input face image, thereby achieving a dense correspondence between the 3D face and the input face image. In recent years, the rapid development of deep learning technology has greatly optimized the fitting process of model-based methods. For example, the 3DDFA method uses a cascaded CNN network to directly regress the face parameters of 3DMMs to solve the problem of dense 3D face alignment, achieving considerable success. However, model-based methods are inherently limited by the linear combination characteristics of predefined 3D face templates in the latent space, resulting in limited expressive power. Many researchers have begun using deep neural networks to directly predict special representations of the 3D information corresponding to the input face image, such as UV mapping and volumetric mapping; these methods are also known as model-free methods.

[0004] PRN is one of the representative model-free methods in recent years. PRN uses an encoder-decoder network based on a deep neural network of ResNet50 to predict the UV location map corresponding to the input face image. The UV location map saves the 3D position information of the input face as its RGB information value. By regressing the UV location map, end-to-end training of the spatial position information of the corresponding 3D vertices of the input face can be achieved, realizing efficient 3D dense face alignment. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and readable storage medium for constructing a 3D dense face alignment model. It proposes a model framework that utilizes the deformation information of the face to enhance the 3D face corresponding to the face in the input image, thereby achieving more accurate 3D dense face alignment.

[0006] The first aspect of this invention provides a method for constructing a 3D dense face alignment model, comprising the following steps:

[0007] Preprocessing the face images in the face dataset yields face deformation information;

[0008] A multi-scale dual attention basic block and a similarity information enhancement module are established to build a 3D face regression model using the face deformation information. The 3D face regression model includes at least the multi-scale dual attention basic block and the similarity information enhancement module.

[0009] The 3D face regression model is trained under supervision using a preset loss function to obtain the 3D dense face alignment model.

[0010] In this solution, the preprocessing of face images in the face dataset to obtain face deformation information specifically includes:

[0011] The face image is cropped based on the location information of facial feature points to convert the face image into a uniform size;

[0012] Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map;

[0013] Based on the average face model in the face dataset, a projection transformation is performed according to the target parameters to obtain the face deformation information after projection transformation, and the face deformation information after projection transformation is saved in the form of UV mapping.

[0014] In this solution, the difference between the UV location map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation. The specific formula is as follows:

[0015]

[0016] Where r_D represents the face deformation information after projection transformation, F represents the set of three-dimensional vertices of the face, f is a scaling factor, and R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset. is the set of vertices on the average face model, and D is the deformation information of the face shape in the face image.

[0017] In this scheme, the 3D face information regression model consists of two parallel Encoder-Decoder networks. The Encoder network structure is composed of the multi-scale dual attention basic blocks, and the Decoder network structure is composed of six layers of standard transposed convolutional blocks. The standard transposed convolutional blocks are composed of transposed convolution plus BN normalization. The output decoded feature map of the fifth layer of the standard transposed convolutional block in the two parallel networks is used as the input data of the similarity information enhancement module.

[0018] In this scheme, the establishment of the multi-scale dual attention base block and the similarity information enhancement module specifically includes:

[0019] The multi-scale dual attention block is composed of a 1×1 convolutional block, a 3×3 convolutional block, a 5×5 convolutional block, an SE channel attention module, and an SGE spatial grouping attention module.

[0020] After the face deformation information after projection transformation is saved in the form of UV map, r_D represents the enhanced information feature map and F represents the 3D face feature map to be enhanced.

[0021] The steps for establishing the similarity information enhancement module include:

[0022] The inner product is obtained by multiplying the enhanced information feature map and the 3D face feature map to be enhanced, and the inner product result is sent to the Sigmoid activation module to obtain the similarity probability distribution s′;

[0023] The similarity probability distribution s′ is calculated as the inner product of the 3D face feature map F to be enhanced, thus obtaining the similarity information points on the 3D face feature map F. The similarity information points are then added to the 3D face feature map F to complete the establishment of the similarity information enhancement module. The specific formula is as follows:

[0024] s′=Sigmoid(F·r_D);

[0025] F = F + (F·s′).

[0026] In this scheme, the step of using a preset loss function to supervise the training of the 3D face regression model to obtain the 3D dense face alignment model specifically includes:

[0027] Supervisory data for network training is generated based on the aforementioned 3D vertex information of the face;

[0028] Using loss function L F Supervise the regression results of the 3D face subnetwork using the loss function L. r_D The regression results of the facial deformation information branch network were monitored;

[0029] Using loss function L lrr To supervise the alignment results of 3D faces in face alignment tasks;

[0030] Supervised training of the 3D face regression model to obtain the loss function L of the 3D dense face alignment model total The calculation formula is as follows: L total =ω F L F +ω r_D L r_D +ω lrr L lrr , where ω F ω r_D and ω lrr This is the corresponding loss factor.

[0031] A second aspect of the present invention also provides a 3D dense face alignment model construction system, including a memory and a processor. The memory includes a 3D dense face alignment model construction method program, which, when executed by the processor, performs the following steps:

[0032] Preprocessing the face images in the face dataset yields face deformation information;

[0033] A multi-scale dual attention basic block and a similarity information enhancement module are established to build a 3D face regression model using the face deformation information. The 3D face regression model includes at least the multi-scale dual attention basic block and the similarity information enhancement module.

[0034] The 3D face regression model is trained under supervision using a preset loss function to obtain the 3D dense face alignment model.

[0035] In this solution, the preprocessing of face images in the face dataset to obtain face deformation information specifically includes:

[0036] The face image is cropped based on the location information of facial feature points to convert the face image into a uniform size;

[0037] Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map;

[0038] Based on the average face model in the face dataset, a projection transformation is performed according to the target parameters to obtain the face deformation information after projection transformation, and the face deformation information after projection transformation is saved in the form of UV mapping.

[0039] In this solution, the difference between the UV location map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation. The specific formula is as follows:

[0040]

[0041] Where r_D represents the face deformation information after projection transformation, F represents the set of three-dimensional vertices of the face, f is a scaling factor, and R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset. is the set of vertices on the average face model, and D is the deformation information of the face shape in the face image.

[0042] In this scheme, the 3D face information regression model consists of two parallel Encoder-Decoder networks. The Encoder network structure is composed of the multi-scale dual attention basic blocks, and the Decoder network structure is composed of six layers of standard transposed convolutional blocks. The standard transposed convolutional blocks are composed of transposed convolution plus BN normalization. The output decoded feature map of the fifth layer of the standard transposed convolutional block in the two parallel networks is used as the input data of the similarity information enhancement module.

[0043] In this scheme, the establishment of the multi-scale dual attention base block and the similarity information enhancement module specifically includes:

[0044] The multi-scale dual attention block is composed of a 1×1 convolutional block, a 3×3 convolutional block, a 5×5 convolutional block, an SE channel attention module, and an SGE spatial grouping attention module.

[0045] After the face deformation information after projection transformation is saved in the form of UV map, r_D represents the enhanced information feature map and F represents the 3D face feature map to be enhanced.

[0046] The steps for establishing the similarity information enhancement module include:

[0047] The inner product is obtained by multiplying the enhanced information feature map and the 3D face feature map to be enhanced, and the inner product result is sent to the Sigmoid activation module to obtain the similarity probability distribution s′;

[0048] The similarity probability distribution s′ is calculated as the inner product of the 3D face feature map F to be enhanced, thus obtaining the similarity information points on the 3D face feature map F. The similarity information points are then added to the 3D face feature map F to complete the establishment of the similarity information enhancement module. The specific formula is as follows:

[0049] s′=Sigmoid(F·r_D);

[0050] F = F + (F·s′).

[0051] In this scheme, the step of using a preset loss function to supervise the training of the 3D face regression model to obtain the 3D dense face alignment model specifically includes:

[0052] Supervisory data for network training is generated based on the aforementioned 3D vertex information of the face;

[0053] Using loss function L F Supervise the regression results of the 3D face subnetwork using the loss function L. r_D The regression results of the facial deformation information branch network were monitored;

[0054] Using loss function L lrr To supervise the alignment results of 3D faces in face alignment tasks;

[0055] Supervised training of the 3D face regression model to obtain the loss function L of the 3D dense face alignment model total The calculation formula is as follows: L total =ω F L F +ω r_D L r_D +ω lrr L lrr , where ω F ω r_D and ω lrr This is the corresponding loss factor.

[0056] A third aspect of the present invention provides a computer-readable storage medium including a machine program for constructing a 3D dense face alignment model, wherein when the 3D dense face alignment model construction method program is executed by a processor, it implements the steps of a 3D dense face alignment model construction method as described in any of the preceding claims.

[0057] This invention discloses a method, system, and readable storage medium for constructing a 3D dense face alignment model. Based on deep learning technology with strong fitting capabilities, it utilizes face deformation information enhancement technology to provide an efficient end-to-end training model for further improving the accuracy of 3D dense face alignment tasks. It can enhance the 3D face corresponding to the face in the input image by utilizing the face deformation information, and obtain a more accurate 3D dense face alignment model framework. Attached Figure Description

[0058] Figure 1 A flowchart of a method for constructing a 3D dense face alignment model according to the present invention is shown;

[0059] Figure 2 This invention illustrates a 3D face regression model based on a 3D dense face alignment model construction method.

[0060] Figure 3 The flowchart illustrates the implementation of the scale-based dual attention basic block of a 3D dense face alignment model construction method of the present invention.

[0061] Figure 4 The flowchart illustrates the implementation of the similarity information enhancement module in the 3D dense face alignment model construction method of the present invention.

[0062] Figure 5 A block diagram of a 3D dense face alignment model construction system according to the present invention is shown. Detailed Implementation

[0063] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0064] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0065] Figure 1 A flowchart of a method for constructing a 3D dense face alignment model according to this application is shown.

[0066] like Figure 1 As shown, this application discloses a method for constructing a 3D dense face alignment model, including the following steps:

[0067] S102, Preprocess the face images in the face dataset to obtain face deformation information;

[0068] S104, Establish a multi-scale dual attention basic block and a similarity information enhancement module to build a 3D face regression model using the face deformation information;

[0069] S106, The 3D face regression model is trained under supervision using a preset loss function to obtain the 3D dense face alignment model.

[0070] It should be noted that, in this embodiment, the 3D face regression model proposed in this application mainly consists of two parallel Encoder-Decoder branch networks. The two networks are used to directly regress the 3D face corresponding to the input face and to regress the deformation information of the input face relative to the average face template, respectively. In this invention, 3D face information and face deformation information are stored in the form of UV mapping. Furthermore, the two Encoder-Decoder networks have the same structure and depth. The Encoder part consists of the multi-scale dual attention basic blocks, and the Decoder part consists of standard transposed convolutional blocks. The feature maps output from the Encoder ends of the two branch networks are concatenated and fed into the Decoder in the network that directly regresses the 3D face, thereby achieving the first step of regressing the deformation information on the directly regressed 3D face. Secondary enhancement: Considering that some points on the directly regressed 3D face should have similar feature distributions to the directly regressed face deformation information, a similarity information enhancement module was constructed. The output feature maps of the Decoder ends of the two branch networks were fed into the similarity information enhancement module, and then the output feature map of the similarity information enhancement module was fed into the transposed output layer after the branch network of the directly regressed 3D face, realizing secondary enhancement of the deformation information on the 3D face. During the training of the 3D face regression model, the two Encoder-Decoder networks used for directly regressing the 3D face and face deformation information, respectively, were trained simultaneously. The output UV position map of the branch network for directly regressing the 3D face saves the final 3D face result regressed by this model. During the training process, the preset loss function was used for supervision. The loss function includes the loss function L. F Loss function L r_D and loss function L lrr The method proposed in this application includes two parts: model training and model derivation. In the dense face alignment task, the UV position map representing the final 3D face regression result and the three-dimensional face vertex information can be obtained by directly deriving the UV position map by loading the trained model, thus completing the dense face alignment task and completing the construction of the 3D dense face alignment model.

[0071] According to an embodiment of the present invention, the preprocessing of face images in the face dataset to obtain face deformation information specifically includes:

[0072] The face image is cropped based on the location information of facial feature points to convert the face image into a uniform size;

[0073] Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map;

[0074] Based on the average face model in the face dataset, a projection transformation is performed according to the target parameters to obtain the face deformation information after projection transformation, and the face deformation information after projection transformation is saved in the form of UV mapping.

[0075] It should be noted that, in this embodiment, the publicly available face dataset 300W-LP is used as an example, and the corresponding target parameter is the 3DMM parameter. First, the original input face images in the face dataset are standardized. In the 300W-LP face dataset, the facial landmarks of each face image have been pre-annotated manually based on the 3DMM method. Therefore, the cropping range can be determined based on the landmarks. All input face images are cropped to a size of "256x256". Then, the three-dimensional vertex information of each face image is obtained based on the annotation information. Based on the three-dimensional vertex information, the image coordinates are calculated from the world coordinates of the three-dimensional vertices according to the following formula:

[0076] F = f·R·S + t;

[0077]

[0078] Where f is the scaling factor, R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset, and S is the set of 3D vertices of the face in the world coordinate system. Here, D represents the set of vertices on the average face model, D represents the deformation information of the face shape in the face image, and F represents the set of three-dimensional vertices of the face, i.e., the set of three-dimensional vertices of the face in the image coordinate system. Based on the target parameters (3DMM parameters) in the 300W-LP face dataset, the three-dimensional vertex information of the face is calculated and saved into a UV map to obtain a UV position map. Then, the average face model provided in the 300W-LP dataset is projected and transformed according to the 3DMM parameter information. The difference between the UV position map and the projected average face model is calculated to obtain the face deformation information after the projection transformation. Similarly, the face deformation information after the projection transformation is saved in the form of a UV map. Preferably, the image coordinate system and the world coordinate system are common coordinate marking methods used by those skilled in the art when processing images, and will not be elaborated here.

[0079] Furthermore, according to an embodiment of the present invention, the difference between the UV position map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation, and the specific formula is as follows:

[0080]

[0081] Where r_D represents the face deformation information after projection transformation, F represents the set of three-dimensional vertices of the face, f is a scaling factor, and R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset. is the set of vertices on the average face model, and D is the deformation information of the face shape in the face image.

[0082] It should be noted that, in this embodiment, the face deformation information r_D after projection transformation is obtained by performing a difference operation between the UV position map and the average face model after projection transformation. Specifically, the set of three-dimensional vertices of the face in the image coordinate system is saved in the UV space to obtain the UV position map corresponding to the face image. The face deformation information after projection transformation is also saved in the form of a UV map. In this embodiment, the UV position map and / or the face deformation information will be used as the supervision data (GT) to train the network model.

[0083] According to an embodiment of the present invention, the 3D face information regression model consists of two parallel Encoder-Decoder networks. The Encoder network structure consists of the multi-scale dual attention basic blocks, and the Decoder network structure consists of six layers of standard transposed convolutional blocks. The standard transposed convolutional blocks are composed of transposed convolution plus BN normalization. The output decoded feature map of the fifth layer of the standard transposed convolutional block in the two parallel networks serves as the input data of the similarity information enhancement module.

[0084] It should be noted that, in this embodiment, as Figure 2The diagram shows the 3D face regression model, which mainly consists of two parallel Encoder-Decoder networks. These two networks are used to directly regress 3D faces and face deformation information, respectively. They have the same network structure. The Encoder part consists of five layers of multi-scale dual-attention blocks. For an input face image of size "256x256x3", it first passes through a fully connected layer to transform its channel count to "16", and then is input into the Encoder. Each layer of the encoding network consists of a multi-scale dual-attention block with a stride of "2" and... The system consists of a multi-scale dual attention block with a stride of "1". The encoder outputs feature maps of size "8x8x512" from the last two parallel networks. These two output feature maps are concatenated along the channel dimension and then converted back to "512" channels using a standard convolutional layer consisting of "3" layers with a kernel size of "3" and a stride of "1". The standard convolutional layer is composed of convolution operations, BN normalization, and ReLU activation. The conversion result is used as the input to the decoder part of the branch network that directly regresses the 3D face, realizing the first enhancement using face deformation information.

[0085] Furthermore, the Decoder parts of both branch networks are composed of 6 layers of standard transposed convolutional blocks, which are formed by transposed convolution plus BN normalization. The first three layers of the Decoder in both branch networks contain one transposed convolutional block with a stride of 2 and two transposed convolutional blocks with a stride of 1. The last two layers are composed of one transposed convolutional block with a stride of 2 and one transposed convolutional block with a stride of 1. The last layer is composed of three transposed convolutional blocks with a stride of 1, which is responsible for converting the number of channels from 16 back to 3, so as to realize the end-to-end training and derivation process of the entire model. The 256x256x16 dimension decoded feature map output from the 5th layer of the Decoder part in both branch networks is sent to the similarity information enhancement module. The output of the similarity information enhancement module is used as the input feature map of the last layer of the Decoder part of the direct regression 3D face branch network, thereby realizing secondary enhancement using face deformation information.

[0086] According to an embodiment of the present invention, the establishment of the multi-scale dual attention base block and the similarity information enhancement module specifically includes:

[0087] The multi-scale dual attention block is composed of a 1×1 convolutional block, a 3×3 convolutional block, a 5×5 convolutional block, an SE channel attention module, and an SGE spatial grouping attention module.

[0088] After the face deformation information after projection transformation is saved in the form of UV map, r_D represents the enhanced information feature map and F represents the 3D face feature map to be enhanced.

[0089] The steps for establishing the similarity information enhancement module include:

[0090] The inner product is obtained by multiplying the enhanced information feature map and the 3D face feature map to be enhanced, and the inner product result is sent to the Sigmoid activation module to obtain the similarity probability distribution s';

[0091] The inner product of the similarity probability distribution s' and the 3D face feature map F to be enhanced is obtained to get the similarity information points on the 3D face feature map F. The similarity information points are added to the 3D face feature map F to complete the establishment of the similarity information enhancement module. The specific formula is as follows:

[0092] s' = Sigmoid(F·r_D);

[0093] F = F + (F·s′).

[0094] It should be noted that, in this embodiment, the implementation process of the multi-scale dual attention basic block is as follows: Figure 3 As shown, it is mainly composed of a carefully designed combination of 1x1 convolutional blocks, 3x3 convolutional blocks, 5x5 convolutional blocks, an SE channel attention module, and an SGE spatial grouping attention module. The implementation process of the multi-scale dual attention basic block can be described as follows: For an input feature map with dimensions H x W x C_in, it is fed into the processing flow of 1x1 scale convolution, 3x3 scale convolution, and 5x5 scale convolution respectively. Then, the feature maps processed by each scale convolution are concatenated and the number of channels is transformed by the 1x1 convolutional block before being fed into the SE channel attention module and the SGE spatial grouping attention module in sequence. The feature map obtained at this time after the multi-scale convolution operation and the dual attention module operation is added to the input feature map to obtain the output feature map of the entire multi-scale dual attention basic block.

[0095] Furthermore, before each scale of convolutional block operation, an additional 1x1 convolutional block is added to convert the number of channels and reduce the number of model parameters. For example, before the 5x5 convolutional block in this invention, the input feature map passes through a 1x1 convolutional block, and the feature dimension becomes H x W x C_out / 2. After passing through the 5x5 convolutional block, the feature map dimension returns to H x W x C_out. The operation flow of the 1x1 scale convolution and the 3x3 scale convolution is consistent with the above description of the 5x5 convolution. In addition, it should be noted that the convolutional blocks used in the entire multi-scale dual attention basic block are basically followed by a BN normalization operation and a ReLU activation operation after the convolution operation. Only in the 1x1 convolutional block following the concatenation result of the multi-scale convolution processing is only a BN normalization operation used. Therefore, the final output feature map is obtained by adding the residuals at the end of the entire multi-scale dual attention basic block processing flow and then using ReLU activation.

[0096] Furthermore, the SE channel attention module mentioned in the above embodiments can enhance the learning of useful features in the channel dimension while suppressing ineffective features. For example, in a face input image, it can enhance the feature extraction of the face region while reducing the model's attention to background information. The processing flow of the SE module can be described as follows: an input feature map with dimensions H x W x C is processed by a global average pooling operation to obtain a 1 x 1 x C feature map, which is then fed into a fully connected layer with C / 16 neurons. After ReLU activation, it is fed into a fully connected layer with C neurons, and then activated by Sigmoid to obtain a 1 x 1 x C feature channel descriptor. This descriptor is then multiplied with the original input feature map to obtain the final output.

[0097] Furthermore, this invention introduces the SGE spatial grouping attention mechanism. It is generally believed that the human face contains complex feature information, composed of many different sub-features, and these features are hierarchically distributed across different feature dimensions. Therefore, facial information can be grouped along the channel dimension to enhance the learning of sub-features. For example, in facial feature extraction, the SGE mechanism can further enhance the learning of detailed features such as the nose and eyes. The processing flow of the SGE module can be described as follows: an input feature map of size H x W x C is grouped along the channel axis, with the number of groups being "32". Then, the following identical operations are performed on each grouped feature map: first, a global average pooling operation is performed on the grouped feature map; then, the average pooling result and the grouped feature map are multiplied together to obtain an initial grouped attention mask; next, the attention mask is normalized, and then Sigmoid activation is used to obtain the final grouped attention mask; finally, the grouped feature map and the final grouped attention mask are multiplied together to obtain the final output.

[0098] In this embodiment, to further enhance the performance of the Encoder-Decoder network in regressing 3D faces, a similarity information enhancement module is introduced at the decoder end. As can be seen from the derivation process of the training supervision data (GT), the 3D face directly regressed by the model contains facial deformation information. In fact, during the learning process of the Encoder-Decoder model for 3D faces, facial pose transformation information is easily learned, but the model is relatively insensitive to changes in facial shape information. Therefore, an additional parallel network focused on learning facial deformation information can be set up. Obviously, the directly regressed 3D face should contain facial deformation information. Therefore, some information in the feature regression results of the network directly regressing the 3D face and the network directly regressing facial deformation information is quite similar. The feature information extracted from the two parallel networks is fed into the similarity information enhancement module proposed in this invention, thereby enhancing the deformation information on the directly regressed 3D face.

[0099] Furthermore, after saving the projected face deformation information as a UV map, r_D represents the enhanced information feature map, and F represents the 3D face feature map to be enhanced. The implementation process of the similarity information enhancement module can be described as follows: For the input 3D face feature map to be enhanced F and the enhanced information feature map r_D, they are first multiplied to obtain an inner product. The inner product result, i.e., the similarity, is fed into the Sigmoid activation module to obtain the similarity probability distribution s′. Then, the inner product of s′ and F is obtained to obtain the similarity information points on F. Adding this result to F achieves the information enhancement on F. This process can be referred to... Figure 4 Understanding, and describing it using symbolic formulas, is as follows:

[0100] s′=Sigmoid(F·r_D);

[0101] F = F + (F·s′);

[0102] According to an embodiment of the present invention, the step of supervising the training of the 3D face regression model using a preset loss function to obtain the 3D dense face alignment model specifically includes:

[0103] Supervisory data for network training is generated based on the aforementioned 3D vertex information of the face;

[0104] Using loss function L F Supervise the regression results of the 3D face subnetwork using the loss function L. r_D The regression results of the facial deformation information branch network were monitored;

[0105] Using loss function L lrr To supervise the alignment results of 3D faces in face alignment tasks;

[0106] Supervised training of the 3D face regression model to obtain the loss function L of the 3D dense face alignment model total The calculation formula is as follows: L total =ω F L F +ω r_D L r_D +ω lrr L lrr , where ω F ω r_D and ω lrr This is the corresponding loss factor.

[0107] It should be noted that, in this embodiment, after establishing the 3D face regression network model enhanced with face deformation information, the two Encoder-Decoder networks, one for directly regressing the 3D face and the other for regressing face deformation information, are trained simultaneously during the entire model training process. The output UV location map of the branch network for directly regressing the 3D face saves the final 3D face regression result of this model. The loss function L is used. F To supervise the regression results of the 3D face branch network in this invention, a loss function L is used. r_D Supervise the regression results of the face deformation information branch network in this invention. L F and L r_D The loss function can be substituted into the following form to calculate:

[0108]

[0109] Where N is either F or r_D, h and w represent the height and width of the UV map, respectively; N(u, v) represents the predicted 3D coordinate information of the point at position coordinate (u, v) in UV space; the symbol ^ represents the supervision data of the corresponding item; M(u, v) represents a preset weight value for the point at position coordinate (u, v). Here, the weight mask M is a mask designed according to the size of different regions of the face in a "16:12:3:0" ratio. The M value corresponding to the key feature points (landmarks) on the face is set to "16", the M value corresponding to the points on the eye, nose and mouth regions is set to "12", the M value corresponding to the points on the cheek, chin and forehead regions is set to "3", and the M value corresponding to the points on the neck region is set to "0". L is calculated using the above representation. F and L r_D .

[0110] Furthermore, to further improve the performance of the 3D faces regressed by the method of this invention on face alignment tasks, this invention also applies a constraint term on the regressed 3D faces regarding facial landmarks, using L... lrr Indicates. L lrr The specific representation is as follows:

[0111]

[0112] Where N = 68, and P(u, v) represents the three-dimensional coordinate information of the landmark point on the 3D face predicted by the model with position coordinates (u, v) in the corresponding UV space. It is the three-dimensional coordinate information of the landmark point on the 3D face with position coordinates (u, v) in the UV space corresponding to the supervised data.

[0113] In summary, the loss function of the network described in this method can be expressed by the following formula:

[0114] L total =ω F L F +ω r_D L r_D +ω lrr L lrr ;

[0115] Where, ω F =1, ω r_D =1, ω lrr =0.5.

[0116] Figure 5 A block diagram of a 3D dense face alignment model construction system according to the present invention is shown.

[0117] like Figure 5 As shown, this invention discloses a 3D dense face alignment model construction system, including a memory and a processor. The memory includes a 3D dense face alignment model construction method program. When the 3D dense face alignment model construction method program is executed by the processor, it performs the following steps:

[0118] Preprocessing the face images in the face dataset yields face deformation information;

[0119] A multi-scale dual attention basic block and a similarity information enhancement module are established to build a 3D face regression model using the face deformation information. The 3D face regression model includes at least the multi-scale dual attention basic block and the similarity information enhancement module.

[0120] The 3D face regression model is trained using a preset loss function to obtain the 3D dense face alignment model.

[0121] It should be noted that, in this embodiment, the 3D face regression model proposed in this application mainly consists of two parallel Encoder-Decoder branch networks. The two networks are used to directly regress the 3D face corresponding to the input face and to regress the deformation information of the input face relative to the average face template, respectively. In this invention, 3D face information and face deformation information are stored in the form of UV mapping. Furthermore, the two Encoder-Decoder networks have the same structure and depth. The Encoder part consists of the multi-scale dual attention basic blocks, and the Decoder part consists of standard transposed convolutional blocks. The feature maps output from the Encoder ends of the two branch networks are concatenated and fed into the Decoder in the network that directly regresses the 3D face, thereby achieving the first step of regressing the deformation information on the directly regressed 3D face. Secondary enhancement: Considering that some points on the directly regressed 3D face should have similar feature distributions to the directly regressed face deformation information, a similarity information enhancement module was constructed. The output feature maps of the Decoder ends of the two branch networks were fed into the similarity information enhancement module, and then the output feature map of the similarity information enhancement module was fed into the transposed output layer after the branch network of the directly regressed 3D face, realizing secondary enhancement of the deformation information on the 3D face. During the training of the 3D face regression model, the two Encoder-Decoder networks used for directly regressing the 3D face and face deformation information, respectively, were trained simultaneously. The output UV position map of the branch network for directly regressing the 3D face saves the final 3D face result regressed by this model. During the training process, the preset loss function was used for supervision. The loss function includes the loss function L. F Loss function L r_D and loss function L lrr The method proposed in this application includes two parts: model training and model derivation. In the dense face alignment task, the UV position map representing the final 3D face regression result and the three-dimensional face vertex information can be obtained by directly deriving the UV position map by loading the trained model, thus completing the dense face alignment task and completing the construction of the 3D dense face alignment model.

[0122] According to an embodiment of the present invention, the preprocessing of face images in the face dataset to obtain face deformation information specifically includes:

[0123] The face image is cropped based on the location information of facial feature points to convert the face image into a uniform size;

[0124] Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map;

[0125] Based on the average face model in the face dataset, a projection transformation is performed according to the target parameters to obtain the face deformation information after projection transformation, and the face deformation information after projection transformation is saved in the form of UV mapping.

[0126] It should be noted that, in this embodiment, the publicly available face dataset 300W-LP is used as an example, and the corresponding target parameter is the 3DMM parameter. First, the original input face images in the face dataset are standardized. In the 300W-LP face dataset, the facial landmarks of each face image have been pre-annotated manually based on the 3DMM method. Therefore, the cropping range can be determined based on the landmarks. All input face images are cropped to a size of "256x256". Then, the three-dimensional vertex information of each face image is obtained based on the annotation information. Based on the three-dimensional vertex information, the image coordinates are calculated from the world coordinates of the three-dimensional vertices according to the following formula:

[0127] F = f·R·S + t;

[0128]

[0129] Where f is the scaling factor, R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset, and S is the set of 3D vertices of the face in the world coordinate system. Here, D represents the set of vertices on the average face model, D represents the deformation information of the face shape in the face image, and F represents the set of three-dimensional vertices of the face, i.e., the set of three-dimensional vertices of the face in the image coordinate system. Based on the target parameters (3DMM parameters) in the 300W-LP face dataset, the three-dimensional vertex information of the face is calculated and saved into a UV map to obtain a UV position map. Then, the average face model provided in the 300W-LP dataset is projected and transformed according to the 3DMM parameter information. The difference between the UV position map and the projected average face model is calculated to obtain the face deformation information after the projection transformation. Similarly, the face deformation information after the projection transformation is saved in the form of a UV map. Preferably, the image coordinate system and the world coordinate system are common coordinate marking methods used by those skilled in the art when processing images, and will not be elaborated here.

[0130] Furthermore, according to an embodiment of the present invention, the difference between the UV position map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation, and the specific formula is as follows:

[0131]

[0132] Where r_D represents the face deformation information after projection transformation, F represents the set of three-dimensional vertices of the face, f is a scaling factor, and R and t are the rotation matrix and translation vector calculated from the target parameters in the face dataset. is the set of vertices on the average face model, and D is the deformation information of the face shape in the face image.

[0133] It should be noted that, in this embodiment, the face deformation information r_D after projection transformation is obtained by performing a difference operation between the UV position map and the average face model after projection transformation. Specifically, the set of three-dimensional vertices of the face in the image coordinate system is saved in the UV space to obtain the UV position map corresponding to the face image. The face deformation information after projection transformation is also saved in the form of a UV map. In this embodiment, the UV position map and / or the face deformation information will be used as the supervision data (GT) to train the network model.

[0134] According to an embodiment of the present invention, the 3D face information regression model consists of two parallel Encoder-Decoder networks. The Encoder network structure consists of the multi-scale dual attention basic blocks, and the Decoder network structure consists of six layers of standard transposed convolutional blocks. The standard transposed convolutional blocks are composed of transposed convolution plus BN normalization. The output decoded feature map of the fifth layer of the standard transposed convolutional block in the two parallel networks serves as the input data of the similarity information enhancement module.

[0135] It should be noted that, in this embodiment, as Figure 2The diagram shows the 3D face regression model, which mainly consists of two parallel Encoder-Decoder networks. These two networks are used to directly regress 3D faces and face deformation information, respectively. They have the same network structure. The Encoder part consists of five layers of multi-scale dual-attention blocks. For an input face image of size "256x256x3", it first passes through a fully connected layer to transform its channel count to "16", and then is input into the Encoder. Each layer of the encoding network consists of a multi-scale dual-attention block with a stride of "2" and... The system consists of a multi-scale dual attention block with a stride of "1". The encoder outputs feature maps of size "8x8x512" from the last two parallel networks. These two output feature maps are concatenated along the channel dimension and then converted back to "512" channels using a standard convolutional layer consisting of "3" layers with a kernel size of "3" and a stride of "1". The standard convolutional layer is composed of convolution operations, BN normalization, and ReLU activation. The conversion result is used as the input to the decoder part of the branch network that directly regresses the 3D face, realizing the first enhancement using face deformation information.

[0136] Furthermore, the Decoder parts of both branch networks are composed of 6 layers of standard transposed convolutional blocks, which are formed by transposed convolution plus BN normalization. The first three layers of the Decoder in both branch networks contain one transposed convolutional block with a stride of 2 and two transposed convolutional blocks with a stride of 1. The last two layers are composed of one transposed convolutional block with a stride of 2 and one transposed convolutional block with a stride of 1. The last layer is composed of three transposed convolutional blocks with a stride of 1, which is responsible for converting the number of channels from 16 back to 3, so as to realize the end-to-end training and derivation process of the entire model. The 256x256x16 dimension decoded feature map output from the 5th layer of the Decoder part in both branch networks is sent to the similarity information enhancement module. The output of the similarity information enhancement module is used as the input feature map of the last layer of the Decoder part of the direct regression 3D face branch network, thereby realizing secondary enhancement using face deformation information.

[0137] According to an embodiment of the present invention, the establishment of the multi-scale dual attention base block and the similarity information enhancement module specifically includes:

[0138] The multi-scale dual attention block is composed of a 1×1 convolutional block, a 3×3 convolutional block, a 5×5 convolutional block, an SE channel attention module, and an SGE spatial grouping attention module.

[0139] After the face deformation information after projection transformation is saved in the form of UV map, r_D represents the enhanced information feature map and F represents the 3D face feature map to be enhanced.

[0140] The steps for establishing the similarity information enhancement module include:

[0141] The inner product is obtained by multiplying the enhanced information feature map and the 3D face feature map to be enhanced, and the inner product result is sent to the Sigmoid activation module to obtain the similarity probability distribution s';

[0142] The inner product of the similarity probability distribution s' and the 3D face feature map F to be enhanced is obtained to get the similarity information points on the 3D face feature map F. The similarity information points are added to the 3D face feature map F to complete the establishment of the similarity information enhancement module. The specific formula is as follows:

[0143] s' = Sigmoid(F·r_D);

[0144] F = F + (F·s′).

[0145] It should be noted that, in this embodiment, the implementation process of the multi-scale dual attention basic block is as follows: Figure 3 As shown, it is mainly composed of a carefully designed combination of 1x1 convolutional blocks, 3x3 convolutional blocks, 5x5 convolutional blocks, an SE channel attention module, and an SGE spatial grouping attention module. The implementation process of the multi-scale dual attention basic block can be described as follows: For an input feature map with dimensions H x W x C_in, it is fed into the processing flow of 1x1 scale convolution, 3x3 scale convolution, and 5x5 scale convolution respectively. Then, the feature maps processed by each scale convolution are concatenated and the number of channels is transformed by the 1x1 convolutional block before being fed into the SE channel attention module and the SGE spatial grouping attention module in sequence. The feature map obtained at this time after the multi-scale convolution operation and the dual attention module operation is added to the input feature map to obtain the output feature map of the entire multi-scale dual attention basic block.

[0146] Furthermore, before each scale of convolutional block operation, an additional 1x1 convolutional block is added to convert the number of channels and reduce the number of model parameters. For example, before the 5x5 convolutional block in this invention, the input feature map passes through a 1x1 convolutional block, and the feature dimension becomes H x W x C_out / 2. After passing through the 5x5 convolutional block, the feature map dimension returns to H x W x C_out. The operation flow of the 1x1 scale convolution and the 3x3 scale convolution is consistent with the above description of the 5x5 convolution. In addition, it should be noted that the convolutional blocks used in the entire multi-scale dual attention basic block are basically followed by a BN normalization operation and a ReLU activation operation after the convolution operation. Only in the 1x1 convolutional block following the concatenation result of the multi-scale convolution processing is only a BN normalization operation used. Therefore, the final output feature map is obtained by adding the residuals at the end of the entire multi-scale dual attention basic block processing flow and then using ReLU activation.

[0147] Furthermore, the SE channel attention module mentioned in the above embodiments can enhance the learning of useful features in the channel dimension while suppressing ineffective features. For example, in a face input image, it can enhance the feature extraction of the face region while reducing the model's attention to background information. The processing flow of the SE module can be described as follows: an input feature map with dimensions H x W x C is processed by a global average pooling operation to obtain a 1 x 1 x C feature map, which is then fed into a fully connected layer with C / 16 neurons. After ReLU activation, it is fed into a fully connected layer with C neurons, and then activated by Sigmoid to obtain a 1 x 1 x C feature channel descriptor. This descriptor is then multiplied with the original input feature map to obtain the final output.

[0148] Furthermore, this invention introduces the SGE spatial grouping attention mechanism. It is generally believed that the human face contains complex feature information, composed of many different sub-features, and these features are hierarchically distributed across different feature dimensions. Therefore, facial information can be grouped along the channel dimension to enhance the learning of sub-features. For example, in facial feature extraction, the SGE mechanism can further enhance the learning of detailed features such as the nose and eyes. The processing flow of the SGE module can be described as follows: an input feature map of size H x W x C is grouped along the channel axis, with the number of groups being "32". Then, the following identical operations are performed on each grouped feature map: first, a global average pooling operation is performed on the grouped feature map; then, the average pooling result and the grouped feature map are multiplied together to obtain an initial grouped attention mask; next, the attention mask is normalized, and then Sigmoid activation is used to obtain the final grouped attention mask; finally, the grouped feature map and the final grouped attention mask are multiplied together to obtain the final output.

[0149] In this embodiment, to further enhance the performance of the Encoder-Decoder network in regressing 3D faces, a similarity information enhancement module is introduced at the decoder end. As can be seen from the derivation process of the training supervision data (GT), the 3D face directly regressed by the model contains facial deformation information. In fact, during the learning process of the Encoder-Decoder model for 3D faces, facial pose transformation information is easily learned, but the model is relatively insensitive to changes in facial shape information. Therefore, an additional parallel network focused on learning facial deformation information can be set up. Obviously, the directly regressed 3D face should contain facial deformation information. Therefore, some information in the feature regression results of the network directly regressing the 3D face and the network directly regressing facial deformation information is quite similar. The feature information extracted from the two parallel networks is fed into the similarity information enhancement module proposed in this invention, thereby enhancing the deformation information on the directly regressed 3D face.

[0150] Furthermore, after saving the projected facial deformation information as a UV map, r_D represents the enhanced information feature map, and F represents the 3D facial feature map to be enhanced. The implementation process of the similarity information enhancement module can be described as follows: For the input 3D facial feature map to be enhanced F and the enhanced information feature map r_D, they are first multiplied to obtain an inner product. The inner product result, i.e., the similarity, is fed into the Sigmoid activation module to obtain the similarity probability distribution s'. Then, the inner product of s' and F is obtained to obtain the similarity information points on F. Adding this result to F achieves the information enhancement on F. This process can be referred to... Figure 4 Understanding, and describing it using symbolic formulas, is as follows:

[0151] s' = Sigmoid(F·r_D);

[0152] F = F + (F·s');

[0153] According to an embodiment of the present invention, the step of supervising the training of the 3D face regression model using a preset loss function to obtain the 3D dense face alignment model specifically includes:

[0154] Supervisory data for network training is generated based on the aforementioned 3D vertex information of the face;

[0155] Using loss function L F Supervise the regression results of the 3D face subnetwork using the loss function L. r_D The regression results of the facial deformation information branch network were monitored;

[0156] Using loss function L lrr To supervise the alignment results of 3D faces in face alignment tasks;

[0157] Supervised training of the 3D face regression model to obtain the loss function L of the 3D dense face alignment model total The calculation formula is as follows: L total =ω F L F +ω r_D L r_D +ω lrr L lrr , where ω F ω r_D and ω lrr This is the corresponding loss factor.

[0158] It should be noted that, in this embodiment, after establishing the 3D face regression network model enhanced with face deformation information, the two Encoder-Decoder networks, one for directly regressing the 3D face and the other for regressing face deformation information, are trained simultaneously during the entire model training process. The output UV location map of the branch network for directly regressing the 3D face saves the final 3D face regression result of this model. The loss function L is used. F To supervise the regression results of the 3D face branch network in this invention, a loss function L is used. r_D Supervise the regression results of the face deformation information branch network in this invention. L F and L r_D The loss function can be substituted into the following form to calculate:

[0159]

[0160] Where N is either F or r_D, h and w represent the height and width of the UV map, respectively; N(u, v) represents the predicted 3D coordinate information of the point at position coordinate (u, v) in UV space; the symbol ^ represents the supervision data of the corresponding item; M(u, v) represents a preset weight value for the point at position coordinate (u, v). Here, the weight mask M is a mask designed according to the size of different regions of the face in a "16:12:3:0" ratio. The M value corresponding to the key feature points (landmarks) on the face is set to "16", the M value corresponding to the points on the eye, nose and mouth regions is set to "12", the M value corresponding to the points on the cheek, chin and forehead regions is set to "3", and the M value corresponding to the points on the neck region is set to "0". L is calculated using the above representation. F and L r_D .

[0161] Furthermore, to further improve the performance of the 3D faces regressed by the method of this invention on face alignment tasks, this invention also applies a constraint term on the regressed 3D faces regarding facial landmarks, using L... lrr Indicates. L lrr The specific representation is as follows:

[0162]

[0163] Where N = 68, and P(u, v) represents the three-dimensional coordinate information of the landmark point on the 3D face predicted by the model with position coordinates (u, v) in the corresponding UV space. It is the three-dimensional coordinate information of the landmark point on the 3D face with position coordinates (u, v) in the UV space corresponding to the supervised data.

[0164] In summary, the loss function of the network described in this method can be expressed by the following formula:

[0165] L total =ω F L F +ω r_D L r_D +ω lrr L lrr ;

[0166] Where, ω F =1, ω r_D =1, ω lrr =0.5.

[0167] A third aspect of the present invention provides a computer-readable storage medium comprising a 3D dense face alignment model construction method program, wherein when the 3D dense face alignment model construction method program is executed by a processor, it implements the steps of a 3D dense face alignment model construction method as described in any of the preceding claims.

[0168] This invention discloses a method, system, and readable storage medium for constructing a 3D dense face alignment model. Based on deep learning technology with strong fitting capabilities, it utilizes face deformation information enhancement technology to provide an efficient end-to-end training model for further improving the accuracy of 3D dense face alignment tasks. It can enhance the 3D face corresponding to the face in the input image by utilizing the face deformation information, and obtain a more accurate 3D dense face alignment model framework.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0170] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0172] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for constructing a 3D dense face alignment model, characterized in that, Includes the following steps: The face image is cropped and converted to a uniform size based on the location information of the facial feature points; Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map; The difference between the UV location map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation, and then saved in the form of a UV map. The formula is as follows: ； in, This represents the facial deformation information after projection transformation. Represents the set of three-dimensional vertices of a human face. It is the scaling factor. and It is the rotation matrix and translation vector obtained from the face dataset after calculating the target parameters. It is the set of vertices on the average face model. It is the deformation information of the face shape in the face image; A multi-scale dual attention basic block and a similarity information enhancement module are established to build a 3D face regression model using face deformation information. The 3D face regression model consists of two parallel Encoder-Decoder network structures. Supervised data for generating networks based on 3D vertex information of faces; Using loss function Supervise the regression results of the 3D face subnetwork using the loss function. The regression results of the facial deformation information branch network were monitored; , represent or , and These represent the height and width of the UV map, respectively. This represents the position coordinates in UV space. The three-dimensional coordinate information predicted by the model of the point; symbol The corresponding supervisory data is represented; The position coordinates are represented as A preset weight value for the point; Using loss function To supervise the alignment results of 3D faces in face alignment tasks; Supervised training of a 3D face regression model to obtain the loss function of a 3D dense face alignment model. The calculation formula is as follows: Among them, loss factor =1、 =1、 =0.5; Apply a constraint term on facial landmarks to the regressed 3D face, using... express: ； Where N=68, The corresponding position coordinates in UV space are: The model predicts the 3D coordinates of landmark points on a 3D human face. The corresponding position coordinates in the UV space in the supervised data are The three-dimensional coordinate information of landmark points on a 3D human face.

2. The method for constructing a 3D dense face alignment model according to claim 1, characterized in that, The establishment of the multi-scale dual attention base block and the similarity information enhancement module specifically includes: The multi-scale dual attention block is composed of a 1×1 convolutional block, a 3×3 convolutional block, a 5×5 convolutional block, an SE channel attention module, and an SGE spatial grouping attention module. After saving the projected facial deformation information as a UV map, the steps for establishing the similarity information enhancement module include: The inner product is obtained by multiplying the face deformation information after projection transformation and the face 3D vertex set, and the inner product result is sent to the Sigmoid activation module to obtain the similarity probability distribution. The similarity probability distribution is calculated as the inner product of the three-dimensional vertex set of the face to obtain similarity information points on the three-dimensional vertex set of the face. The similarity information points are then added to the three-dimensional vertex set of the face to complete the establishment of the similarity information enhancement module.

3. A 3D dense face alignment model construction system, characterized in that, The system includes a memory and a processor. The memory contains a 3D dense face alignment model construction method program. When the processor executes the 3D dense face alignment model construction method program, it performs the following steps: The face image is cropped and converted to a uniform size based on the location information of the facial feature points; Based on the target parameters in the face dataset, the 3D vertex information of the face is calculated and saved into the UV map to obtain the UV position map; The difference between the UV location map and the average face model after projection transformation is calculated to obtain the face deformation information after projection transformation, and then saved in the form of a UV map. The formula is as follows: ； in, This represents the facial deformation information after projection transformation. Represents the set of three-dimensional vertices of a human face. It is the scaling factor. and It is the rotation matrix and translation vector obtained from the face dataset after calculating the target parameters. It is the set of vertices on the average face model. It is the deformation information of the face shape in the face image; A multi-scale dual attention basic block and a similarity information enhancement module are established to build a 3D face regression model using face deformation information. The 3D face regression model consists of two parallel Encoder-Decoder network structures. Supervised data for generating networks based on 3D vertex information of faces; Using loss function Supervise the regression results of the 3D face subnetwork using the loss function. The regression results of the facial deformation information branch network were monitored; , represent or , and These represent the height and width of the UV map, respectively. This represents the position coordinates in UV space. The three-dimensional coordinate information predicted by the model of the point; symbol The corresponding supervisory data is represented; The position coordinates are represented as A preset weight value for the point; Using loss function To supervise the alignment results of 3D faces in face alignment tasks; Supervised training of a 3D face regression model to obtain the loss function of a 3D dense face alignment model. The calculation formula is as follows: Among them, loss factor =1、 =1、 =0.5; Apply a constraint term on facial landmarks to the regressed 3D face, using... express: ； Where N=68, The corresponding position coordinates in UV space are: The model predicts the 3D coordinates of landmark points on a 3D human face. The corresponding position coordinates in the UV space in the supervised data are The three-dimensional coordinate information of landmark points on a 3D human face.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a 3D dense face alignment model construction method program, which, when executed by a processor, implements the steps of a 3D dense face alignment model construction method as described in any one of claims 1 to 2.

Images