An end-to-end 3D face recognition system based on data augmentation structure

Abstract

The application discloses an end-to-end 3D face recognition system based on a data augmentation structure, and has the characteristics of comprising a training set generation module, an end-to-end network training module and an end-to-end 3D face recognition network. The technical scheme disclosed by the application optimizes the feature fusion and feature extraction module of the existing end-to-end 3D face recognition network to obtain the end-to-end 3D face recognition network, and uses the improved SSM to obtain the GSSM to train the IMPNet, so that the precision and inference speed of face recognition are greatly improved. The application creatively proposes a GSSM data augmentation structure. The application proposes a multi-scale feature average division and insertion fusion strategy in the IMPNet. The application reconstructs the SA module. The Pointnet in the SA module of the original Pointnet++ backbone network is reconstructed into MLP and MAX, so that the model parameters are reduced, and the training speed is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to an end-to-end 3D face recognition system, belonging to the field of artificial intelligence-3D face recognition technology. BACKGROUND

[0002] Face recognition as a non-invasive biometric authentication method is widely used in criminal investigation, autonomous driving, medical care, finance and other fields. 2D face recognition technology is greatly affected by factors such as posture, expression, and lighting, while the inherent structural characteristics of 3D face (containing depth and topological information) make the above unfavorable recognition environment have less impact on 3D face recognition. Therefore, 3D face recognition has attracted much attention, and end-to-end 3D face recognition based on deep learning is currently a hot research and application topic.

[0003] Currently, researchers mainly expand the training data for training 3D face recognition network through single 2D image reconstruction based on GAN and 3DMM reconstruction. In 2021, Toshpulatov discussed using GAN network to reconstruct a single 2D face into a 3D face and use it as a training set for 3D face recognition network. However, the results of GAN network reconstruction are highly homogeneous, and the similarity of faces reconstructed from different 2D images is large. In 2017, Dou used 3DMM to reconstruct 3D face, input the feature points of the face into the 3DMM model, adjust the model parameters, and obtain the 3D face. However, 3DMM is sensitive to the accuracy of feature points, and there is still a large gap in the reconstruction results of the same person multiple times, and the generalization ability of the reconstruction is poor. Inspired by 3DMM, SSM directly performs principal component analysis mapping on 3D real faces, and obtains different parameters through statistical analysis, and finally generates training data. At the same time, the faces generated by SSM are not sensitive to the accuracy of feature points and the generated faces have rich expressions. However, SSM is limited by 3D real face data, and the expression ability of the model is very limited. SUMMARY

[0004] The technical problem to be solved by the present application is that the expansion of training data for training 3D face recognition network through single 2D image reconstruction based on GAN exists the problem of serious homogenization, and the expansion of training data for training 3D face recognition network based on 3DMM reconstruction exists the problem of poor generalization ability of reconstruction.

[0005] In order to solve the above calculation problem, the technical scheme of the present application provides an end-to-end 3D face recognition system based on data augmentation structure, characterized in that it comprises a training set generation module, an end-to-end network training module and an end-to-end 3D face recognition network, wherein:

[0006] The training set generation module generates a training set for training the IMPNet based on an existing reference data set. The training set generation module further includes a generator GDAS. After the reference data set is divided into a face shape, a face expression, and a face texture, the generator GDAS respectively establishes a face shape GSSM model, a face expression GSSM model, and a face texture GSSM model. After the generator GDAS derives a statistical shape model from m 3D faces, the statistical shape model is subjected to Gaussian regression and K-L expansion to obtain the face shape GSSM model, the face expression GSSM model, or the face texture GSSM model. The generator GDAS linearly combines the face shape GSSM model, the face expression GSSM model, and the face texture GSSM model to obtain a final face model F new The training set generation module generates a training set for training the IMPNet based on the face model F new

[0007] The end-to-end network training module trains an end-to-end 3D face recognition network using the training set. The end-to-end 3D face recognition network takes a point cloud as input and outputs a 1×512 feature vector. The feature vector is subjected to a dropout layer and a fully connected layer to obtain a final classification result. The end-to-end 3D face recognition network includes an SA module one, an SA module two, and a feature fusion module. The SA module one performs feature fusion and feature extraction based on MLP operation and MAX operation to generate output. The output of the SA module one is input into the SA module two, and a 1×256 feature vector one is obtained after the output of the SA module one is subjected to a MAX operation again. The SA module two performs feature fusion and feature extraction based on MLP operation and MAX operation to generate output. A 1×256 feature vector two is obtained after the output of the SA module two is subjected to a MAX operation again. The feature fusion module fuses the 1×256 feature vector one and the 1×256 feature vector two into a 1×512 feature vector.

[0008] Preferably, the face shape GSSM model is represented as s(x), the face expression GSSM model is represented as e(x), and the face texture GSSM model is represented as t(x). Then,

[0009]

[0010] In the formula, μ s (x), μ e (x), and μ t (x) are mean functions; d s , d e , and d t are eigenvalues corresponding to basis functions after K-L expansion; and U​s , U e , U t are the basis function sets, respectively responsible for the generation of face shape, expression and texture; and a, b, g are parameter vectors.

[0011] Preferably, the generator GDAS linearly combines the face shape GSSM model, the face expression GSSM model and the face texture GSSM model by the following formula:

[0012]

[0013] In the formula, m1, m2, m3 represent deformation coefficients, and the three parameters are independent of each other.

[0014] Preferably, obtaining the feature vector one or the feature vector two by the SA module one or the SA module two comprises the following steps:

[0015] Step 101, performing farthest point sampling on the input of the SA module one or the SA module two to obtain N1 points;

[0016] Step 102, grouping: taking the N1 points as the centers of circles, the radius of the circle is R, and K points are taken in each obtained circle, thereby obtaining N1 grouped point sets

[0017] Step 103, the N1 grouped point sets are sent to the MLP, and the N1 grouped point sets are subjected to MLP dimension conversion operation;

[0018] Step 104, the N1 grouped point sets are subjected to MAX operation respectively to obtain the output of the SA module one or the SA module two

[0019] Step 105, the is subjected to MAX operation again to obtain a 1x256 feature vector.

[0020] Preferably, the feature fusion module fuses the 1x256 feature vector one and the 1x256 feature vector two into a 1x512 feature vector by multi-scale feature average interpolation fusion, specifically comprising the following steps:

[0021] Step 201, the 1x256 feature vector one is expressed as X=[x0, x1,..., x 255 ];

[0022] The 1x256 feature vector two is expressed as Y=[y0, y1,..., y 255 ];

[0023] The 1*512 feature vector is defined as Z = [z0, z1,..., z 511 ];

[0024] Step 202, initialize n = 0;

[0025] Step 203, if n < 256, then go to step 204; otherwise, go to step 206;

[0026] Step 204, z 2n+1 = x n ; z n = y n ;

[0027] Step 205, n = n + 1, return to step 203;

[0028] Step 206, obtain the final 1*512 feature vector Z.

[0029] Preferably, the end-to-end network training module adopts an angular loss function to represent the distance between a predicted category and a label category when training the end-to-end 3D face recognition network, and the smaller the value of the angular loss function, the better the classification effect.

[0030] Preferably, the angular loss function has the following expression form:

[0031]

[0032] In the formula, N represents the number of samples participating in loss calculation; θ i represents the included angle between the weight vector of the end-to-end 3D face recognition network and the fusion face feature obtained by the end-to-end 3D face recognition network; (θ i , y i + m) represents the included angle between θ i and y i + m, y i represents the fusion face feature obtained by the end-to-end 3D face recognition network, m represents an angle penalty value for increasing the distance between classes and reducing the distance within a class; s represents a deformation vector; and z represents a vector obtained after L2 regularization of the fusion face feature.

[0033] The technical solution disclosed in the present application optimizes the feature fusion and feature extraction module (hereinafter referred to as “SA module”) of the existing end-to-end 3D face recognition network (hereinafter referred to as “IMPNet”), and trains the IMPNet by improving the SSM to obtain the GSSM, so that the accuracy and inference speed of face recognition are greatly improved.

[0034] Compared with the prior art, the present application has the following innovations:

[0035] (1) Creatively propose GSSM data augmentation structure. In the training phase of IMPNet, the GSSM data augmentation structure obtains a large amount of training data from a small amount of real data. First, the face GSSM is established, and then the model is linearly combined and the combination coefficient is adjusted to generate a large amount of training data.

[0036] (2) Propose a multi-scale feature average interval fusion strategy in IMPNet. When fusing features of different scales, first block the features, then average interval fusion, thereby increasing the correlation between similar features and the irrelevance of distinguishing features, and improving the network classification effect.

[0037] (3) Reconstruct the SA module. Reconstruct the Pointnet in the SA module of the original Pointnet++ backbone network into MLP and MAX, reduce the model parameters, and improve the training speed. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 Diagram of end-to-end 3D face recognition algorithm with GSSM data augmentation structure;

[0039] Figure 2 Illustrates the data augmentation structure diagram;

[0040] Figure 3a Illustrates the SSM base vector In the xoy projection;

[0041] Figure 3b Illustrates the base function φ of the GSSM data augmentation model i (x) in the xoy projection;

[0042] Figure 4a Illustrates the base function set U of the face shape GSSM s In the xoy projection;

[0043] Figure 4b Illustrates the base function set U of the face expression GSSM e In the xoy projection;

[0044] Figure 4c Illustrates the base function set U of the face texture GSSM t In the xoy projection;

[0045] Figure 4d Illustrates the base function of F new in the xoy projection;

[0046] Figure 5 Illustrates the SSM and GSSM generated face comparison;

[0047] Figure 6 The structure of IMPNet is illustrated;

[0048] Figure 7 An illustration of multi-scale feature average split interpolation fusion;

[0049] Figure 8 The improved SA module is illustrated;

[0050] Figure 9 The original SA module of Pointnet++ is illustrated;

[0051] Figure 10 The ROC curve on the FRGCv2 dataset is illustrated;

[0052] Figure 11 The ROC curve of the Bosphorus dataset is illustrated;

[0053] Figure 12 The training efficiency under different strategies is illustrated. DETAILED DESCRIPTION

[0054] The application will be further described below in connection with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not used to limit the scope of the application. Furthermore, it should be understood that after reading the content taught by the application, those skilled in the art can make various modifications or changes to the application, and these equivalent forms also fall within the scope of the appended claims of the application.

[0055] In combination Figure 1 , the embodiment discloses an end-to-end 3D face recognition system based on a data augmentation structure, which comprises a training set generation module, an end-to-end network training module and an IMPNet. The training set generation module generates a training set for training the IMPNet based on an existing reference data set, and the end-to-end network training module trains the IMPNet using the training set.

[0056] Unlike traditional 3D face recognition algorithms, the training set generation module in the application generates a training set based on a GSSM data augmentation structure, which can generate effective training data by itself and improve the recognition performance of the network. In addition, the IMPNet adopts a brand-new feature fusion strategy (i.e. a multi-scale feature average split interpolation fusion strategy) and a redesigned SA module, which significantly improves the performance of the end-to-end network.

[0057] In this embodiment, the reference data set used is the 513 real human faces in the spring subset of FRGCv2, the data in the reference data set is preprocessed to form a densely aligned point cloud, and the densely aligned point cloud is input into the GSSM data augmentation structure of the training set generation module, so that a large amount of data for training the IMPNet can be obtained, and the training set is generated. For the trained IMPNet, the verification data set is used for verification again. In the test stage of the IMPNet, whether it belongs to the same face is judged by calculating the cosine distance between the 256-dimensional feature vector output by the IMPNet and the 256-dimensional feature vector in the Gallery feature set (the vector obtained by the first neutral face of each object in the test set through the IMPNet).

[0058] The training set generation module, the end-to-end network training module and the IMPNet will be introduced in detail below.

[0059] I) Training set generation module

[0060] A statistical shape model (hereinafter referred to as “SSM”) is derived from m 3D human faces, which makes the SSM subject to the PCA generated space of the data, and the expression ability of the model is very limited. In order to improve the expression ability of the model, the training set generation module performs Gaussian regression (GPR) on multiple SSMs (subject to Gaussian distribution) to obtain a GSSM data augmentation model, which effectively improves the expression and generation ability of the model.

[0061] The original SSM is represented as:

[0062]

[0063] In formula (1), s represents a human face point cloud vector, represents an average face, α i represents a parameter vector, λ i represents a basis vector corresponding to the eigenvalue, represents a basis vector.

[0064] The SSM is regressed as:

[0065]

[0066] In the formula, s1, s2, …, s n represent multiple human face point cloud vectors, μ1, μ2, …, μ n represent average faces, Σ1, Σ2, …, Σ n represent covariances, x1, x2, …, x n represent independent variables, s(x1), s(x2), …, s(x n ) represents a function value, and N(·) represents a Gaussian distribution.

[0067] Since s(x1), s(x2), …, s(x n ) are independent of each other, any s(x n ) obeys a joint Gaussian distribution, and the following is obtained:

[0068]

[0069] In formula (3), s(·) represents a random variable function, μ(·) represents a mean function, K(·) represents a covariance function, x* represents a known finite-dimensional random variable, and x represents an infinite-dimensional random variable to be solved. Formula (3) is the process of Gaussian regression, and the basis of this process is that any s(x n ) obeys a joint Gaussian distribution

[0070] s(x)|s(x*)~N(μ(x),∑(x)) (4)

[0071] In formula (4), μ(x)=K(x,x*)K(x*,x*) -1 s(x*),∑(x)=K(x,x)-K(x,x*)K(x*,x*) -1 K(x*,x)。

[0072] The K-L expansion of formula (4) is a GSSM data augmentation model, as shown in the following formula (5).

[0073]

[0074] In formula (5), λ i represents the eigenvalue corresponding to φ i (x), φ i (x) represents a basis function, GP(μ,Σ) represents a Gaussian random process, μ represents a mean function, and Σ represents a covariance function.

[0075] The basis vector directly affects the generation effect of the model, and the basis vector of the SSM and the basis function of the GSSM data augmentation model are visualized respectively. In order to compare more clearly, the basis space is projected onto the xoy plane, as shown in Figure 3a and Figure 3b Compared with the SSM, the vector space of the GSSM data augmentation model is more abundant, and more refined face generation can be given. However, for the generation of face data, not only the shape factor needs to be considered, but also the expression and texture factors need to be considered.

[0076] The application establishes an unsupervised generator GDAS, and the input of the generator GDAS is a 3D face point cloud that has been densely aligned, and the output is a 3D face scan, as shown in Figure 2The GDAS comprehensively considers the shape, expression and texture factors, and respectively establishes the face shape GSSM, expression GSSM and texture GSSM. According to the properties of the Gaussian random process, the above three GSSMs are linearly combined to obtain the generated face.

[0077] The reference dataset is divided into shape faces, expression faces and texture faces, and the face shape GSSM s(x), expression GSSM e(x) and texture GSSM t(x) are respectively established by the generator GDAS:

[0078]

[0079] In formula (6), μ s (x), μ e (x) and μ t (x) are mean functions; d s , d e and d t are eigenvalues corresponding to basis functions after K-L expansion; U s , U e and U t are basis function sets (the initial values are statistical median functions), which are respectively responsible for the generation of face shape, expression and texture; α, β and γ are parameter vectors of the three models.

[0080] The visualization of the xoy projection is shown in Figure 4a , Figure 4b , Figure 4c It can be found that the projection of U s is mainly distributed between the 45-degree line (dashed line) and the 135-degree line (dashed line) and the y-axis; the projection of U e is mainly distributed between the 135-degree line and the x-axis; and the projection of U t is mainly distributed between the 45-degree line and the x-axis.

[0081] In order to make the generated face comprehensively consider the shape, expression and texture factors, it is necessary to make the basis functions of the model cover all regions as much as possible. Therefore, the linear combination of the three GSSMs is:

[0082]

[0083] In formula (7), μ1, μ2 and μ3 represent the deformation coefficients of the three models, and the three parameters are independent of each other and are optimized through statistical analysis. F new The visualization of the basis functions of F Figure 4d is shown in the figure, and it can be seen that the basis functions cover the shape, expression and texture regions, and have comprehensive generation performance.

[0084] The face generated by the SSM and the face generated by the GDAS are shown in Figure 5The details generated by GDAS are rich (especially at the eyes, nose, and mouth), and more realistic. The face generated by SSM lacks texture information, and the expression is also relatively rigid. The experimental results show that, compared with the current mainstream 3D face generation algorithm, the face error generated by the structure is the lowest, as shown in Table 1, and the addition of the structure can greatly improve the training effect of the network, as shown in Table 2. With the increase of generated data, the performance of the network is stronger, as shown in Table 2.

[0085] Table 1 Comparison of different generation algorithms value

[0086]

[0087] Table 2 Comparison of Rank-1 recognition rates of different generation algorithms

[0088]

[0089] Note: B represents Bosphorus, and F represents FRGCv2.

[0090] II) IMPNet

[0091] IMPNet adopts an end-to-end design, directly takes point cloud as input, and outputs a 512-dimensional feature vector. IMPNet can be represented as follows:

[0092]

[0093] In equation (8): is an unordered input point cloud, N0 is the number of points; L∈R 512 is the output feature.

[0094] The specific process of IMPNet is as follows:

[0095] Step 1, input the original point cloud (N, C1) into the network, where N is the number of point clouds, and C1 is the dimension;

[0096] Step 2, the original point cloud (N, C1) is sent to the first SA module, i.e. ISA0 module, to obtain a 1x256 feature vector through the ISA0 module, including the following steps:

[0097] Step 201, perform farthest point sampling (FPS) on the original point cloud (N, C1) to obtain N1 points;

[0098] Step 202, grouping: take N1 points as the center of the circle, and draw a circle with a radius R, and take K points in each obtained circle to obtain N1 grouped point sets

[0099] Step 203, N1 group point sets are sent into the MLP, and the N1 group point sets are processed MLP dimension conversion operations are performed;

[0100] Step 204, the N1 group point sets are processed again MAX operations are performed respectively to obtain

[0101] Step 205, the 1x256 feature vector is processed through a MAX operation.

[0102] Step 3, the 1x256 feature vector obtained by the ISA0 module is sent into a second SA module, that is, the ISA1 module The processing steps of the ISA0 module are the same as steps 201 to 205.

[0103] Step 4, the two 1x256 feature vectors obtained in steps 2 and 3 are sent into a multi-scale feature average division and insertion fusion module to obtain a final 1x512 feature vector.

[0104] Step 5, the 1x512 feature vector is subjected to dropout (0.5) and a full connection layer to obtain a final classification result, as shown in Figure 6 Table 3 is the meanings and selected values of related symbols in the table. Figure 6

[0105] Table 3 Parameter settings of the IMPNet

[0106]

[0107] The IMPNet proposed in the application extends the backbone network of the Pointnet++, and the following improvements are made in view of the problems of slow inference speed and low training efficiency of the Pointnet++: 1, a multi-scale feature average division and insertion fusion strategy is proposed to reduce the training time, improve the inference speed and recognition accuracy; 2, the SA module in the Pointnet++ network is reconstructed, the Pointnet++ module is optimized, the network parameters are reduced, and the inference speed is accelerated. The following will introduce the two points.

[0108] 1, multi-scale feature average division and insertion fusion strategy

[0109] In the process of point cloud feature information processing, feature information fusion is an indispensable step to improve the network classification performance. Unlike traditional feature splicing, the application proposes a new feature fusion strategy, that is, a multi-scale feature average division and insertion fusion strategy. Figure 6 ​​Pointnet++ uses max pooling to aggregate point cloud features into a vector, which makes the originally independent point cloud establish relevance with each point after MAX operation. In the process of feature fusion, the relevance is consciously used to reduce the randomness of features, reduce the training time and improve the inference speed of the network.

[0110] The extracted features will be spliced before entering the classification network. Common sequential splicing is equivalent to two groups of features acting independently on the network, and the corresponding similar features are far apart and difficult to aggregate. Through the average interpolation fusion strategy, the relevance of similar features is actively used to reduce the distance of similar features and increase the distance of distinguishing features, thereby reducing the classification difficulty of features and improving the classification efficiency of the network.

[0111] The specific algorithm pseudo code is as follows:

[0112] Feature 1: X = [x0, x1,..., x 255 ];

[0113] Feature 2: Y = [y0, y1,..., y 255 ];

[0114] Define feature: Z = [z0, z1,..., z 511 ]

[0115] For (N = 0; N < 256; N++)

[0116] {Z 2N+1 = X N ; Z N = Y N ;};end for;

[0117] return Z;

[0118] The process corresponding to the pseudo code is shown in Figure 7 . Experiments show that compared with the original Pointnet++, the introduction of the structure improves the training efficiency, as shown in Figure 12 .

[0119] 2、Reconstruction SA module

[0120] The improved SA module (as shown in Figure 8 ) and the original Pointnet++ SA module (as shown in Figure 9The Pointnet module in the original SA module is replaced by an MLP and a MAX, as shown in FIG. 6. Because the original Pointnet++ is used for scene segmentation and semantic analysis of point clouds, the two scenarios emphasize the transformation invariance of the point cloud, and the order of the point cloud does not need to be emphasized in face recognition. At the same time, the Pointnet part contains a coordinate transformation structure, and the calculation amount is very large, with a parameter amount of 4.6M, which seriously slows down the calculation speed of the network. The MLP and the MAX of the present application only have a parameter amount of 1.8M, and the extraction effect is equivalent. Therefore, the Pointnet module is replaced by an MLP module and a MAX module in this chapter, wherein the MLP module is used to realize dimension conversion, and the MAX module is used to realize dimension reduction. Experiments show that, compared with the original Pointnet++, the introduction of the new SA module improves the training efficiency (such as Figure 12 as shown in Table 4.

[0121] Table 4 Comparison of test Rank-1 recognition rates of different generation methods

[0122]

[0123] III) End-to-end network training module

[0124] The end-to-end network training module is used to train the IMPNet. In the training of the IMPNet, an angular loss function is used to represent the distance between the predicted category and the label category. The smaller the value of the angular loss function, the better the classification effect. The expression form of the angular loss function is as shown in the following formula (9):

[0125]

[0126] In formula (9), N represents the number of samples participating in the calculation of the loss; θ i represents the weight vector W of the end-to-end 3D face recognition network j and the fused face feature y obtained by the end-to-end 3D face recognition network i is the included angle of (1, C1+C2); (θ i ,y i +m) represents the included angle of θ i and y i +m, and y i represents the fused face feature y obtained by the end-to-end 3D face recognition network i =(1, C1+C2), m represents an angle penalty value, which is used to increase the distance between classes and reduce the distance within the class; s represents a deformation vector; and z represents a vector obtained after L2 regularization of the fused face feature

[0127] Compared with the representative algorithms on FRGCv2 and Bosphorus, as shown in Table 4 and Table 5, the VR (verification accuracy) of the present application is the highest, the RR1 (Rank-1 recognition rate) is the highest, and the average leads by 6.61% (FRGCv2) and 5.17% (Bosphorus), which leads by 5.1% (GBNet-FRGCv2) and 2.9% (GBNet-Bosphorus) than the best algorithm. GBNet and KPconv are the algorithms with the highest accuracy at present, both of which use the algorithm of Finetune, but in actual use, the face is dynamically changing, and the effect of Finetune is not good. The present application does not use Finetune, but achieves an accuracy of 98.9%, which shows the superiority of the present application. Figure 10 、 Figure 11 The ROC curve of IMPNet in the middle is located at the uppermost, and the corresponding AUC value is the largest (as shown in Table 5 and Table 6), which is 98.7 (FRGCv2) and 97.2 (Bosphorus) respectively, indicating that the classification performance of the network is the best. The present application not only has high accuracy, but also uses end-to-end design of the network without complex preprocessing process; IMPNet with GDAS does not need a large amount of real training data, and these advantages make the present application have strong competitiveness.

[0128] Table 5 Test and verification results on FRGCv2 dataset

[0129]

[0130] Note: F in the table represents Finetune, and 2.5D represents the combination of 2D and 3D

[0131] Table 6 Test and verification results on Bosphorus dataset

[0132] .

Claims

1. An end-to-end 3D face recognition system based on data augmentation structure, characterized in that, The method comprises a training set generation module, an end-to-end network training module and an end-to-end 3D face recognition network. The training set generation module generates a training set for training the IMPNet based on an existing reference data set, wherein the training set generation module further comprises a generator GDAS, after the reference data set is divided into shape faces, expression faces and texture faces, the generator GDAS respectively establishes a face shape GSSM model, a face expression GSSM model and a face texture GSSM model, after the generator GDAS obtains a statistical shape model from m 3D faces, the statistical shape model is subjected to Gaussian regression and K-L expansion to obtain the face shape GSSM model, the face expression GSSM model or the face texture GSSM model; the generator GDAS linearly combines the face shape GSSM model, the face expression GSSM model and the face texture GSSM model to obtain a final face model F new ; the training set generation module generates a training set for training the IMPNet based on the face model F new ​ The end-to-end network training module trains the end-to-end 3D face recognition network by using the training set, the end-to-end 3D face recognition network takes a point cloud as input and outputs a 1*512 feature vector, the feature vector is subjected to a dropout layer and a full connection layer to obtain a final classification result, the end-to-end 3D face recognition network comprises an SA module 1, an SA module 2 and a feature fusion module, the SA module 1 performs feature fusion and feature extraction on the input point cloud based on MLP operation and MAX operation to generate an output, the output of the SA module 1 is input into the SA module 2, and a 1*256 feature vector 1 is obtained by performing MAX operation on the output of the SA module 1 again, the SA module 2 performs feature fusion and feature extraction on the input data based on MLP operation and MAX operation to generate an output, and a 1*256 feature vector 2 is obtained by performing MAX operation on the output of the SA module 2 again, and the feature fusion module fuses the 1*256 feature vector 1 and the 1*256 feature vector 2 into a 1*512 feature vector.

2. The end-to-end 3D face recognition system based on data augmentation structure of claim 1, wherein, The face shape GSSM model is denoted as s(x), the face expression GSSM model is denoted as e(x), and the face texture GSSM model is denoted as t(x), and the following equation is obtained: where μ s (x), μ e (x), μ t (x) is the mean function; d s , d e , d t are the eigenvalues corresponding to the basis functions after K-L expansion; U s , U e , U t are the basis function sets, respectively responsible for the generation of face shape, expression and texture; α, β, γ are parameter vectors.

3. The end-to-end 3D face recognition system based on data augmentation structure of claim 2, wherein, The generator GDAS linearly combines the face shape GSSM model, the face expression GSSM model and the face texture GSSM model by using the following equation: In the equation, μ1, μ2 and μ3 represent deformation coefficients, and the three parameters are independent of each other.

4. The end-to-end 3D face recognition system based on data augmentation structure of claim 1, wherein, The feature vector 1 or the feature vector 2 is obtained by the SA module 1 or the SA module 2, and the following steps are included: Step 101, performing farthest point sampling on the input of the SA module 1 or the SA module 2 to obtain N1 points; Step 102, grouping: taking N1 points as the center of a circle, the radius of the circle is R, and K points are taken in each obtained circle, thereby obtaining N1 grouped point sets Step 103, N1 sets of grouping points are fed into the MLP, the N1 sets of grouping points are subjected to an MLP dimension conversion operation; Step 104, again on the N1 set of points MAX operation is performed on each of the SA module one and the SA module two to obtain the output of the SA module one and the SA module two respectively Step 105, to A 1x256 eigenvector is obtained by performing a MAX operation again.

5. The end-to-end 3D face recognition system based on data augmentation structure of claim 1, wherein, The feature fusion module fuses the 1*256 feature vector 1 and the 1*256 feature vector 2 into a 1*512 feature vector by using multi-scale feature average interpolation fusion, and the following steps are included: Step 201, a 1 x 256 feature vector one is expressed as X = [x0, x1,..., x255]T. 255 ] The 1 x 256 feature vector two is denoted as Y = [y0, y1,..., y255]T. 255 ] The eigenvector of a 1×512 matrix is ​​defined as Z = [z0, z1, ..., z]. 511 ]; Step 202, initializing n = 0; Step 203, if n < 256, then entering step 204; otherwise, entering step 206; Step 204, z 2n+1 = x n ; z n = y n ; Step 205, n = n + 1, and returning to step 203; Step 206, obtaining a final 1*512 feature vector Z.

6. The end-to-end 3D face recognition system based on data augmentation structure of claim 1, wherein, When training the end-to-end 3D face recognition network, the end-to-end network training module uses an angular loss function to represent the distance between a predicted category and a label category, and the smaller the value of the angular loss function, the better the classification effect.

7. The end-to-end 3D face recognition system based on data augmentation structure of claim 6, wherein, The expression form of the angular loss function is shown in the following equation: wherein: N represents the number of samples participating in the calculation of loss; θ i represents the weight vector of the end-to-end 3D face recognition network and the included angle between the fused face features obtained by the end-to-end 3D face recognition network; (θ i , y i +m) represents the included angle between θ i and y i +m, y i represents the fused face features obtained by the end-to-end 3D face recognition network, m represents an angle penalty value, used to increase the inter-class distance and reduce the intra-class distance; s represents a deformation vector; and z represents a vector obtained after L2 regularization of the fused face features.

Images