A 3D face recognition system with dual sampling mechanism and multi-branch parallel training

Abstract

The present invention discloses a 3D face recognition system with a dual sampling mechanism and multi-branch parallel training, characterized in that it includes: an end-to-end backbone network PointMLP, further including a point cloud downsampling module RSFPS, a normal estimation module, an Embeding layer, N residual MLP layers, an FC BN RELU layer, a Dropout layer and an FC layer; and an end-to-end network training module. The present invention proposes a dual sampling mechanism, which not only adds RSFPS in the initial stage of Res-GLNet, but also adds an adaptive downsampling branch micro-network ADP-CLA during the sampling process when training Res-GLNet, continuously supervising Res-GLNet so that Res-GLNet extracts features that are more conducive to face classification. The two sampling mechanisms work together to form a self-supervisory mechanism. Compared with existing algorithms, the present invention has significant advantages in the network's Rank-1 recognition accuracy, test speed and training speed.

Description

Technical Field

[0001] The present invention relates to a 3D face recognition system, and belongs to the technical field of artificial intelligence-3D face recognition. Background Art

[0002] Facial recognition technology, as an effective contactless authentication method, has widespread applications in healthcare, finance, industrial production, and other fields. Currently, 3D facial recognition technology is becoming increasingly popular and a hot topic in research and application. Compared to traditional 2D facial recognition, 3D facial recognition incorporates one additional dimension of depth information, making it more robust to changes in lighting, facial expression, posture, and liveness detection. However, current 3D facial recognition algorithms suffer from low sampling accuracy and inefficient training. Summary of the Invention

[0003] The technical problem to be solved by the present invention is that the current 3D face recognition algorithm has low sampling accuracy and low training efficiency.

[0004] In order to solve the above-mentioned computational problem, the technical solution of the present invention is to provide a 3D face recognition system with a dual sampling mechanism and multi-branch parallel training, which is characterized by comprising:

[0005] The end-to-end backbone network PointMLP further includes a point cloud downsampling module RSFPS, a normal estimation module for performing normal estimation on the output of the point cloud downsampling module RSFPS, an Embeding layer for processing the features output by the normal estimation module, N residual MLP layers for sequentially processing the outputs of the Embeding layer, an FC BN RELU layer for processing the features output by the N residual MLP layers, a Dropout layer for processing the FC BN RELU layer, and an FC layer for processing the features output by the Dropout layer and obtaining the final classification result, N ≥ 2, where:

[0006] Point cloud downsampling module RSFPS input point set Output point set Then we have:

[0007]

[0008]

[0009] Where, Γ j ∈S represents the point selected by the point cloud downsampling module RSFPS, Γ i ∈Γ represents the points screened during the iteration process, d(·) represents the Euclidean distance, e j is Γ j The eigenvalue of , p is the weight factor, λ iis the weight, R is the effective radius;

[0010] The end-to-end network training module is used to perform parallel training on the adaptive downsampling branch micro-network ADP-CLA and the end-to-end backbone network PointMLP. The adaptive downsampling branch micro-network ADP-CLA and the end-to-end backbone network PointMLP constitute the total network Res-GLNet, where there are N adaptive downsampling branch micro-networks ADP-CLA. The input of the nth adaptive downsampling branch micro-network ADP-CLA is the input of the nth residual MLP layer. The point cloud coordinates of the nth adaptive downsampling branch micro-network ADP-CLA are input to the adaptive downsampling branch micro-network ADP-CLA to obtain the coordinates after dimension conversion. The values ​​of all dimensions of each point are summed and averaged, and the points above the average are retained and fed back to the nth residual MLP layer as additional input to the nth residual MLP layer. At the same time, the adaptive downsampling branch micro-network ADP-CLA obtains the corresponding prediction value based on the retained points. During the training process, the loss of the adaptive downsampling branch micro-network ADP-CLA is implanted into the total loss of the overall network Res-GLNet. The loss of the adaptive downsampling branch micro-network ADP-CLA continuously reversely updates the network parameters of the overall network Res-GLNet, while some parameters of the backbone network PointMLP and the adaptive downsampling branch micro-network ADP-CLA are shared. The loss gradient of the overall network Res-GLNet is the result of the joint action of the adaptive downsampling branch micro-network ADP-CLA and the backbone network PointMLP. The predicted output of the adaptive downsampling branch micro-network ADP-CLA is set to be the same as the expected output of the overall network Res-GLNet.

[0011] Preferably, the value of the effective radius R adopts a random swing strategy and is randomly selected in the interval (60, 80).

[0012] Preferably, the value of the weight factor p adopts a random swing strategy and is randomly selected in the interval (-0.15, 0.15).

[0013] Preferably, the loss of the total network Res-GLNet is training The losses Loss1, Loss2, ..., Loss of N adaptive downsampling branch micro-network ADP-CLA are N and end-to-end backbone network PointMLP loss Loss a The sum is: Loss training =Loss a +Loss1+Loss2+...+Loss N .

[0014] This paper proposes a dual sampling mechanism. It not only incorporates RSFPS into the initial stages of Res-GLNet, but also incorporates an adaptive downsampling branch micronetwork (ADP-CLA) during the sampling process during Res-GLNet training. This continuously supervises Res-GLNet, enabling it to extract features that are more conducive to face classification. These two sampling mechanisms work together to form a self-supervisory mechanism. Compared to existing algorithms, this paper demonstrates significant advantages in network Rank-1 recognition accuracy, test speed, and training speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a simplified diagram of the Res-GLNet structure;

[0016] Figure 2 The random swing farthest point sampling (RSFPS) analysis is illustrated;

[0017] Figure 3 The composition of ADP-CLA is shown schematically;

[0018] Figure 4 Schematic diagram of ADP-CLA feature screening. DETAILED DESCRIPTION

[0019] The present invention will be further described below in conjunction with specific examples. It should be understood that these examples are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalents fall within the scope limited by the appended claims of the present invention.

[0020] This embodiment discloses a 3D face recognition system with a dual sampling mechanism and multi-branch parallel training, including an end-to-end network Res-GLNet and an end-to-end network training module.

[0021] The end-to-end network Res-GLNet designed by the present invention takes a point cloud as input and outputs a feature vector. The feature vector output by the end-to-end network Res-GLNet is then classified through a fully connected network.

[0022] The end-to-end network Res-GLNet can be expressed as the following formula:

[0023] f=Res-GLnet(Γ) (1)

[0024] In formula (1): is the unordered input point cloud, N0 is the number of points, f∈R 256 is the output feature.

[0025] In this embodiment, Res-GLNet follows the end-to-end design of PointMLP, and its network structure is as follows Figure 1 As shown in the figure: the input of Res-GLNet is the original point cloud N0×3. Res-GLNet estimates the normal of the original point cloud to obtain N0×7 point cloud features, and then obtains N0×32 Embeding features after passing through the Embeding layer. The N0×32 Embeding features are then processed by multiple layers of residual MLP layers (including geometric affine modules, local feature modules and global feature extraction modules), enter the FC BNRELU layer and Dropout (0.5), and finally enter the FC layer to obtain the final classification result.

[0026] Res-GLNet is an improvement on the existing end-to-end PointMLP network. PointMLP's poor performance in face recognition tasks is primarily due to the network's lack of a downsampling unit and the network's ability to extract highly global features, while lacking the ability to integrate local information. This is detrimental to face recognition. Therefore, the present invention incorporates a downsampling mechanism into the PointMLP network. RSFPS is incorporated into the network's initial stages.

[0027] Currently, point cloud downsampling is mainly divided into random downsampling and farthest point sampling FPS (Farthest Point Sampling). Compared with random downsampling, FPS sampling is uniform and has better overall coverage. It can more comprehensively obtain the edge structure information and overall information of the sampled target. The specific process of FPS is as follows: The input point set is Output point set The formula for FPS iteration is as follows:

[0028]

[0029]

[0030] In formula (2) and (3), Γ j ∈S represents the point selected by FPS, Γ i ∈Γ represents the points screened during the iteration process, and d(·) represents the Euclidean distance. The sampling results of FPS are as follows Figure 2As shown in (a) in the figure. The capture process of FPS gives higher priority to protruding corner edge points, and the corner edge points in 3D faces are mostly concentrated on the nose, eyes and facial edges of the face, which makes FPS have certain advantages in facial feature extraction. However, in actual use, the facial point cloud is captured from several directions, so the collection of edge feature information is very unstable, and the recognition rate is low in face recognition tests. At the same time, FPS adopts a ball query clustering strategy to concentrate the sampling points on one side of the center, which makes the sampling very sensitive to changes in expression. In response to the above two problems, the present invention proposes RSFPS, as shown in the following formula:

[0031]

[0032]

[0033] In formula (4) and formula (5): d(Γ j ,x NT ) represents Γ j and x NT The Euclidean distance between NT is the coordinate of the nose tip point; e j is Γ j The eigenvalue of the point Γ j The smoothness near the edge; p is the weight factor; R is the effective radius, when d(Γ j ,x NT )>R, λ i will be set to 0, the corresponding Γ j will not be selected, such as Figure 2 As shown in (b) in .

[0034] In the first row of formula (5), R reflects the effective area of ​​the face. When R is set to (60, 80), it can ensure that most faces do not contain edge areas. The points output by RSFPS are mostly selected from the central area with rich facial features. Since the value of R is a range, the present invention adopts a random swing strategy to randomly select a value in the interval. The second row of formula (5) eigenvalue e j Reflection point Γ j When the value of p decreases, RSFPS tends to select points on smooth surfaces; when the value of p increases, RSFPS tends to select corner points. Figure 2 (c) in the figure. However, in practical applications, different acquisition devices can produce different degrees of smoothness in the point cloud, making the value of p difficult to determine specifically. Therefore, the present invention also adopts a random oscillation strategy for the value of p, setting P to a random value between (-0.15, 0.15). Experiments show that when the proposed random oscillation strategy is added, the network achieves better results by adding a small amount of randomness to the sampling.

[0035] The end-to-end network training module is used to train Res-GLNet. During training, the adaptive downsampling branch micro-network ADP-CLA is added to the sampling process. The adaptive downsampling branch micro-network ADP-CLA continuously supervises Res-GLNet, allowing Res-GLNet to extract features that are more conducive to face classification.

[0036] In the training process of Res-GLNet, actively guiding Res-GLNet to select points that better reflect the characteristics of the object is extremely important for improving the training efficiency of Res-GLNet. The present invention adds an adaptive downsampling branch micro-network ADP-CLA to the local feature module of PointMLP, such as Figure 3 As shown in Figure 2, ADP-CLA guides Res-GLNet to extract more discriminative features that are more conducive to classification during training. ADP-CLA takes point clouds of different dimensions during training as input and outputs features in the category dimension. ADP-CLA is a pure MLP design, containing: 2 MLP layers, 2 BN layers, and 2 ReLU layers. The entire ADP-CLA can be expressed as:

[0037] F`=ADP-CLA(F) (6)

[0038] In formula (6), F' represents the output feature (feature information with category dimension), and F represents the input feature. In this embodiment, Res-GLNet has a total of 4 ADP-CLAs, and the input of each ADP-CLA is a point cloud of different dimensions.

[0039] ADP-CLA follows the following two principles: 1. Point cloud coordinates are transformed through ADP-CLA. The values ​​of all dimensions for each point are summed. Larger values ​​indicate greater feature information. The sums are averaged, and points above the average are retained. These retained points are then fed into the residual MLP layer.

[0040] 2. Use the following method to save the points: Figure 4 The method shown above performs MAX operation twice, and the last MAX operation is the corresponding predicted value. (used to calculate Loss).

[0041] The adaptive adjustment of the dual sampling mechanism relies on continuous feature learning. During Res-GLNet training, ADP-CLA and the backbone network PointMLP are trained in parallel, with the ADP-CLA loss incorporated into the overall loss of the backbone network PointMLP. During Res-GLNet training, the ADP-CLA loss continuously updates the network parameters in reverse order. The backbone network PointMLP and ADP-CLA share some parameters, and the loss gradient of Res-GLNet is the result of the combined effect of ADP-CLA and the backbone network PointMLP. ADP-CLA sets the predicted output to be the same as the expected output of Res-GLNet, which causes the shallow features of the network to iterate in a direction that is more conducive to deep network classification, thereby optimizing network parameters, reducing classification difficulty, and enhancing classification accuracy.

[0042] Assume the output of ADP-CLA is The cross entropy loss function is used for both the branch network and the backbone network:

[0043]

[0044] in, is the predicted value of the i-th ADP-CLA, and y is the actual value. The loss of the i-th ADP-CLA is:

[0045]

[0046] The loss of Res-GLNet consists of two parts: the loss of the branch network i and backbone network loss Loss a The sum is:

[0047] Loss training =Loss a +Loss1+Loss2+...+Loss n (9)

[0048] Res-GLNet is compared with mainstream 3D face recognition algorithms on two datasets, Bosphorus and FRGCv2, to comprehensively evaluate the performance of the proposed network. Using the original code from the relevant paper, the same datasets (Bosphorus and FRGCv2) and data partitioning criteria are used. This paper compares the following aspects: 1. Rank-1 recognition accuracy of the network; 2. Testing speed; and 3. Training speed. Below, we discuss the comparative results of FRGCv2 and Bosphorus, using the datasets used as the partitioning criteria.

[0049] 1. FRGCv2

[0050] Table 1 shows the comparison results of the proposed network, Res-GLNet, and more advanced algorithms on the FRGCv2 dataset. As can be seen from the table below, current deep learning-based 3D face recognition algorithms have achieved relatively good recognition performance, but Res-GLNet achieved a higher recognition accuracy of 98.9%, an average of 2.6% higher than the other nine algorithms. The proposed algorithm also had the fastest test speed of 163, and the optimal training speed of 112. In summary, thanks to the dual-sampling mechanism and parallel training strategy proposed in this paper, the proposed algorithm achieved the best recognition performance on this dataset. Furthermore, Res-GLNet's backbone network utilizes an end-to-end PointMLP design, eliminating the need for complex preprocessing operations and improving the network's testing and training speeds. Furthermore, the end-to-end network is highly user-friendly and easier to deploy on embedded devices. Therefore, the end-to-end network, Res-GLNet, based on dual-sampling parallel optimization training, achieved the best performance on this public dataset.

[0051] Table 1 FRGCv2 test results

[0052]

[0053] 2. Bosphorus

[0054] Res-GLNet was compared with mainstream algorithms on the Bosphorus dataset, with the results shown in Table 2. It can be seen that Res-GLNet achieved the best results on this dataset in terms of Rank-1 recognition accuracy, test speed, and training speed, with scores of 98.8%, 163%, and 112, respectively. This comparison further validates the effectiveness of the proposed algorithm.

[0055] Table 2 Bosphorus test results

[0056]

Claims

1. A 3D face recognition system with a dual sampling mechanism and multi-branch parallel training, characterized in that: include: The end-to-end backbone network PointMLP further includes a point cloud downsampling module RSFPS, a normal estimation module for performing normal estimation on the output of the point cloud downsampling module RSFPS, an Embeding layer for processing the features output by the normal estimation module, N residual MLP layers for sequentially processing the outputs of the Embeding layer, an FC BN RELU layer for processing the features output by the N residual MLP layers, a Dropout layer for processing the FC BN RELU layer, and an FC layer for processing the features output by the Dropout layer and obtaining the final classification result, N ≥ 2, where: Point cloud downsampling module RSFPS input point set Output point set Then we have: Where, Γ j ∈S represents the point selected by the point cloud downsampling module RSFPS, Γ i ∈Γ represents the points screened during the iteration process, d(·) represents the Euclidean distance, e j is Γ j The eigenvalue of , p is the weight factor, λ i is the weight, R is the effective radius; The end-to-end network training module is used to perform parallel training on the adaptive downsampling branch micro-network ADP-CLA and the end-to-end backbone network PointMLP. The adaptive downsampling branch micro-network ADP-CLA and the end-to-end backbone network PointMLP constitute the total network Res-GLNet, where there are N adaptive downsampling branch micro-networks ADP-CLA. The input of the nth adaptive downsampling branch micro-network ADP-CLA is the input of the nth residual MLP layer. The point cloud coordinates of the nth adaptive downsampling branch micro-network ADP-CLA are input to the adaptive downsampling branch micro-network ADP-CLA to obtain the coordinates after dimension conversion. The values ​​of all dimensions of each point are summed and averaged, and the points above the average are retained and fed back to the nth residual MLP layer as additional input to the nth residual MLP layer. At the same time, the adaptive downsampling branch micro-network ADP-CLA obtains the corresponding prediction value based on the retained points. During the training process, the loss of the adaptive downsampling branch micro-network ADP-CLA is implanted into the total loss of the overall network Res-GLNet. The loss of the adaptive downsampling branch micro-network ADP-CLA continuously reversely updates the network parameters of the overall network Res-GLNet, while some parameters of the backbone network PointMLP and the adaptive downsampling branch micro-network ADP-CLA are shared. The loss gradient of the overall network Res-GLNet is the result of the joint action of the adaptive downsampling branch micro-network ADP-CLA and the backbone network PointMLP. The predicted output of the adaptive downsampling branch micro-network ADP-CLA is set to be the same as the expected output of the overall network Res-GLNet.

2. A 3D face recognition system with a dual sampling mechanism and multi-branch parallel training as claimed in claim 1, characterized in that: The value of the effective radius R adopts a random swing strategy and is randomly selected in the interval (60, 80).

3. A 3D face recognition system with a dual sampling mechanism and multi-branch parallel training as claimed in claim 1, characterized in that: The value of the weight factor p adopts a random swing strategy and is randomly selected in the interval (-0.15, 0.15).

4. A 3D face recognition system with a dual sampling mechanism and multi-branch parallel training as claimed in claim 1, characterized in that: The loss of the total network Res-GLNet training The losses Loss1, Loss2, ..., Loss of N adaptive downsampling branch micro-network ADP-CLA are N and end-to-end backbone network PointMLP loss Loss a The sum is: Loss training =Loss a +Loss1+Loss2+...+Loss N .

Images