A face recognition method based on asymmetric features and Riemannian manifold structure
By adopting the GoogleLeNet and ResNet network structure in facial recognition technology, combining the Inception module and SPDNet network, multi-scale feature extraction and manifold data conversion are solved, and the problem of low facial recognition efficiency and accuracy in the existing technology is realized. More efficient and accurate facial recognition effects.
Patent Information
- Application Number
- CN202111433120.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-11-29
AI Technical Summary
The existing facial recognition technology has shortcomings in recognition efficiency and accuracy. The traditional manual feature extraction method is inefficient. Although the deep learning method improves the speed, the accuracy is low. The network structure is complex and time-consuming, and it is impossible to effectively explore the inner world of the face. Logical structure.
Using a convolutional neural network based on GoogleLeNet and ResNet network structure, a multi-channel feature map is constructed through a 3×3 convolution kernel, a ReLU activation function is introduced, and an Inception module with different convolution kernel sizes is constructed, multi-scale features are obtained, and covariance is used. The matrix is regularized and converted into manifold data, and Riemann calculation is performed in combination with the SPDNet network to generate discriminant low-dimensional features.
It improves the accuracy and speed of face recognition, reduces the data dimension, enables the face image data to be processed in a smooth space, optimizes feature extraction and network structure, and significantly improves the recognition effect and efficiency.
Smart Images

Figure CN113989907B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to image processing technology, in particular to a face recognition method based on asymmetric features and Riemann manifold structure. Background Art
[0002] With the advent of Internet technology and the 5G era, the new generation of scientific and technological revolution is developing rapidly. In the natural and social sciences, such as transportation, finance, monitoring systems, social management, and people's livelihood services, a large amount of data is being generated exponentially all the time. Not only is the amount of collected data increasing, but the dimensions of the data are also increasing. At the same time, there are internal relationships and logical structures in these data that are difficult to be intuitively discovered by human vision, which brings great difficulties to the use of data. Therefore, it is our urgent task to study better algorithms to process and analyze the internal relationships and logical structures of data.
[0003] Before the emergence of deep learning, face recognition feature extraction was done using traditional manual feature extraction. This method not only has low recognition efficiency, but also has a complex network structure. It is not an end-to-end network structure, and each part needs to be trained separately, which is also time-consuming. Based on the emergence of deep learning methods, face recognition technology uses network structures to learn and acquire features for recognition, such as R-CNN, YOLO, VGG, GoogLeNet and other network structures. Among them, R-CNN is a two-step detection method. First, 2,000 candidate frames are generated, and then these candidate frames are processed and input into the network for convolution and pooling operations. Finally, classification detection is performed through the fully connected layer. This network structure needs to be trained in two steps, and because all 2,000 candidate frames need to be detected, it takes a long time. Slow training is not conducive to improving the rate of face recognition; YOLO It is an end-to-end network detection structure that can train the entire network structure and perform face recognition in a short time, but its recognition accuracy is low; the VGG network structure has not only achieved good improvements in speed and accuracy, but the number of parameters in the first fully connected layer of the network structure after the convolution pooling operation has exploded, greatly increasing the demand for hardware resources and the network structure is too bloated; GoogLeNet uses multi-scale parallel feature extraction, which significantly reduces the number of parameters in the network structure, but it is located in the Euclidean space and cannot explore the inherent logic of the face. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a face recognition method based on asymmetric features and Riemann manifold structure, so as to improve the accuracy and speed of face recognition.
[0005] Technical solution: The face recognition method based on asymmetric features and Riemann manifold structure described in the present invention comprises the following steps:
[0006] S1, randomly divide the labeled face image dataset into training set and validation set;
[0007] S2, using a convolutional neural network based on the improved GoogLeNet and ResNet network structures;
[0008] S2.1, construct a layer of multi-channel feature map through a 3×3 convolution kernel;
[0009] S2.2. After outputting the multi-channel feature map, a ReLU activation function is added to introduce a nonlinear structure.
[0010] S3, using multiple convolution kernels of different sizes to stack and construct different Inception modules;
[0011] S3.1, using multiple convolution kernels of different sizes to stack and construct different Inception modules;
[0012] S3.2. In each Inception module, the feature map obtained by each convolution kernel is stacked according to its depth. By training N layers of features of different scales, all hidden layers are trained to obtain excellent facial image features to obtain multi-scale features of facial images.
[0013] S4. Perform supervised fine-tuning on the parameters of the network framework for obtaining the multi-scale feature map of the face image; the fine-tuning adopts the stochastic gradient descent method to obtain excellent multi-scale features of the face image that can improve the face recognition effect by minimizing the loss function of the output type and the face image feature label error.
[0014] S5. The multi-scale feature map of the face image obtained by the convolutional neural network structure is flattened for each feature map and combined into a matrix form according to the rules, the matrix is transformed to obtain its covariance matrix, and then the covariance matrix is regularized so that the covariance matrix has a symmetric positive definite property, thereby converting the multi-scale feature map data of the face image into manifold data;
[0015] S5.1. Obtain multi-scale feature maps of facial images through a convolutional neural network structure, and form vectors by flattening each feature map;
[0016] S5.2, combining each column vector into a matrix form, and performing covariance operation on the matrix to obtain a covariance matrix;
[0017] S5.3, through certain regularization conditions, such as: C +=C+λtrace(C)A, where λ is the regularization parameter, generally set to 0.0001, trace(C) is the trace of C, and A is the unit matrix, which can make the covariance matrix have the characteristics of symmetric positive definite, making the covariance matrix have the characteristics of symmetric positive definite;
[0018] S5.4. Convert the multi-scale feature map data of the face image into manifold data.
[0019] S6. The converted manifold data SPD matrix is passed through the SPDNet network structure. After the BiMap layer, the ReEig layer and the LogEig layer, a more discriminative SPD matrix with significantly reduced data dimension can be generated. At the same time, it can also be applied to the subsequent other fully connected layers and softmax layers for face recognition output;
[0020] S6.1, converting the converted manifold data SPD matrix into a low-dimensional SPD matrix through a bilinear mapping B of a BiMap layer;
[0021] S6.2, the ReEig layer modifies the SPD matrix using a nonlinear function R, which corrects the smaller eigenvalues of the SPD matrix by adjusting them;
[0022] S6.3, the LogEig layer performs Riemann calculation on the obtained SPD matrix so that it can be applied to subsequent fully connected layers and softmax layers for face recognition output.
[0023] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned face recognition method based on asymmetric features and Riemann manifold structure.
[0024] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned face recognition method based on asymmetric features and Riemann manifold structure is implemented.
[0025] Beneficial effects: Compared with the prior art, the present invention has the following advantages:
[0026] 1. Compared with the existing mainstream solutions, by stacking convolution kernels of different sizes and adopting asymmetric feature extraction, it has the advantages of widening the network structure and optimizing feature extraction;
[0027] 2. Similar to the GoogLeNet network structure, the face recognition structure of the convolutional network structure technology described in the present invention is also a modular idea, which is easy to retain the shallow features of the face image as a whole and can also mine deep semantic features;
[0028] 3. Compared with the convolutional neural network, the present invention incorporates the Riemann manifold structure of the symmetric positive definite matrix of manifold learning into the convolutional neural network, further reducing the data dimension so that the face image data can be located in a smooth space;
[0029] 4. In order to integrate Riemannian manifold learning into convolutional neural networks, the present invention adopts a method for calculating its covariance matrix and adds certain regularization terms so that the obtained facial image feature manifold representation has the properties of a symmetric positive definite matrix. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is a flow chart of the steps of the present invention;
[0031] Figure 2 This is a schematic diagram of the feature manifold representation of a face image;
[0032] Figure 3 It is a multi-scale network structure diagram;
[0033] Figure 4 Network structure diagram for asymmetric feature extraction;
[0034] Figure 5 This is a flowchart for the research on multi-scale feature extraction of face images based on convolutional neural networks. DETAILED DESCRIPTION
[0035] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0036] like Figure 1 As shown, a face recognition method based on asymmetric features and Riemann manifold structure includes the following steps:
[0037] S1, randomly divide the labeled face image dataset into training set and validation set;
[0038] S2, such as Figure 3 As shown, a convolutional neural network based on the improved GoogLeNet and ResNet network structures is used;
[0039] S2.1, construct a layer of multi-channel feature map through a 3×3 convolution kernel;
[0040] S2.2. After outputting the multi-channel feature map, a ReLU activation function is added to introduce a nonlinear structure.
[0041] S3, such as Figure 4 As shown in the figure, different Inception modules are constructed by stacking multiple convolution kernels of different sizes;
[0042] S3.1, using multiple convolution kernels of different sizes to stack and construct different Inception modules;
[0043] S3.2. In each Inception module, the feature map obtained by each convolution kernel is stacked according to its depth. By training N layers of features of different scales, all hidden layers are trained to obtain excellent facial image features to obtain multi-scale features of facial images.
[0044] S4, such as Figure 5 As shown, the network framework parameters for obtaining the multi-scale feature map of the face image are fine-tuned in a supervised manner; the fine-tuning adopts the stochastic gradient descent method to obtain excellent multi-scale features of the face image that can improve the face recognition effect by minimizing the loss function of the output type and the face image feature label error.
[0045] S5, such as Figure 2 As shown in the figure, the multi-scale feature map of the face image obtained by the convolutional neural network structure is flattened for each feature map and combined into a matrix form according to the rules. The matrix is transformed to obtain its covariance matrix, and then the covariance matrix is regularized to have the characteristic of symmetric positive definite, so that the multi-scale feature map data of the face image can be converted into manifold data.
[0046] S5.1. Obtain multi-scale feature maps of facial images through a convolutional neural network structure, and form vectors by flattening each feature map;
[0047] S5.2, combining each column vector into a matrix form, and performing covariance operation on the matrix to obtain a covariance matrix;
[0048] S5.3, through certain regularization conditions, such as: C + =C+λtrace(C)A, where λ is the regularization parameter, generally set to 0.0001, trace(C) is the trace of C, and A is the unit matrix, which can make the covariance matrix have the characteristics of symmetric positive definite, making the covariance matrix have the characteristics of symmetric positive definite;
[0049] S5.4. Convert the multi-scale feature map data of the face image into manifold data.
[0050] S6. The converted manifold data SPD matrix is passed through the SPDNet network structure. After the BiMap layer, the ReEig layer and the LogEig layer, a more discriminative SPD matrix with significantly reduced data dimension can be generated. At the same time, it can also be applied to the subsequent other fully connected layers and softmax layers for face recognition output;
[0051] S6.1, converting the converted manifold data SPD matrix into a low-dimensional SPD matrix through a bilinear mapping B of a BiMap layer;
[0052] S6.2, the ReEig layer modifies the SPD matrix using a nonlinear function R, which corrects the smaller eigenvalues of the SPD matrix by adjusting them;
[0053] S6.3, the LogEig layer performs Riemann calculation on the obtained SPD matrix so that it can be applied to subsequent fully connected layers and softmax layers for face recognition output.
[0054] This embodiment uses an open source dataset on the Internet, and randomly divides it into eleven equal parts, takes one of them as the final test, and uses the remaining nine for training and one for cross-validation, and adds the validation loss to the network structure, so that the face recognition effect can be cross-validated; secondly, a 128-channel face image feature map is constructed through a 3×3 convolution kernel, and a ReLU activation function is connected after the output of the 128-channel feature map to introduce a nonlinear structure, and then multiple convolution kernel sizes of different sizes are used, such as 1×1, 3×3, 1×3, 3×1, 5×5, 1×5 and 5×1, which are stacked to construct different Ince ption module, through different Inception module combinations to obtain excellent multi-scale features of face images, where in each Inception module, the feature map obtained by each convolution kernel will be stacked according to its depth, by training N layers of features of different scales, all hidden layers are trained to obtain excellent face image features to obtain multi-scale features of face images, and at the same time, the random gradient descent method is adopted to update the weights of the convolution kernel, and the loss function of the output type and face image feature label error is minimized to obtain excellent multi-scale features of face images that can improve face recognition effect.
[0055] The multi-scale feature map of the face image is obtained through the convolutional neural network structure, and each feature map is flattened to form a vector, and then each column of the vector is combined into a matrix form, and the covariance operation is performed on the matrix to obtain the covariance matrix, that is, the covariance of the multi-scale feature map of the face image is calculated. The calculation method is: in is the average vector of the face image feature set F, which can be expressed as: Then through certain regularization conditions, the method is: C + =C+λtrace(C)A, where λ is the regularization parameter, generally set to 0.0001, trace(C) is the trace of C, and A is the unit matrix. This can make the covariance matrix symmetric and positively definite, and can convert the multi-scale feature map data of the face image into manifold data.
[0056] The transformed manifold data SPD matrix is converted into a low-dimensional SPD matrix through the bilinear mapping B of the BiMap layer. The mapping method is: in is the SPD matrix of the K-th layer input, (d k <d k-1 ) is the transformation matrix (i.e. weight), is the output low-dimensional SPD matrix. To ensure that the output matrix X k is a valid SPD matrix, the transformation matrix W k The row rank constraint needs to be satisfied. By using the BiMap layer, in the original SPD manifold The input matrix on is transformed into another SPD manifold That is, the data space on the BiMap layer corresponds to an SPD manifold; then the ReEig layer uses a nonlinear function R to correct the SPD matrix, which corrects them by adjusting the smaller eigenvalues of the SPD matrix. The calculation method is: Among them U k-1 and Σ k-1 By eigenvalue decomposition We get, ∈ is the correction threshold, I is the unit matrix, max(∈I,∑ k-1 ) is the operation of taking the maximum value of the diagonal elements; the LogEig layer is mainly used to perform Riemann calculation on the obtained SPD matrix so that it can be applied to other subsequent layers. The Riemann calculation function applied in the Kth layer is: Where X k-1 The eigenvalue decomposition of is expressed as It is a diagonal matrix of eigenvalue logarithms. Finally, the matrix form of facial image features can generate a more compact and discriminative symmetric positive definite matrix through the Riemann manifold network structure (SPDNet) of symmetric positive definite matrix, thereby improving the face recognition effect and accelerating the recognition rate. The present invention has achieved good results in the accuracy and rate of recognition of the CIFAR-10 data set, which is significantly higher than the recognition rate and rate of the convolutional neural network.
Claims
1. A face recognition method based on asymmetric features and Riemannian manifold structure, characterized in that: The following steps are involved: S1, randomly divide the labeled face image dataset into training set and validation set; S2, using a convolutional neural network based on the improved GoogLeNet and ResNet network structures; specifically: S2.1, construct a layer of multi-channel feature map through a 3×3 convolution kernel; S2.2, a ReLU activation function is added after the multi-channel feature map to introduce a nonlinear structure; S3, using multiple convolution kernels of different sizes to stack and build different Inception modules; specifically: S3.1, use multiple convolution kernels of different sizes to stack and build different Inception modules; S3.
2. In each Inception module, the feature maps obtained by each convolution kernel are stacked according to the depth. By training N layers of features at different scales, all hidden layers are trained to obtain multi-scale features of the face image. S4, fine-tuning the network framework parameters for obtaining the multi-scale feature map of the face image in a supervised manner; S5, obtain the multi-scale feature map of the face image by the convolutional neural network structure, flatten each feature map and combine them into a matrix form according to the rules, transform the matrix to obtain its covariance matrix, and then use the regularization method to make the covariance matrix have the characteristics of symmetric positive definite, thereby converting the multi-scale feature map data of the face image into manifold data; S6. The transformed manifold data SPD matrix is passed through the SPDNet network structure, and a more discriminative SPD matrix with significantly reduced data dimension is generated through the BiMap layer, the ReEig layer, and the LogEig layer. At the same time, it can also be applied to subsequent fully connected layers and softmax layers for face recognition output.
2. A face recognition method based on asymmetric features and Riemannian manifold structure according to claim 1, characterized in that: The fine-tuning described in step S4 adopts the stochastic gradient descent method.
3. The face recognition method based on asymmetric features and Riemannian manifold structure according to claim 1, characterized in that: The step S6 is specifically as follows: S6.1, converting the converted manifold data SPD matrix into a low-dimensional SPD matrix through a bilinear mapping B of a BiMap layer; S6.2, ReEig layer uses nonlinear function R to modify the SPD matrix; S6.3, the LogEig layer performs Riemann calculation on the obtained SPD matrix.
4. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a face recognition method based on asymmetric features and Riemann manifold structure as described in any one of claims 1 to 3 is implemented.
5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, a face recognition method based on asymmetric features and Riemann manifold structure is implemented as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Manifold learning network and computer vision image set classification method based on the same
CN109376787A
Lie group dictionary learning classification method based on image set
CN109978064A
Visual classification method based on Riemannian kernel dictionary learning algorithm
CN110689049A
KR20200093970A
