Mobile terminal face recognition method, electronic device and storage medium

Abstract

The present invention belongs to the technical field of digital image processing and pattern recognition, and discloses a mobile terminal face recognition method, electronic device and storage medium, including: constructing a matrix decomposition convolution unit as the basis for subsequent lightweight face anti-spoofing network and face recognition network; constructing feature extraction modules and classification modules for lightweight face anti-spoofing and face recognition networks respectively; training lightweight face anti-spoofing network and lightweight face recognition network respectively; inputting face images through face anti-spoofing module and face recognition module in sequence to realize safe and efficient mobile terminal face recognition. Based on the matrix decomposition theory and convolution separation ideas, the present invention proposes a complete lightweight, safe and efficient face recognition technology for mobile terminals; it can solve the problems and defects of existing mobile terminal face recognition models, such as low security, complex structure, slow operation speed and difficulty in deployment on mobile terminals.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing and pattern recognition, and in particular relates to a mobile face recognition method, electronic equipment and storage medium. Background Art

[0002] At present, face recognition is a technology that uses computers or mobile terminal cameras to automatically identify or verify the identity of an object from an image or video. Due to its advantages of low acquisition cost and non-invasiveness, face recognition has become one of the most popular biometric recognition modalities. With the continuous improvement of the performance of face recognition technology, it has gradually become popular in all aspects of society. In recent years, face recognition has received great attention on mobile smart terminals such as smartphones, bringing users a better experience. In addition, face recognition based on smartphones has also attracted the attention of a large number of companies and researchers in mobile payments. Consumer payments are made by "swiping faces", which solves the inconvenience of carrying bank cards or cash at all times.

[0003] However, the face recognition systems currently in practical use are extremely vulnerable to attacks. For example, criminals use fake methods such as printed photos to attack, log into other people's accounts, and steal other people's information. In recent years, with the promotion of face recognition, various face recognition systems have been cracked. For example, in October 2019, a "scientific team" in Zhejiang found that it was easy to crack the face-scanning pickup system of the "Fengchao Smart Cabinet" with just a printed photo and take out other people's express parcels. Later, some media conducted tests and found that this was indeed the case. This weakness of face recognition makes the use of face recognition for mobile payments and other security risks. The solution is to require the face recognition system to have the function of efficiently and securely judging whether it is alive, that is, face anti-spoofing technology.

[0004] At the same time, most of the mobile terminal face recognition systems in reality still use traditional face recognition algorithms. Traditional methods are usually based on manual features designed based on personal experience. These features are shallow features, which leads to weak recognition capabilities and poor user experience when the algorithm is used in a specific system. Deep learning is an end-to-end learning process that can automatically learn deep features with strong representation capabilities from the original pixels of the input image and use the learned features for classification.

[0005] Previous deep learning methods have complex structures and large amounts of computation. Therefore, in order to improve the speed of face recognition in specific scenarios, it is particularly important to design an efficient and lightweight neural network.

[0006] Through the above analysis, the problems and defects of the prior art are as follows:

[0007] (1) When traditional face recognition algorithms are used in specific systems, their recognition capabilities are weak and user experience is poor;

[0008] (2) Existing deep learning methods have complex structures and large computational workloads;

[0009] (3) The existing facial recognition systems are not very secure or do not take security into consideration.

[0010] The difficulty of solving the above problems and defects is:

[0011] At present, deep learning methods with complex structures and large computational workloads are applied to mobile devices, resulting in slow system response and poor user experience. At the same time, the accuracy of face recognition with simple convolutional neural networks is low. While ensuring high accuracy, designing lightweight convolutional neural networks to improve system response speed has become a huge challenge that current research needs to address.

[0012] The significance of solving the above problems and defects is:

[0013] While face recognition provides convenience, the security issues and recognition accuracy issues are more worthy of our consideration. However, in current practical applications, most mobile face recognition systems either use traditional methods or complex deep learning models, which makes the face recognition system low in security and accuracy, and easy to be attacked by criminals. Therefore, it is very important to design a mobile-oriented face recognition system that is secure and efficient. The present invention designs a lightweight convolution unit MDCU, designs a high-security face anti-spoofing network LWASNet using the lightweight convolution unit MDCU, and designs an efficient face recognition network LWFace using the lightweight convolution unit MDCU. Under the premise of ensuring the accuracy of face recognition, it solves the problems of large computational complexity of deep learning networks and slow response speed on mobile terminals, and will have good application value in mobile devices in the future. Summary of the invention

[0014] In view of the problems existing in the prior art, the present invention provides a mobile face recognition method, an electronic device and a storage medium.

[0015] The present invention is implemented as follows: a mobile terminal face recognition method, the mobile terminal face recognition method comprising the following steps:

[0016] Step 1: Use convolution separation and matrix singular value decomposition theory to build a lightweight convolution unit MDCU, which reduces the amount of calculation of the convolutional neural network;

[0017] Step 2: Based on the lightweight convolutional unit, a lightweight convolutional neural network feature extraction module and a classification module are constructed for face anti-spoofing to form a lightweight face anti-spoofing network LWASNet. The face anti-spoofing module is added before face recognition to improve the security of face recognition.

[0018] Step 3: Based on the lightweight convolutional unit, a lightweight convolutional neural network feature extraction module and a classification module for face recognition are constructed to form a lightweight face recognition network LWFace, which improves the response speed of the face recognition system.

[0019] Step 4: Use the real and forged face image training sets to train the lightweight face anti-spoofing network LWASNet to obtain a trained high-security face anti-spoofing model;

[0020] Step 5: Use the face recognition image training set to train the lightweight face recognition network LWFace to obtain a trained efficient face recognition model.

[0021] Furthermore, the mobile terminal face recognition method further includes:

[0022] The input face image passes through the lightweight face anti-spoofing neural network LWASNet and the lightweight face recognition neural network LWFace in turn to complete the face anti-spoofing and face recognition tasks respectively, so as to achieve the ultimate goal of safe and efficient face recognition.

[0023] Furthermore, before constructing the lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory in step 1, the following steps are also included:

[0024] A camera is used to collect face images, and the collected images are processed for background removal through a fast face detection module SSD for use in face recognition data sets. The images are divided into training sets and test sets to construct face recognition training sets and test sets. The face recognition data sets are re-photographed to obtain real and forged face image data sets. The real and forged face image data sets are patched to obtain small blocks of face images, and the final real and forged face image data sets are constructed, thereby expanding the real and forged face image data sets. The training set and test set are divided to construct real and forged face image training sets and test sets.

[0025] Further, the step 1 of constructing a lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory includes:

[0026] According to the convolution separation theory, the ordinary two-dimensional convolution kernel is decomposed into single-channel convolution and point convolution;

[0027] The two-dimensional single-channel convolution is decomposed into one-dimensional vertical strip convolution and horizontal strip convolution using matrix singular value decomposition theory;

[0028] Vertical strip convolution layer C1, the convolution kernel size is K*1, K can be 3, 5, or 7;

[0029] Horizontal strip convolution layer C2, the convolution kernel size is 1*K, K can be 3, 5, or 7;

[0030] Point convolution layer C3, the convolution kernel size is 1*1.

[0031] Furthermore, in the step 2, a lightweight face anti-spoofing network LWASNet is designed, and feature learning is performed on a training set of real and forged face images through the lightweight face anti-spoofing network to perform face anti-spoofing; the lightweight face anti-spoofing neural network LWASNet includes a convolutional layer, a pooling layer and a fully connected layer composed of multiple MDCU units connected.

[0032] Furthermore, in the step three, a lightweight face recognition network LWFace is designed, and feature learning of the face recognition training set is performed through the lightweight face anti-spoofing network to classify and recognize faces; the lightweight face recognition neural network LWFace includes a convolutional layer composed of multiple MDCU units connected, a residual learning module, a pooling layer and a fully connected layer.

[0033] Furthermore, the step 4 of training the lightweight face anti-spoofing network LWASNet using the real and forged face image training sets includes:

[0034] The lightweight face anti-spoofing network LWASNet is trained using a training set consisting of pairs of real and forged face images to obtain a trained mobile high-security face anti-spoofing model; the real and forged face image test set is input into the trained mobile high-security face anti-spoofing model to achieve mobile high-security face anti-spoofing.

[0035] Further, the step 5 of training the lightweight face recognition network LWFace using the face recognition image training set includes:

[0036] The lightweight face recognition network LWFace is trained using a face recognition training set to obtain a trained mobile-end efficient face recognition model; the face recognition test set is input into the trained mobile-end efficient face recognition model for classification processing to achieve mobile-end efficient face recognition.

[0037] Another object of the present invention is to provide an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0038] A lightweight convolution unit MDCU is constructed using convolution separation and matrix singular value decomposition theory; a lightweight convolutional neural network feature extraction module and a classification module for face anti-spoofing are constructed to form a lightweight face anti-spoofing network LWASNet; a lightweight convolutional neural network feature extraction module and a classification module for face recognition are constructed to form a lightweight face recognition network LWFace; the lightweight face anti-spoofing network LWASNet is trained using a training set of real and forged face images to obtain a trained high-security face anti-spoofing model; the lightweight face recognition network LWFace is trained using a face recognition image training set to obtain a trained efficient face recognition model.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0040] A lightweight convolution unit MDCU is constructed using convolution separation and matrix singular value decomposition theory; a lightweight convolutional neural network feature extraction module and a classification module for face anti-spoofing are constructed to form a lightweight face anti-spoofing network LWASNet; a lightweight convolutional neural network feature extraction module and a classification module for face recognition are constructed to form a lightweight face recognition network LWFace; the lightweight face anti-spoofing network LWASNet is trained using a training set of real and forged face images to obtain a trained high-security face anti-spoofing model; the lightweight face recognition network LWFace is trained using a face recognition image training set to obtain a trained efficient face recognition model.

[0041] In combination with all the above technical solutions, the advantages and positive effects of the present invention are as follows: the mobile face recognition method provided by the present invention independently designs a lightweight convolution unit MDCU according to the convolution separation and matrix singular value decomposition theory. The module has good feature extraction capability, fast speed and high efficiency in feature extraction; the present invention designs a lightweight face anti-spoofing network LWASNet by cascading multiple lightweight convolution units MDCU for feature extraction; the present invention designs a lightweight face recognition network LWFace by performing feature extraction through multiple convolution units MDCU and residual learning units; at the same time, the present invention proposes adding a face anti-spoofing module before face recognition, and the input image is first subjected to face anti-spoofing by the face anti-spoofing module, and then face recognition is performed, so as to solve the security problem of mobile face recognition and improve the low security and slow system response speed of the existing method.

[0042] The present invention proposes a complete set of safe and efficient mobile face recognition technology, which can solve the many defects of traditional face recognition technology, such as narrow application scope, low recognition performance, slow response speed, etc. This embodiment provides new theory and new method support for the practical application of face recognition technology, making face recognition technology more practical, reliable and popular. At the same time, compared with the existing technology, the present invention also has the following advantages:

[0043] (1) The present invention independently designs a lightweight convolution unit MDCU based on convolution separation and matrix singular value decomposition theory. The module has good feature extraction capability, fast speed and high efficiency in feature extraction.

[0044] (2) The present invention designs a lightweight face recognition network LWFace for facial feature extraction, which has the advantages of small computational complexity and strong feature learning ability.

[0045] (3) The present invention proposes to consider the high security of face recognition, that is, to design a lightweight face anti-spoofing network LWASNet during the face recognition process, thereby improving the security of the entire face recognition process and making it less vulnerable to attacks by criminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 It is a schematic diagram of a mobile face recognition method provided by an embodiment of the present invention.

[0048] Figure 2 It is a schematic diagram of the high-security face anti-spoofing process on a mobile terminal provided by an embodiment of the present invention.

[0049] Figure 3 It is a schematic diagram of the efficient face recognition process on a mobile terminal provided by an embodiment of the present invention.

[0050] Figure 4 It is a schematic diagram of the structure of a lightweight convolution unit MDCU provided in an embodiment of the present invention.

[0051] Figure 5 It is a schematic diagram of single-channel convolution provided by an embodiment of the present invention.

[0052] Figure 6 Schematic diagram of point convolution provided by an embodiment of the present invention.

[0053] Figure 7It is a schematic diagram of vertical strip convolution provided by an embodiment of the present invention.

[0054] Figure 8 It is a schematic diagram of the horizontal strip convolution provided by an embodiment of the present invention.

[0055] Fig. 9 It is a schematic diagram of the structure of the lightweight face anti-spoofing network LWASNet provided in an embodiment of the present invention.

[0056] Fig.10 It is a schematic diagram of a residual learning module provided in an embodiment of the present invention.

[0057] Fig.11 It is a schematic diagram of the structure of the lightweight face recognition network LWFace provided by an embodiment of the present invention.

[0058] Fig.12 It is a schematic diagram of an example of a real face image for anti-face spoofing on a mobile terminal provided by an embodiment of the present invention.

[0059] Fig.13 It is a schematic diagram of an example of a mobile face anti-spoofing face forgery image provided by an embodiment of the present invention.

[0060] Fig.14 It is a structural diagram of a safe and efficient mobile face recognition system provided by an embodiment of the present invention.

[0061] Fig.15 It is a schematic diagram of a safe and efficient mobile face recognition electronic device provided by an embodiment of the present invention.

[0062] Fig.16 It is a schematic diagram of a computer-readable storage medium for secure and efficient mobile face recognition provided by an embodiment of the present invention.

[0063] Fig.17 It is a flow chart of a mobile face recognition method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0065] In view of the problems existing in the prior art, the present invention provides a mobile face recognition method, an electronic device and a storage medium. The present invention is described in detail below with reference to the accompanying drawings.

[0066] like Fig.17 As shown, the mobile terminal face recognition method provided by the embodiment of the present invention includes the following steps:

[0067] S101, uses convolution separation and matrix singular value decomposition theory to build a lightweight convolution unit MDCU;

[0068] S102, constructing a lightweight convolutional neural network feature extraction module and a classification module for face anti-spoofing, forming a lightweight face anti-spoofing network LWASNet;

[0069] S103, constructing a lightweight convolutional neural network feature extraction module and a classification module for face recognition to form a lightweight face recognition network LWFace;

[0070] S104, training the lightweight face anti-spoofing network LWASNet using a training set of real and forged face images to obtain a trained high-security face anti-spoofing model;

[0071] S105, training the lightweight face recognition network LWFace using the face recognition image training set to obtain a trained efficient face recognition model.

[0072] The technical solution of the present invention is further described below in conjunction with specific embodiments.

[0073] Example 1

[0074] The mobile terminal face recognition method provided by the embodiment of the present invention comprises the following steps:

[0075] A lightweight convolution unit, namely the Matrix Decomposition Convolutional Unit (MDCU), is constructed based on the theory of matrix singular value decomposition.

[0076] Based on MDCU, the feature extraction module and classification module of the face anti-spoofing lightweight convolutional neural network are constructed to form a lightweight face anti-spoofing network (Light-Weight Anti-Spoofing Net, LWASNet);

[0077] Based on MDCU, the feature extraction module and classification module of the lightweight convolutional neural network for face recognition are constructed to form a lightweight face recognition network (Light-Weight Face, LWFace);

[0078] The lightweight face anti-spoofing network LWASNet is trained according to the real and forged face image training set to obtain a trained high-security face anti-spoofing model;

[0079] Inputting the test set of real and forged face images into the trained high-security face anti-spoofing model to achieve high-security face anti-spoofing;

[0080] The lightweight face recognition network LWFace is trained according to the face recognition training set to obtain a trained efficient face recognition model;

[0081] The face recognition test set is input into the trained efficient face recognition model for classification processing to achieve efficient face recognition.

[0082] The input image is detected by the face detection module to obtain a face image, and the high-security face anti-spoofing model and the efficient face recognition model are used to achieve safe and efficient face recognition. Fig.14 It is a structural diagram of a safe and efficient mobile face recognition system provided by an embodiment of the present invention.

[0083] The mobile terminal face recognition method provided by the embodiment of the present invention includes:

[0084] Step 1: Use a camera to collect face images, remove the background of the collected images through the fast face detection module SSD for face recognition data set, divide the training set and test set, and build face recognition training set and test set; re-shoot the face recognition data set to obtain real and forged face image data sets. In order to further expand the real and forged face image data sets, the real and forged face image data sets are patched to obtain small blocks of face images, and the final real and forged face image data sets are constructed. The training set and test set are divided to construct the real and forged face image training set and test set;

[0085] Step 2: Based on the theories of convolution separation and matrix singular value decomposition, a lightweight convolution unit MDCU is designed, including:

[0086] According to the convolution separation theory, the ordinary two-dimensional convolution kernel is decomposed into single-channel convolution and point convolution;

[0087] Then use the matrix singular value decomposition theory to decompose the two-dimensional single-channel convolution into one-dimensional vertical strip convolution and horizontal strip convolution;

[0088] Vertical strip convolution layer C1, the convolution kernel size is K*1 (K can be 3, 5, 7);

[0089] Horizontal strip convolution layer C2, the convolution kernel size is 1*K (K can be 3, 5, 7);

[0090] Point convolution layer C3, the convolution kernel size is 1*1;

[0091] Step 3: Design a lightweight face anti-spoofing network LWASNet, and use the lightweight face anti-spoofing network to learn features of a training set of real and forged face images to perform face anti-spoofing. The lightweight face anti-spoofing network LWASNet includes:

[0092] The connection of multiple MDCU units constitutes a convolutional layer;

[0093] Pooling layer;

[0094] Fully connected layers;

[0095] Step 4: Design a lightweight face recognition network LWFace, perform feature learning on the face recognition training set through a lightweight face anti-spoofing network, and classify and recognize faces. The lightweight face recognition network LWFace includes:

[0096] The connection of multiple MDCU units constitutes a convolutional layer;

[0097] The connection of MDCU units and the identity mapping of the input constitute the residual learning module;

[0098] Pooling layer;

[0099] Fully connected layer.

[0100] The present invention independently designs a lightweight convolution unit MDCU according to the convolution separation and matrix singular value decomposition theory. The module has good feature extraction capability, and is fast and efficient in feature extraction. The present invention designs a lightweight face anti-spoofing network LWASNet by cascading multiple lightweight convolution units MDCU for feature extraction. The present invention designs a lightweight face recognition network LWFace by performing feature extraction through multiple convolution units MDCU and residual learning units. At the same time, the present invention proposes adding a face anti-spoofing module before face recognition, and the input image is first subjected to face anti-spoofing by the face anti-spoofing module, and then face recognition is performed, so as to solve the security problem of face recognition on the mobile terminal and improve the problems of low security and slow system response speed of the existing method.

[0101] Example 2

[0102] At present, most of the mobile-based face recognition technologies use traditional methods. Therefore, this technology is mostly limited to good backgrounds and user cooperation. It usually performs poorly in complex backgrounds and environments with diverse facial expressions, and this method has low security. With the maturity and development of Android technology, face recognition technology based on smart terminals and smartphones is widely used in people's daily lives. Face recognition technology based on traditional methods is increasingly difficult to meet the requirements. Safe and efficient face recognition technology for mobile terminals is particularly important. The use of face recognition technology based on deep learning makes up for the shortcomings of traditional methods such as security and low accuracy. The use of lightweight convolutional neural networks makes up for the shortcomings of complex neural networks such as large amount of calculation and slow response speed. Therefore, please refer to Figure 1 , Figure 1: is a schematic diagram of a mobile face recognition method provided by an embodiment of the present invention. This embodiment provides a mobile face recognition method, which includes the following steps:

[0103] Step 1: Obtain a face recognition dataset and divide it into a face recognition training set and a test set. Obtain a real face and a forged face image dataset and divide it into a real face and a forged face image training set and a test set.

[0104] Specifically, this embodiment uses a camera to collect facial images of individuals and obtains a face recognition dataset through a fast face detection module SSD. The face recognition dataset is re-photographed and patched to obtain a dataset of real and forged face images. Fig.12 is a schematic diagram of an example of a real face image for anti-face spoofing on a mobile terminal provided by an embodiment of the present invention, Fig.13 It is a schematic diagram of an example of a mobile face anti-spoofing face forgery image provided by an embodiment of the present invention.

[0105] Step 2: construct a lightweight convolution unit MDCU. Specifically, in order to make the mobile face recognition system highly secure and responsive, this embodiment proposes a new lightweight convolution unit MDCU based on convolution separation and matrix singular value decomposition theory. Figure 4 : is a schematic diagram of the structure of a lightweight convolution unit MDCU provided in an embodiment of the present invention. Specifically, step 2 of this embodiment includes:

[0106] The ordinary convolution is divided into single channel convolution and point convolution. Figure 5 is a schematic diagram of a single-channel convolution provided by an embodiment of the present invention, Figure 6 is a schematic diagram of point convolution provided by an embodiment of the present invention; the single-channel convolution is then divided into vertical strip convolution and horizontal strip convolution, Figure 7 is a schematic diagram of a vertical strip convolution provided by an embodiment of the present invention, Figure 8 It is a schematic diagram of the horizontal strip convolution provided by an embodiment of the present invention; the first convolution layer C1 of the lightweight convolution unit MDCU is a vertical strip convolution layer, and the convolution kernel size is K*1 (K can be 3, 5, or 7); the second convolution layer C2 of the lightweight convolution unit MDCU is a horizontal strip convolution layer, and the convolution kernel size is 1*K (K can be 3, 5, or 7); the third convolution layer C3 of the lightweight convolution unit MDCU is a point convolution layer, and the convolution kernel size is 1*1. The input image is single-channel convolved through the first convolution layer and the second convolution layer, and then the convolution results of each channel are superimposed and input into the third convolution layer, and the output image of the third convolution layer is input into the pooling layer.

[0107] Specifically, in the process of constructing MDCU in this embodiment, based on the convolution separation and matrix singular value decomposition theory, the matrix singular value decomposition decomposes the two-dimensional convolution kernel into a one-dimensional vertical strip convolution kernel and a horizontal strip convolution kernel. The specific construction process is as follows:

[0108] P=U∑V T (1)

[0109]

[0110] As shown in formula (1), for any two-dimensional convolution kernel, assuming that P is its corresponding matrix, the matrix P can be decomposed into two unitary matrices U and V through matrix singular value decomposition; then the matrix is ​​decomposed through formula (2) to obtain the eigenvector u of the U matrix i As the vertical strip convolution kernel, take V T Eigenvectors of a matrix As the horizontal strip convolution kernel, P is decomposed into the sum of the products of several vertical strip convolution kernels and horizontal strip convolution kernels, σ i is its coefficient; finally, according to formula (3), any input image I in The corresponding output image I out Expressed as input image I in Convolution with the convolution kernel P, and further expressed as the superposition of one-dimensional convolution operations between the input image and the vertical strip convolution kernel and the horizontal strip convolution kernel.

[0111] Step 3: Construct a lightweight convolutional neural network feature extraction module and classification module for face anti-spoofing to form a lightweight face anti-spoofing network LWASNet.

[0112] Specifically, see Fig. 9 , Fig. 9 It is a schematic diagram of the structure of the lightweight face anti-spoofing network LWASNet provided by an embodiment of the present invention. This embodiment proposes a fast, highly secure, and specially designed high-security face anti-spoofing network model LWASNet. Specifically: the constructed high-security face anti-spoofing network model LWASNet includes a lightweight convolution unit MDCU, a pooling layer, and a fully connected layer, wherein the input image is input into the lightweight convolution unit MDCU, and the image output by the lightweight convolution unit MDCU is connected to the input of the pooling layer, and the output image of the pooling layer is connected to the input of the fully connected layer. Each lightweight convolution unit MDCU includes a first convolution layer C1, a second convolution layer C2, and a third convolution layer C3.

[0113] For the parameter design of each layer in the lightweight face anti-spoofing network LWASNet during the verification process of this embodiment, please refer to Table 1. The padding method during the convolution process is 0 padding.

[0114] Table 1 Parameter design of each layer in the lightweight face anti-spoofing network (LWASNet)

[0115]

[0116]

[0117]

[0118] Step 4, see Figure 2 , Figure 2 This is a schematic diagram of a high-security face anti-spoofing process on a mobile terminal in a mobile terminal face recognition method provided by an embodiment of the present invention. The lightweight face anti-spoofing network LWASNet is trained according to a training set of real and forged face images to obtain a trained high-security face anti-spoofing model. The trained model is tested according to a test set of real and forged face images, and the test set of real and forged face images is input into the trained high-security face anti-spoofing model to extract face features. The face feature set, real face labels, and forged face labels are input into a Sigmoid function, and a classification label is output to achieve high-security face anti-spoofing.

[0119] Step 5: Build a lightweight convolutional neural network feature extraction module and classification module for face recognition on the mobile terminal to form a lightweight face recognition network LWFace.

[0120] Specifically, see Fig.11 , Fig.11 : This is a schematic diagram of the structure of a lightweight face recognition network LWFace provided by an embodiment of the present invention. This embodiment proposes a lightweight face recognition network LWFace with fast speed, high accuracy, and specifically for mobile terminals. Specifically: the constructed lightweight face recognition network LWFace includes a lightweight convolution unit MDCU, a residual learning module, a pooling layer, and a fully connected layer. The residual learning module is constructed by connecting the MDCU unit and the identity mapping of the input, such as Fig.10 : is a schematic diagram of the residual learning module provided in this embodiment. The residual module of the input image input cascade is connected to the input of the pooling layer, and the output image of the pooling layer is connected to the input of the fully connected layer. Each lightweight convolution unit MDCU includes a first convolution layer C1, a second convolution layer C2, and a third convolution layer C3.

[0121] In the verification process of this embodiment, the parameter design of each layer in the lightweight face recognition network LWFace is specifically shown in Table 2. The padding method in the convolution process is 0 padding.

[0122] Table 2 Parameter design of each layer in the lightweight face recognition network (LWFace)

[0123]

[0124]

[0125] Step 6, see Figure 3 , Figure 3 This is a schematic diagram of a mobile terminal efficient face recognition process in a mobile terminal face recognition method provided by an embodiment of the present invention. The lightweight face recognition network LWFace is trained according to the face image training set to obtain a trained efficient face recognition model. The face recognition test set is input into the trained efficient face recognition model to obtain a face feature set, and the face feature set and its label are input into the Softmax function for classification, and the individual identity ID information is output to achieve safe and efficient face recognition.

[0126] Specifically, this embodiment uses the Euclidean distance as the metric function, inputs the face test set into the trained efficient face recognition model for feature extraction to obtain the face feature set, inputs the face feature set and its label into the Softmax function to output the individual ID information, and obtains the individual ID information through Softmax to achieve efficient face recognition. The specific steps are as follows:

[0127] (1) The maximum Euclidean distance D within the class max As the threshold, the minimum Euclidean distance D between the predicted label and the current image is set bw_min With D max Compare, if less than D max If the predicted classification is correct, it is considered that the predicted classification is correct, otherwise it is considered that the predicted classification is wrong, and the number of correct and wrong classification categories is counted.

[0128] (2) According to the number of correct and incorrect classifications counted in step (1), the classification accuracy is calculated. The specific formula is designed as follows:

[0129]

[0130] TP: true positive, the number of samples that are actually positive and predicted to be positive

[0131] FP: false positive, the number of samples that are actually negative and predicted to be positive

[0132] TN: true negative, the number of samples that are actually negative and predicted to be negative

[0133] FN: false negative, the number of samples that are actually positive but predicted to be negative

[0134] In summary, the introduction of efficient mobile face recognition technology in this embodiment improves the network computing speed through the lightweight convolution unit MDCU.

[0135] The experiment designed in this embodiment is demonstrated from the following aspects:

[0136] (1) In order to illustrate that the efficiency of the lightweight face anti-spoofing network LWASNet and the lightweight face recognition network LWFace of the present invention is better than traditional face recognition and other mainstream deep learning methods, the calculation amount of the lightweight convolution unit MDCU is compared with the calculation amount of the ordinary convolution module. Please refer to Table 3, which is a comparison result of the calculation amount of the lightweight convolution unit MDCU and the calculation amount of the ordinary convolution module provided in this embodiment. In the table, the convolution kernel size of 3*3 is taken as an example, where the number of image channels (M), the number of convolution kernels (N), and the input image size (DF), the smaller the calculation number value, the faster the neural network reasoning speed.

[0137] Table 3 Comparison of computational complexity between matrix decomposition convolution unit (MDCU) and ordinary convolution module

[0138]

[0139] (2) To prove the safe and efficient face recognition performance of the present invention, it is compared with another currently mainstream lightweight network MobileNet, and the number of operations and face recognition rate during the reasoning process of each neural network are calculated. Table 4 shows the comparison results of the number of operations of different face recognition methods provided in this example.

[0140] Table 4 Comparison of the number of operations of different face recognition algorithms

[0141]

[0142] (3) To demonstrate the high security of the lightweight face anti-spoofing network LWASNet and the accuracy of the lightweight face recognition network LWFace of the present invention, Table 5 is a comparison of the accuracy of different face anti-spoofing methods provided in this example, and Table 6 is a comparison of the accuracy of different face recognition methods provided in this example. It can be seen that while ensuring the accuracy, the number of neural network calculations is reduced, the security of face recognition is improved, the accuracy of face recognition is improved, and thus the overall performance of face recognition is improved.

[0143] Table 5 Comparison of the accuracy of different face anti-spoofing methods

[0144]

[0145] Table 6 Comparison of the accuracy of different face recognition methods

[0146]

[0147] It can be seen that this embodiment introduces a lightweight face anti-spoofing network LWASNet to address the low security of traditional face recognition methods. The detected face image is input into the face anti-spoofing module for face anti-spoofing, so as to determine whether the input image is a real face or a fake face before face recognition, thereby achieving a high security effect of the face recognition system; this embodiment designs a lightweight face recognition network LWFace to address the problems of complex traditional deep learning models and large computational complexity. Compared with traditional methods, the lightweight face recognition network LWFace model proposed by the present invention is simple and has a small computational complexity, and is suitable for mobile face recognition systems. It can successfully solve the problems of low security and slow response speed of traditional face recognition systems. Experiments show that the use of the method of the present invention can significantly improve system efficiency.

[0148] This embodiment proposes a complete set of secure and efficient mobile face recognition technology, which can solve the defects of traditional face recognition technology such as low security, complex network structure, and slow response speed; this embodiment provides new theoretical and methodological support for the practical application of face recognition technology, making face recognition technology more practical, reliable and popular; this embodiment can be widely used in smart terminals, attendance, civil monitoring, public security law enforcement, access control, community entrances and other application scenarios.

[0149] Example 3

[0150] Based on the above examples, see Fig.15 , Fig.15 Schematic diagram of a secure and efficient mobile face recognition electronic device provided by an embodiment of the present invention. A secure and efficient mobile face recognition electronic device provided by an embodiment of the present invention comprises an image collector, a display, a processor, a communication interface, a memory, a communication bus and a peripheral application system, wherein the image collector, the display, the processor, the communication interface, the memory and the peripheral calling system communicate with each other via the communication bus;

[0151] The image collector is used to collect face image data;

[0152] The display is used to display the face image and display the face anti-spoofing and face recognition results;

[0153] Memory, used to store computer programs;

[0154] Peripheral application system, used to realize peripheral system functions, acts as a host computer, and calls face recognition results for other face recognition-based application purposes;

[0155] The processor is used to execute a computer program stored in the memory, and the computer program is executed by the processor to implement the following steps:

[0156] Step 1: Turn on the camera to collect face images.

[0157] Specifically, in step 1 of the embodiment of the present invention, a camera is used to collect a face image; and the collected image is pre-processed by removing the background through a fast face detection module SSD to obtain a face image.

[0158] Step 2: Construct a lightweight convolution unit MDCU.

[0159] Specifically, step 2 of the embodiment of the present invention constructs a first convolutional layer C1, a second convolutional layer C2, and a third convolutional layer C3.

[0160] Step three, build a lightweight face anti-spoofing network LWASNet, use the real and forged face image training set to train the lightweight face anti-spoofing network LWASNet, and obtain a trained high-security face anti-spoofing model.

[0161] Specifically, step three of the embodiment of the present invention includes:

[0162] (1) The dataset of real and forged face images is divided into two parts, with the training set accounting for 3 / 4 and the test set accounting for 1 / 4.

[0163] (2) Use the designed lightweight face anti-spoofing network LWASNet to train the training set and obtain a trained high-security face anti-spoofing model;

[0164] Step 4: Build a lightweight face recognition network LWFace, use the face recognition training set to train the lightweight face recognition network LWFace, and obtain a trained efficient face recognition model.

[0165] Specifically, step 4 of the embodiment of the present invention includes:

[0166] (1) The face recognition dataset is divided into two parts: a training set and a test set. The training set accounts for 3 / 4 and the test set accounts for 1 / 4.

[0167] (2) Use the designed lightweight face recognition network LWFace to train the training set and obtain a trained efficient face recognition model;

[0168] Step 5: The face image captured by the camera is subjected to face anti-spoofing by the face anti-spoofing module to determine whether the captured image is a real face image; the face recognition module is further used to perform face recognition on the captured face image.

[0169] Step 6: In response to the request of the peripheral application system, the face recognition result of step 5 is sent to the peripheral system to achieve other peripheral application purposes based on the face recognition result. The peripheral application system is not limited in actual implementation, and the following are just examples: (1) Campus access control system based on face recognition, such as can be used to limit the entry and exit of campus personnel at the entrance and exit of the campus, facilitating the entry and exit management of campus personnel; (2) The punch-in system based on face recognition can be deployed in occasions where punch-in is required, such as recording the daily attendance of employees in the company.

[0170] A safe and efficient mobile face recognition electronic device provided by an embodiment of the present invention can execute the above-mentioned face recognition program embodiment based on the mobile face recognition method embodiment. The electronic device provided by this embodiment adopts the mobile face recognition method of the above-mentioned embodiment 2, which solves the problems of low accuracy, slow response speed, and easy attack of traditional face recognition equipment, and can be widely used in attendance, access control and other occasions.

[0171] Example 4

[0172] Based on the above Example 2, see Fig.16 , Fig.16 Schematic diagram of a computer-readable storage medium for secure and efficient mobile face recognition provided by an embodiment of the present invention. A computer-readable storage medium provided by an embodiment of the present invention stores a computer program, which, when executed by a processor, implements the following steps:

[0173] Step 1: The program starts and necessary initialization is performed.

[0174] Step 2: Turn on the camera to collect face images.

[0175] Specifically, in step 2 of the embodiment of the present invention, a camera is used to collect a face image; and the collected image is preprocessed by background removal through a fast face detection module SSD to obtain a face image.

[0176] Step 3: Construct a lightweight convolution unit MDCU.

[0177] Specifically, step three of the embodiment of the present invention constructs a lightweight convolution unit MDCU consisting of a first convolution layer C1, a second convolution layer C2, and a third convolution layer C3 for subsequent facial image feature learning.

[0178] Step 4: Build a lightweight face anti-spoofing network LWASNet, and use the real and forged face image training set to train the lightweight face anti-spoofing network LWASNet to obtain a trained high-security face anti-spoofing model.

[0179] Specifically, step three of the embodiment of the present invention includes:

[0180] (1) The dataset of real and forged face images is divided into two parts, with the training set accounting for 3 / 4 and the test set accounting for 1 / 4.

[0181] (2) Use the designed lightweight face anti-spoofing network LWASNet to train the training set and obtain a trained high-security face anti-spoofing model;

[0182] Step 5: Build a lightweight face recognition network LWFace, use the face recognition training set to train the lightweight face recognition network LWFace, and obtain a trained efficient face recognition model.

[0183] Specifically, step 4 of the embodiment of the present invention includes:

[0184] (1) The face recognition dataset is divided into two parts: a training set and a test set. The training set accounts for 3 / 4 and the test set accounts for 1 / 4.

[0185] (2) Use the designed lightweight face recognition network LWFace to train the training set and obtain a trained efficient face recognition model;

[0186] Step 6: The face image captured by the camera is subjected to face anti-spoofing by the face anti-spoofing module to determine whether the captured image is a real face image; further, the face recognition module is used to perform face recognition on the captured face image.

[0187] Step seven, exit the program when it needs to end.

[0188] A computer-readable storage medium provided in an embodiment of the present invention can execute the above-mentioned embodiment of the mobile-end face recognition method, and a computer program of the above-mentioned mobile-end face recognition method is stored thereon, which can be used by mobile terminals and Internet of Things face recognition devices, thereby improving the security and accuracy of the face recognition devices and bringing a good experience to users.

[0189] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0190] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A mobile terminal face recognition method, characterized in that: The mobile terminal face recognition method comprises the following steps: Step 1: Use convolution separation and matrix singular value decomposition theory to construct a lightweight convolution unit MDCU; Step 2: Based on the lightweight convolutional unit, a lightweight convolutional neural network feature extraction module and a classification module are constructed for face anti-spoofing to form a lightweight face anti-spoofing network LWASNet. Step 3: Based on the lightweight convolutional unit, a lightweight convolutional neural network feature extraction module and a classification module are constructed for face recognition to form a lightweight face recognition network LWFace. Step 4: Use the real and forged face image training sets to train the lightweight face anti-spoofing network LWASNet to obtain a trained high-security face anti-spoofing model; Step 5, using the face recognition image training set to train the lightweight face recognition network LWFace to obtain a trained efficient face recognition model; The step 1 of constructing a lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory includes: According to the convolution separation theory, the ordinary two-dimensional convolution kernel is decomposed into single-channel convolution and point convolution; The two-dimensional single-channel convolution is decomposed into one-dimensional vertical strip convolution and horizontal strip convolution using matrix singular value decomposition theory; Vertical strip convolution layer C1, the convolution kernel size is K*1, K is 3, 5 or 7; Horizontal strip convolution layer C2, the convolution kernel size is 1*K, K is 3, 5 or 7; Point convolution layer C3, the convolution kernel size is 1*1; In the step 2, a lightweight face anti-spoofing network LWASNet is designed, and feature learning is performed on a training set of real and forged face images through the lightweight face anti-spoofing network to perform face anti-spoofing; the lightweight face anti-spoofing neural network LWASNet includes a convolutional layer composed of multiple MDCU units connected, a pooling layer and a fully connected layer; In the step 3, a lightweight face recognition network LWFace is designed, and features of the face recognition training set are learned through the lightweight face anti-spoofing network to classify and recognize faces; the lightweight face recognition neural network LWFace includes a convolutional layer composed of a plurality of MDCU units connected, the connection of the MDCU units and the identity mapping of the input constitute a residual learning module, a pooling layer and a fully connected layer; The mobile face recognition method further includes: inputting a face image and sequentially passing through a lightweight face anti-spoofing neural network LWASNet and a lightweight face recognition neural network LWFace to respectively complete face anti-spoofing and face recognition tasks, so as to achieve the ultimate goal of safe and efficient face recognition; The detected face image is input into the face anti-spoofing network for face anti-spoofing, so as to determine whether the input image is a real face or a fake face before face recognition.

2. The mobile terminal face recognition method according to claim 1, characterized in that: Before constructing the lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory in the step 1, the method also includes: using a camera to collect face images, performing background removal processing on the collected images through a fast face detection module SSD for use in a face recognition data set, dividing the training set and the test set, and constructing a face recognition training set and a test set; reshooting the face recognition data set to obtain a real face and a forged face image data set, patching the real face and the forged face image data set to obtain a small block image in the face image, constructing the final real face and forged face image data set, and then expanding the real face and forged face image data set; dividing the training set and the test set, and constructing a real face and forged face image training set and a test set.

3. The mobile terminal face recognition method according to claim 1, characterized in that: The step 4 of training the lightweight face anti-spoofing network LWASNet using a training set of real and forged face images includes: training the lightweight face anti-spoofing network LWASNet using a training set consisting of real and forged face image pairs to obtain a trained mobile high-security face anti-spoofing model; inputting the real and forged face image test set into the trained mobile high-security face anti-spoofing model to achieve mobile high-security face anti-spoofing.

4. The mobile terminal face recognition method according to claim 1, characterized in that: The step 5 of training the lightweight face recognition network LWFace using the face recognition image training set includes: training the lightweight face recognition network LWFace using the face recognition training set to obtain a trained mobile-end efficient face recognition model; inputting the face recognition test set into the trained mobile-end efficient face recognition model for classification processing to achieve mobile-end efficient face recognition.

5. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the mobile terminal face recognition method according to any one of claims 1 to 4, comprising the following steps: constructing a lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory; constructing a lightweight convolution neural network feature extraction module and a classification module for face anti-spoofing to form a lightweight face anti-spoofing network LWASNet; constructing a lightweight convolution neural network feature extraction module and a classification module for face recognition to form a lightweight face recognition network LWFace; The lightweight face anti-spoofing network LWASNet is trained using a training set of real and forged face images to obtain a trained high-security face anti-spoofing model; the lightweight face recognition network LWFace is trained using a face recognition image training set to obtain a trained efficient face recognition model.

6. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the mobile face recognition method according to any one of claims 1 to 4, comprising the following steps: constructing a lightweight convolution unit MDCU using convolution separation and matrix singular value decomposition theory; constructing a lightweight convolutional neural network feature extraction module and a classification module for face anti-spoofing to form a lightweight face anti-spoofing network LWASNet; constructing a lightweight convolutional neural network feature extraction module and a classification module for face recognition to form a lightweight face recognition network LWFace; training the lightweight face anti-spoofing network LWASNet using a training set of real and forged face images to obtain a trained high-security face anti-spoofing model; training the lightweight face recognition network LWFace using a face recognition image training set to obtain a trained efficient face recognition model.

Images