Method, device, readable storage medium and equipment for silent face liveness detection

Abstract

The present invention discloses a method, device, readable storage medium and equipment for silent liveness detection of human face, which belongs to the field of human face recognition. The method obtains a human face image to be detected and performs preprocessing; the preprocessed human face image is input into a pre-trained multi-cascade network to determine whether the human face image is alive; wherein the multi-cascade network includes a multi-level neural network cascaded front and back, each level of neural network is used for liveness judgment of a prosthesis attack type, when the current level neural network judges it as liveness, it enters the next level neural network, when any level neural network judges it as a prosthesis, the human face image is judged as a prosthesis, and when the last level neural network judges it as liveness, the human face image is judged as liveness. The present invention adopts a deep learning method and uses multiple shallow network cascades to perform silent liveness detection of human face, which improves the accuracy and efficiency of liveness detection and greatly improves the stability of the algorithm.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a method, device, readable storage medium and equipment for silent face liveness detection. Background Art

[0002] In recent years, face recognition has been widely used in finance, security and other fields due to its convenience and non-contact advantages. However, it is also easy to be used by others to print photos or shoot videos to create fake faces, attack face recognition systems, and pass through in the name of others. Therefore, liveness detection based on face images is extremely important and is the prerequisite for face recognition.

[0003] At present, there are two common types of face liveness detection technologies: user-cooperative liveness detection and silent liveness detection. User-cooperative liveness detection means that the liveness detection system randomly sends multiple facial action instructions, requiring the user to make sequential response actions within a specified time. The system determines whether the user's response action is accurate, and then makes a judgment on whether it is a live face. Silent liveness detection means that the user completes the liveness detection task without any perception. It can be intuitively seen from the two definitions of liveness detection that silent face liveness detection breaks the formula of cooperative liveness detection "completing the specified action = I am alive", does not require user cooperation, is more humane, has a fast detection speed and a good user experience.

[0004] At present, the mainstream silent face liveness detection methods can be roughly divided into: traditional manually designed classification feature methods and deep learning-based methods.

[0005] 1. Liveness detection based on traditional artificially designed features

[0006] The goal of early silent face liveness detection was very clear: to find the difference between live and non-live attack images, design highly targeted features, and train live and non-live face classifiers. This difference is usually manifested as texture differences in imaging (such as moiré), color differences (different color distributions in different color spaces), and non-rigid changes in facial micro-movements analyzed from continuous frame images. In response to these differences, LBP features and Gabor features with different parameters are generally designed. Finally, live and non-live feature classifiers are trained based on machine learning methods. Feature classifiers are generally classifiers such as SVM.

[0007] Artificially designed features are subject to the designer's prior knowledge and require repeated experiments and adjustments to find effective artificial features, which takes a long time. In addition, with the development of technology, the imaging of various electronic product lenses is extremely delicate. At the same time, the rise of beauty cameras has weakened the texture differences in imaging, making it difficult to classify artificially designed features.

[0008] 2. Liveness Detection Based on Deep Learning

[0009] When deep learning processes face liveness detection tasks, it regards it as a binary or multi-classification task, driven by a large amount of live and non-live face data, and automatically learns the features that can effectively distinguish live and non-live faces. Binary classification is to classify all prosthetic attack images into one category, and multi-classification is to classify prosthetic attacks according to attack categories, such as print attacks, screen attacks, mask attacks, etc.

[0010] Generally, the more complex the neural network (the deeper and wider the network structure), the better the knowledge extraction of the training data set. Similarly, designing a wide and deep network can simply and directly improve the accuracy of silent face detection. However, complex networks have several disadvantages:

[0011] 1. A more complex network means more parameters and requires more computing resources, which limits the speed performance of the algorithm and makes it difficult to deploy on smart terminals and embedded devices.

[0012] 2. Prosthetic attacks come in a variety of forms, and data collection is difficult. To deal with new types of attacks, it is usually necessary to retrain or fine-tune the neural network. A single network is unlikely to cope with many types of attacks. Summary of the invention

[0013] In order to solve the defects of low efficiency and low stability of the silent face liveness detection method in the prior art, the present invention provides a silent face liveness detection method, device, readable storage medium and equipment, which adopts a deep learning method and uses multiple shallow network cascades to perform silent face liveness detection, thereby improving the accuracy and efficiency of liveness detection and greatly improving the stability of the algorithm.

[0014] The present invention provides the following technical solutions:

[0015] In a first aspect, the present invention provides a method for detecting silent liveness of a face, the method comprising:

[0016] Obtain the face image to be detected and perform preprocessing;

[0017] The preprocessed face image is input into a pre-trained multi-cascade network to determine whether the face image is alive;

[0018] Among them, the multi-cascade network includes a multi-level neural network cascaded front and back, and each level of the neural network is used for liveness judgment of a prosthesis attack type. When the current level neural network judges that it is live, it enters the next level neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live.

[0019] In a second aspect, the present invention provides a silent face liveness detection device, the device comprising:

[0020] An acquisition module is used to acquire the face image to be detected and perform preprocessing;

[0021] A detection module is used to input the pre-processed face image into a pre-trained multi-cascade network to determine whether the face image is alive;

[0022] Among them, the multi-cascade network includes a multi-level neural network cascaded front and back, and each level of the neural network is used for liveness judgment of a prosthesis attack type. When the current level neural network judges that it is live, it enters the next level neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live.

[0023] In a third aspect, the present invention provides a computer-readable storage medium for silent face liveness detection, comprising a memory for storing processor-executable instructions, which, when executed by the processor, implements the steps of the silent face liveness detection method described in the first aspect.

[0024] In a fourth aspect, the present invention provides a device for silent face liveness detection, comprising at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the silent face liveness detection method described in the first aspect are implemented.

[0025] The present invention has the following beneficial effects:

[0026] The present invention uses a multi-level neural network to judge different types of prosthetic attacks. Compared with the existing method of using a whole complex network for live body judgment, the present invention can not only divide and conquer through cascading, but also be flexibly deployed. In addition, each level of neural network is a shallow network with few network parameters and less computing resources required for training. Once a certain level of network judges it as a prosthesis, no subsequent judgment is required. The recognition processing speed is fast and it is easy to deploy on smart terminals and embedded devices.

[0027] For new prosthetic attack types, it is convenient and quick to train a new shallow network with the data of the prosthetic attack type, and cascade it on the existing cascade network without retraining the entire network, thus improving stability. Or if the new prosthetic attack type can be merged into the existing type, it is only necessary to fine-tune the network corresponding to the existing type without retraining the entire network. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the silent liveness detection method of human face of the present invention;

[0029] Figure 2 is a training flow chart of a multi-cascade network of the present invention;

[0030] Figure 3 Schematic diagram of the preprocessing process in the silent liveness detection method for human face of the present invention;

[0031] Figure 4 This is a diagram showing an example of the structure of the first-level neural network net1 when each level of the neural network in a multi-cascade network is independent;

[0032] Figure 5 This is a diagram showing an example of the structure of the second-level neural network net1 when each level of the neural network in the multi-cascade network is independent;

[0033] Figure 6 This is an example diagram of the structure of a multi-cascade network where each level of the neural network shares the same shallow network structure;

[0034] Figure 7 Schematic diagram of the process of liveness detection using a two-stage cascade network;

[0035] Figure 8 It is a schematic diagram of a silent face liveness detection device of the present invention;

[0036] Fig. 9 It is a schematic diagram of a training module of the silent face liveness detection device of the present invention. DETAILED DESCRIPTION

[0037] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0038] Embodiment 1:

[0039] The embodiment of the present invention provides a method for detecting silent liveness of a face. Figure 1 As shown, the Figure 1 The flowchart of the silent liveness detection method of the face of the present invention comprises:

[0040] S100: Acquire a face image to be detected and perform preprocessing.

[0041] The silent liveness detection method for faces of the present invention can be used for near-infrared face images as well as for visible light face images. When a near-infrared device is used to collect images, a near-infrared face image is obtained, wherein the near-infrared device can include a group of near-infrared lamps with a central band of 850nm or 940nm and a lens with an additional filter, wherein the filter only passes near-infrared light in the 850nm or 940nm band.

[0042] In this step, the purpose of preprocessing the face image to be detected is to obtain an aligned and uniformly scaled face image. Preprocessing the face image to be detected may include face detection, key point location and normalization. Among them, the face detection method can be any one of SSD, CRAFT, and ADABOOST, the key point location method can be any one of TDCNN or SDM, and finally the bilinear interpolation normalization method can be used to align the acquired face key points to the same position.

[0043] S200: Input the preprocessed face image into a pre-trained multi-cascade network to determine whether the face image is alive.

[0044] Among them, the multi-cascade network includes a multi-level neural network cascaded front and back, and each level of the neural network is used for liveness judgment of a prosthesis attack type. When the current level neural network judges that it is live, it enters the next level neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live.

[0045] In daily life and work, the non-living sources commonly used to attack face recognition systems are mainly the following:

[0046] 1) Screen display: facial images and videos displayed on screen devices such as mobile phones, pads and notebooks.

[0047] 2) Two-dimensional printing: photos printed on two-dimensional materials, including printing paper, photo paper, coated paper, etc. The photos can be in color or black and white, and the image shapes can be complete face images or face images without eyes, nose or mouth areas.

[0048] 3) Three-dimensional masks: generally 3D masks, ordinary ones are made of plastic, and high-quality imitations are made of materials similar to human skin.

[0049] Among them, two-dimensional printing types are easy to identify through the Internet, while prosthetic attacks such as screen display types or three-dimensional masks are more difficult to identify.

[0050] For the above three types of prosthetic attacks, each prosthetic type can correspond to a first-level neural network. It is also possible to merge two types of prosthetic attack types into one, and the merged type of prosthetic attack type corresponds to a first-level neural network. When merging, it can be based on the difficulty of living body judgment or the similarity of attack methods. Those with similar difficulty of living body judgment and similar attack methods will be merged first. The above three types of prosthetic attacks include multiple situations, and one or several of them can be split out as a separate new prosthetic attack type. When splitting, it can also be based on the difficulty of living body judgment or the similarity of attack methods.

[0051] The prosthetic attack types of the present invention are not limited to the three types exemplified above, nor are they limited to their combination and splitting methods. Other prosthetic attack types not listed are also protected by the present invention.

[0052] The silent face liveness detection task is a binary classification task, which is divided into live and non-live (prosthesis). The present invention decomposes the silent face liveness detection task in the form of a neural network cascade with the idea of ​​divide and conquer: the classification of real faces and faces of many prosthetic attack types is decomposed into a binary classification task of multiple single prosthetic attack type faces (or several similar faces merged together) and real faces. Each level of the neural network in the multi-cascade network should make a prosthetic face attack judgment of one of the prosthetic attack types. After the face image is input into the multi-cascade network, each level of the neural network successively performs liveness judgment in a progressive manner. Only when the current level network judges it as live can it enter the next level network, otherwise it directly returns to the prosthesis and ends. The face image can only be considered alive when all networks judge it as live.

[0053] The present invention has the following beneficial effects:

[0054] The present invention uses a multi-level neural network to judge different types of prosthetic attacks. Compared with the existing method of using a whole complex network for live body judgment, the present invention can not only divide and conquer through cascading, but also be flexibly deployed. In addition, each level of neural network is a shallow network with few network parameters and less computing resources required for training. Once a certain level of network judges it as a prosthesis, no subsequent judgment is required. The recognition processing speed is fast and it is easy to deploy on smart terminals and embedded devices.

[0055] For new prosthetic attack types, it is convenient and quick to train a new shallow network with the data of the prosthetic attack type, and cascade it on the existing cascade network without retraining the entire network, thus improving stability. Or if the new prosthetic attack type can be merged into the existing type, it is only necessary to fine-tune the network corresponding to the existing type without retraining the entire network.

[0056] Therefore, the present invention adopts a deep learning method and uses multiple shallow network cascades to perform silent face liveness detection, which improves the accuracy and efficiency of liveness detection and greatly improves the stability of the algorithm.

[0057] In the present invention, the neural networks at various levels are cascaded in order from easy to difficult according to the difficulty of liveness judgment for each prosthetic attack type.

[0058] The present invention uses the idea of ​​divide and conquer to decompose the silent face liveness detection task from easy to difficult. The difficulty can also be divided according to the different types of prosthetic attacks. Generally, two-dimensional printing attacks are easy to identify, while screen display attacks and three-dimensional mask attacks are more difficult to identify. Each hierarchical network is designed to process a single attack type of data, and finally cascaded from easy to difficult according to the difficulty of liveness judgment.

[0059] Each level of the neural network in the present invention is a shallow network, but although they are shallow networks, the network structures are also different. The type of prosthetic attack that is easy to identify requires a relatively simple neural network structure, so it is placed at the front end of the cascade network. The network at the back end of the cascade is used for more difficult liveness detection tasks. Compared with the complex network at the front end of the cascade, the network is deeper and the network input resolution is also larger.

[0060] Since the two-dimensional printing class is easier to distinguish than the screen display class and the three-dimensional mask class, for example, a smaller network can be designed and trained to distinguish the two-dimensional printing class attacks, and it can be placed in the first level of the cascade network for rapid identification; a network that is more complex than the first level network can be designed and trained to distinguish screen display class or three-dimensional mask class attacks.

[0061] The cascading method from easy to difficult of the present invention has the following advantages: the cascaded front-end network is lighter than the cascaded back-end network, and can quickly screen easily identified prosthetic faces, while the back-end uses a larger input and a deeper network to identify prosthetic faces that are more difficult to identify.

[0062] The present invention maximizes the speed and accuracy of liveness detection in the cascade network: the front-end network in the cascade network uses smaller-scale input and a shallower network to achieve fast classification of easily distinguishable prosthetic faces; the cascade back-end network can use large-resolution images containing more detailed information, and cooperate with a deeper network structure to mine more and better features for distinguishing real and fake faces.

[0063] When a new type of fake attack occurs, the specific processing method is as follows:

[0064] If the new prosthetic attack type can be merged into the existing prosthetic attack type, the neural network corresponding to the merged prosthetic attack type is retrained or fine-tuned.

[0065] If the new prosthetic attack type cannot be merged into the existing prosthetic attack type, a new neural network is designed for liveness judgment of the new prosthetic attack type, and the new neural network is inserted into the target position of the multi-cascade network according to the difficulty of liveness judgment of the new prosthetic attack type.

[0066] The present invention does not limit the specific judgment method when judging whether a new prosthesis attack type can be merged into an existing prosthesis attack type. For example, it can judge whether to merge based on whether the attack means of the new prosthesis attack type are similar to those of the existing prosthesis attack type, or judge whether to merge based on whether the living body judgment difficulty of the new prosthesis attack type and the existing prosthesis attack type are close. If the attack means are similar or the difficulty is close, then merge is performed; otherwise, merge is not performed.

[0067] The input of the neural network is a normalized face based on facial feature points. In actual application scenarios, due to the influence of lighting, posture, etc., the positioning of feature points is often not very accurate, which affects the stability of the model's living and non-living classification.

[0068] To solve the above problems, each level of the neural network in the present invention is a fully convolutional network; during training, the training samples of the multi-cascade network include facial information and background information around the face. When the multi-cascade network is trained, the training samples are randomly cropped (Crop) and then input into the multi-cascade network for training.

[0069] In offline learning tasks, the present invention increases the robustness of the network by means of random cropping, and alleviates the influence of illumination and posture to a certain extent.

[0070] Although random cropping can increase robustness to a certain extent, the problem of illumination and posture images is still prominent when facing real-time liveness detection tasks. To further solve this problem, the preprocessing of the present invention includes:

[0071] Performing face detection, facial feature point positioning and face normalization on the face image to obtain a preprocessed face image, wherein the preprocessed face image includes face information and background information of the surrounding part of the face.

[0072] The size of the preprocessed face image may be different from or the same as the size of the training sample, and the present invention does not limit this. Preferably, the size of the preprocessed face image is larger than the size of the training sample after random cropping.

[0073] In S200 of the present invention, after the preprocessed face image is input into a pre-trained multi-cascade network, a group of sub-images are obtained from the preprocessed face image by means of a sliding window. All sub-images are processed by a certain level of neural network to obtain a group of liveness scores. The group of liveness scores are statistically analyzed to obtain the liveness judgment result of the neural network at that level.

[0074] Specifically, assuming that the resolution of the face frame after face normalization is M*M, the face frame is expanded up, down, left, and right to include background information, with a resolution of N*N, which is the size of the preprocessed face image. During training, assuming that the size of the training sample is also N*N, random cropping is performed online on the training sample with N*N resolution to obtain a training sample sub-block of M*M size for training.

[0075] All the cascaded neural networks of the present invention use a fully convolutional network. During liveness detection, a preprocessed face image with an N*N resolution is used as input. A set of liveness scores is obtained through the fully convolutional network. The set of scores is statistically determined to determine whether it is a live face. The specific implementation method of the statistics can be voting, etc., which is not limited by the present invention.

[0076] The present invention uses a fully convolutional network, performs random cropping on training samples, and performs sliding window selection of sub-images on pre-processed face images, which has the following beneficial effects:

[0077] 1. The network is trained in a random crop manner to crop the training samples, making the trained network more robust.

[0078] 2. The preprocessed face image of the present invention covers part of the background area around the face, which can help to more easily identify the fake attack. The specific principle is as follows:

[0079] The input of the liveness detection network relies on face calibration, but this calibration cannot be accurate at all times. At the same time, due to occlusion and posture problems, a standard frontal face cannot always be obtained. These deviations easily lead to unstable liveness scores. For this reason, the present invention inputs a face image with a higher resolution than that during training during liveness detection, and obtains a set of liveness scores on the preprocessed face image containing the background in a form similar to the face detection sliding window, which is more suitable for the situation in which face calibration is inaccurate due to objective factors (lighting, posture, etc.) in actual scenes, so as to greatly reduce the interference of unstable factors such as lighting and posture, and alleviate the instability of the algorithm in distinguishing true and false faces caused by interference from lighting and posture. At the same time, the relationship between background and face is analyzed, which is more conducive to prosthesis discrimination.

[0080] 3. The images (randomly cropped training samples and preprocessed face images) input to the network during training and detection are of different sizes. The present invention uses a fully convolutional network, which can accept images of any resolution to adapt to image inputs of different resolutions, making the input more flexible. In addition, the fully convolutional network can reduce the parameters of the fully connected layer and accelerate the speed of liveness detection.

[0081] 4. The present invention uses a form similar to the sliding window of face detection to count a group of liveness scores and uses the statistical value to determine the final result. Compared with a single score, the determination result is more convincing.

[0082] In the cascade form, the present invention involves two ideas. One is that each level of the neural network in the multi-cascade network is independent of each other. The other is that each level of the neural network shares the same shallow network structure, and then conducts its own branch network based on the shared shallow network structure, and the cascade back-end network designs a larger and deeper branch network structure.

[0083] like Figure 2 As shown, the multi-cascade network is trained by the following method:

[0084] S100': construct a data set for each level of the neural network, and the prosthetic attack type of the negative samples in the data set of each level of the neural network is the prosthetic attack type corresponding to the neural network.

[0085] Assume that the multi-cascade network includes two levels, net1 and net2. The negative samples (i.e., prosthetic data) in the training samples are divided into two categories according to the type of attack or the difficulty of the attack method. Data sets are constructed for net1 and net2, denoted as Data1 and Data2, where Data1 is a relatively easy-to-discriminate data set, and Data2 is a more difficult-to-discriminate data set. For example, data1 can be a data set formed by two-dimensional printed prosthetic attack images, and Data2 can be a data set formed by screen display or three-dimensional mask prosthetic attack images.

[0086] S200': If each level of the neural network is independent, then each level of the neural network is trained separately, and the training data of each level of the neural network is the collection of data sets of the neural network and the neural network before it.

[0087] Specifically, when training net1, use Data1, and when training net2, use Data1+Data2.

[0088] S300': If each level of the neural network shares the same shallow network structure, the data set of the first level neural network is used to train the entire multi-cascade network, the parameters of the shared shallow network structure are frozen, and then the non-shared parts of each level of the neural network except the first level neural network are fine-tuned. The training data for fine-tuning the non-shared parts of each level of the neural network is the collection of data sets of the neural network and the neural network before it.

[0089] Specifically, first use Data1 to train the entire cascade network, then freeze some of the common parameters of the network, and use Data1+Data2 to fine-tune the network branch of Data2.

[0090] The present invention classifies the training data according to different types of prosthesis attacks, and then trains cascade networks for different data sources, thereby improving the accuracy, robustness and detection speed of silent face liveness detection.

[0091] In the present invention, each level of the cascaded neural network can select network inputs of the same size, or in order to improve speed performance, a relatively small network input can be used in the front-stage neural network, and a larger input can be selected in the cascaded back-end neural network.

[0092] Since deeper networks have more superior performance, the reasoning speed is significantly reduced. In the present invention, each level of neural network shares the same shallow network structure, and then branches out to extract features from each network. The branch network at the cascade back end has a more complex structure than the branch network at the cascade front end. The feature map (FeatureMap) of the shared part of the cascade front end network can be used as the input of the cascade back end network to reduce the amount of calculation of the cascade back end network. Compared with the way that each level of network is completely independent, this way can make the structure of the cascade back end network more complex and deeper.

[0093] The present invention is described in detail below with a specific implementation example. For the convenience of description, the cascade network designed in this example is a two-stage cascade of net1 and net2. Of course, in addition to the two-stage neural network, the multi-cascade network can also use a neural network of more than three levels. The principles and methods of classification detection are the same as those of the two-stage neural network, and no further examples are given.

[0094] (I) Face image preprocessing

[0095] Face image preprocessing includes three steps: face detection, face feature point location and face normalization.

[0096] This example uses the MTCNN detection algorithm to achieve face detection and facial feature point positioning. There are 5 facial feature points in total, left eye, right eye, nose, left corner of mouth, and right corner of mouth. It should be noted that the face detection and facial feature point positioning methods are not limited to the MTCNN algorithm selected in this example. The face detection method can also be Harr-AdaBoost, SSD, Faster RCNN, CenterNet, etc., and the facial feature point positioning method can also be SDM, LBF, LAB, etc.

[0097] Face normalization includes face alignment and face scaling. The normalized size is 112*112, and the coordinates of both eyes are aligned to (38,52) and (74,52). The specific process is as follows: Figure 3 During training, this example uses a 96*96 image as input, translates 112*112 face samples by 8 pixels, and generates 9 96*96 images for training.

[0098] During liveness detection, a 112*112 preprocessed face image is input into the cascade neural network for inference. Since the network is a fully convolutional network (the last layer is a convolutional layer, so that the network can be applied to image inputs of different sizes), a 3*3 score is generated in a sliding window manner for voting and counting liveness scores, thereby alleviating the instability of liveness detection caused by positioning deviation.

[0099] (II) Face liveness detection

[0100] The network design of the present invention uses two methods. One is that each level of neural network is independent of each other. Figure 4 , Figure 5 , and secondly, each level of neural network shares lower level parameters, see Figure 6 .

[0101] During the training phase, the negative samples (i.e., fake data) in the training samples are divided into two categories according to the type of attack or the difficulty of the attack method, and data sets are constructed for net1 and net2, denoted as Data1Data2, where Data1 is a relatively easy-to-discriminate data set and Data2 is a relatively discriminative data set.

[0102] When net1 and net2 are independent of each other, when training net1, use Data1 for training, and when training net2, use Data1+Data2 for training.

[0103] When net1 and net2 share lower-level parameters, first use Data1 to train the entire network, then freeze the common parameters of the network, and use Data1+Data2 to fine-tune the network branch of net2.

[0104] In this application, the training method for the neural network in the training phase is exemplified as follows:

[0105] 1. First, initialize the network parameters. In this step, you can customize the initialization parameters of the convolutional layer, or you can use the model parameters trained by the classic network model. For example, use the model parameters trained by VGG on the Image Net image set to initialize the corresponding convolutional layer parameters.

[0106] 2. Secondly, train the network parameters. In this step, the deep learning framework MatConvNet can be used for training, and the stochastic gradient descent method can be used.

[0107] 3. The method to freeze some common parameters of the network is: when using Data1 to train the entire network, do not return the gradient data, and you can freeze some common parameters of the network.

[0108] During liveness detection, due to the use of a fully convolutional network and a 112*112 input, 9 scores can be obtained through a sliding window. The final true or false liveness discrimination is determined by voting statistics.

[0109] The identification process is as follows Figure 7 As shown, the first-level network obtains 9 scores S1 i , the second level network gets 9 scores S2 i , i=1,2,3,…,8. Figure 7 T1, T2, TN1, and TN2 are the liveness score thresholds and voting thresholds of the first-level network and the second-level network, respectively. These thresholds can be flexibly adjusted according to the actual scene data. The first-level network will enter the second-level network only when it is judged to be live, otherwise it will directly return to the prosthesis.

[0110] Among them, the method for the first-level network to determine the live body is: count the S1 that is less than T1 i and N n1 , if N n1 If it is greater than TN1, it is judged as a prosthesis, otherwise it enters the second-level network. In the second-level network, the S2 less than T2 is counted. i and N n2 , if N n2 If it is greater than TN2, it is judged as a prosthesis, otherwise it is judged as a living organism.

[0111] The present invention uses a multi-cascade network to perform liveness detection on the acquired face image to be tested, which can judge known types of prosthetic attacks. The front-end network of the multi-cascade network is relatively lightweight and can quickly screen out prosthetic faces that are easy to identify. The back-end network is relatively deep and can identify prosthetic faces that are difficult to identify. This hierarchical progressive method decomposes the silent liveness detection task of the face from easy to difficult according to the different types of prosthetic attacks. When a new type of prosthetic attack appears, the trained network can be easily and quickly added to the already trained cascade network. At the same time, through the cascade network + voting method, liveness detection can be performed more accurately.

[0112] Embodiment 2:

[0113] Embodiment 2 of the present invention provides a silent face liveness detection device, such as Figure 8 As shown, the device comprises:

[0114] The acquisition module 100 is used to acquire the face image to be detected and perform preprocessing.

[0115] The detection module 200 is used to input the pre-processed face image into a pre-trained multi-cascade network to determine whether the face image is alive.

[0116] Among them, the multi-cascade network includes a multi-level neural network cascaded front and back, and each level of the neural network is used for liveness judgment of a prosthesis attack type. When the current level neural network judges that it is live, it enters the next level neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live.

[0117] The present invention adopts a deep learning method and uses a cascade of multiple shallow networks to perform silent face liveness detection, which improves the accuracy and efficiency of liveness detection and greatly enhances the stability of the algorithm.

[0118] The present invention cascades neural networks of various levels in order from easy to difficult according to the difficulty of live body judgment for each prosthetic attack type.

[0119] When a new prosthetic attack type appears, if the new prosthetic attack type can be merged into the existing prosthetic attack type, the neural network corresponding to the merged prosthetic attack type is retrained or fine-tuned; if the new prosthetic attack type cannot be merged into the existing prosthetic attack type, a new neural network is designed for liveness judgment of the new prosthetic attack type, and the new neural network is inserted into the target position of the multi-cascade network according to the difficulty of liveness judgment of the new prosthetic attack type.

[0120] Each level of the aforementioned neural network is a fully convolutional network;

[0121] The training samples of the multi-cascade network include face information and background information around the face. When the multi-cascade network is trained, the training samples are randomly cropped and then input into the multi-cascade network for training.

[0122] The pre-processing comprises:

[0123] Performing face detection, facial feature point positioning and face normalization on the face image to obtain a preprocessed face image, wherein the preprocessed face image includes face information and background information of the surrounding part of the face.

[0124] After the preprocessed face image is input into a pre-trained multi-cascade network, a group of sub-images are obtained from the preprocessed face image through a sliding window. All sub-images are processed by a certain level of neural network to obtain a group of liveness scores. The group of liveness scores are statistically analyzed to obtain the liveness judgment result of the neural network at this level.

[0125] In the present invention, each level of neural network in the multi-cascade network is independent of each other, or each level of neural network shares the same shallow network structure.

[0126] The multi-cascade network is implemented as follows Fig. 9 The module shown is trained to obtain:

[0127] The construction module 100' is used to construct a data set for each level of the neural network, and the pseudo attack type of the negative sample in the data set of each level of the neural network is the pseudo attack type corresponding to the neural network.

[0128] The first training module 200' is used to train each level of the neural network separately if each level of the neural network is independent, and the training data of each level of the neural network is the collection of data sets of the neural network and the neural network before it.

[0129] The second training module 300' is used to train the entire multi-cascade network using the data set of the first-level neural network if each level of the neural network shares the same shallow network structure, freeze the parameters of the shared shallow network structure, and then fine-tune the non-shared parts of each level of the neural network except the first-level neural network. The training data for fine-tuning the non-shared parts of each level of the neural network is the collection of data sets of the neural network and the neural network before it.

[0130] The prosthetic attack types include screen display type, two-dimensional printing type and three-dimensional mask type.

[0131] The present invention uses a multi-cascade network to perform liveness detection on the acquired face image to be tested, which can judge known types of prosthetic attacks. The front-end network of the multi-cascade network is relatively lightweight and can quickly screen out prosthetic faces that are easy to identify. The back-end network is relatively deep and can identify prosthetic faces that are difficult to identify. This hierarchical progressive method decomposes the silent liveness detection task of the face from easy to difficult according to the different types of prosthetic attacks. When a new type of prosthetic attack appears, the trained network can be easily and quickly added to the already trained cascade network. At the same time, through the cascade network + voting method, liveness detection can be performed more accurately.

[0132] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment 1. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment 1. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment 1, and will not be repeated here.

[0133] Embodiment 3:

[0134] The method described in the above embodiment 1 provided by the present invention can implement the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in embodiment 1 of this specification. Therefore, the present invention also provides a computer-readable storage medium for silent face liveness detection, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the silent face liveness detection method including embodiment 1 are implemented.

[0135] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0136] The present invention uses a multi-cascade network to perform liveness detection on the acquired face image to be tested, which can judge known types of prosthetic attacks. The front-end network of the multi-cascade network is relatively lightweight and can quickly screen out prosthetic faces that are easy to identify. The back-end network is relatively deep and can identify prosthetic faces that are difficult to identify. This hierarchical progressive method decomposes the silent liveness detection task of the face from easy to difficult according to the different types of prosthetic attacks. When a new type of prosthetic attack appears, the trained network can be easily and quickly added to the already trained cascade network. At the same time, through the cascade network + voting method, liveness detection can be performed more accurately.

[0137] The storage medium described above may also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further details will be given here.

[0138] Embodiment 4:

[0139] The present invention also provides a device for silent face liveness detection, which may be a separate computer, or may include an actual operating device using one or more methods or one or more embodiments of the present specification. The device for silent face liveness detection may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the silent face liveness detection method described in any one or more of Embodiment 1 are implemented.

[0140] The present invention uses a multi-cascade network to perform liveness detection on the acquired face image to be tested, which can judge known types of prosthetic attacks. The front-end network of the multi-cascade network is relatively lightweight and can quickly screen out prosthetic faces that are easy to identify. The back-end network is relatively deep and can identify prosthetic faces that are difficult to identify. This hierarchical progressive method decomposes the silent liveness detection task of the face from easy to difficult according to the different types of prosthetic attacks. When a new type of prosthetic attack appears, the trained network can be easily and quickly added to the already trained cascade network. At the same time, through the cascade network + voting method, liveness detection can be performed more accurately.

[0141] The above-mentioned device can also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further description will be given here.

[0142] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for silent face liveness detection, characterized in that: The method comprises: Obtain the face image to be detected and perform preprocessing; The preprocessed face image is input into a pre-trained multi-cascade network to determine whether the face image is alive; The multi-cascade network includes a multi-level neural network cascaded forward and backward. Each level of the neural network is used for liveness judgment of a type of prosthetic attack. When the current level of neural network judges that it is live, it enters the next level of neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live. According to the difficulty of liveness judgment for each prosthetic attack type, the neural networks at all levels are cascaded in order from easy to difficult; When a new prosthetic attack type appears, if the new prosthetic attack type can be merged into the existing prosthetic attack type, the neural network corresponding to the merged prosthetic attack type is retrained or fine-tuned; if the new prosthetic attack type cannot be merged into the existing prosthetic attack type, a new neural network is designed for liveness judgment of the new prosthetic attack type, and the new neural network is inserted into the target position of the multi-cascade network according to the difficulty of liveness judgment of the new prosthetic attack type.

2. The method for silent face liveness detection according to claim 1, characterized in that: Each level of neural network is a fully convolutional network; The training samples of the multi-cascade network include face information and background information around the face. When the multi-cascade network is trained, the training samples are randomly cut and then input into the multi-cascade network for training; The pre-processing comprises: Performing face detection, facial feature point location and face normalization on the face image to obtain a preprocessed face image, wherein the preprocessed face image includes face information and background information of the surrounding part of the face; After the preprocessed face image is input into a pre-trained multi-cascade network, a group of sub-images are obtained from the preprocessed face image through a sliding window. All sub-images are processed by a certain level of neural network to obtain a group of liveness scores. The group of liveness scores are statistically analyzed to obtain the liveness judgment result of the neural network at this level.

3. The method for silent face liveness detection according to claim 2, characterized in that: Each level of neural network in the multi-cascade network is independent of each other, or each level of neural network shares the same shallow network structure; The multi-cascade network is trained by the following method: Construct a data set for each level of neural network, and the pseudo attack type of the negative sample in the data set of each level of neural network is the pseudo attack type corresponding to the neural network; If each level of neural network is independent, each level of neural network is trained separately, and the training data of each level of neural network is the collection of data sets of the neural network and the neural network before it; If each level of neural network shares the same shallow network structure, the data set of the first-level neural network is used to train the entire multi-cascade network, the parameters of the shared shallow network structure are frozen, and then the non-shared parts of each level of neural network except the first-level neural network are fine-tuned. The training data for fine-tuning the non-shared parts of each level of neural network is the collection of data sets of the neural network and the neural network before it.

4. The method for silent face liveness detection according to claim 3, characterized in that: The prosthetic attack types include screen display type, two-dimensional printing type and three-dimensional mask type.

5. A silent face liveness detection device, characterized in that: The device comprises: An acquisition module is used to acquire the face image to be detected and perform preprocessing; A detection module is used to input the pre-processed face image into a pre-trained multi-cascade network to determine whether the face image is alive; The multi-cascade network includes a multi-level neural network cascaded forward and backward. Each level of the neural network is used for liveness judgment of a type of prosthetic attack. When the current level of neural network judges that it is live, it enters the next level of neural network. When any level of neural network judges that it is a prosthesis, the face image is judged to be a prosthesis. When the last level of neural network judges that it is live, the face image is judged to be live. According to the difficulty of liveness judgment for each prosthetic attack type, the neural networks at all levels are cascaded in order from easy to difficult; When a new prosthetic attack type appears, if the new prosthetic attack type can be merged into the existing prosthetic attack type, the neural network corresponding to the merged prosthetic attack type is retrained or fine-tuned; if the new prosthetic attack type cannot be merged into the existing prosthetic attack type, a new neural network is designed for liveness judgment of the new prosthetic attack type, and the new neural network is inserted into the target position of the multi-cascade network according to the difficulty of liveness judgment of the new prosthetic attack type.

6. The silent face liveness detection device according to claim 5, characterized in that: Each level of neural network is a fully convolutional network; The training samples of the multi-cascade network include face information and background information around the face. When the multi-cascade network is trained, the training samples are randomly cut and then input into the multi-cascade network for training; The pre-processing comprises: Performing face detection, facial feature point location and face normalization on the face image to obtain a preprocessed face image, wherein the preprocessed face image includes face information and background information of the surrounding part of the face; After the preprocessed face image is input into a pre-trained multi-cascade network, a group of sub-images are obtained from the preprocessed face image through a sliding window. All sub-images are processed by a certain level of neural network to obtain a group of liveness scores. The group of liveness scores are statistically analyzed to obtain the liveness judgment result of the neural network at this level.

7. A computer-readable storage medium for silent face liveness detection, characterized in that: It comprises a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the silent face liveness detection method according to any one of claims 1 to 4 are implemented.

8. A device for silent face liveness detection, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the method for silent face liveness detection described in any one of claims 1 to 4 are implemented.

Images