A non-cooperative face liveness detection method and computer storage medium

Abstract

The present invention provides a non-cooperative face liveness detection method and a computer storage medium, the method comprising the following steps: S1, obtaining a liveness detection image set for model training, and processing the image set; S2, designing an Inception Net classification network; S3, designing an SSD detection network; S4, performing model training on the image set through the Inception Net classification network and the SSD detection network; S5, obtaining a real-time image, and predicting whether the real-time image is a live face according to the result of the model training. The non-cooperative face liveness detection method according to an embodiment of the present invention does not require the addition of hardware equipment other than a camera, and does not require the cooperation of the person to be tested, can achieve ultra-real-time face liveness detection speed, is low-cost, and is easy to use.

Description

Technical Field

[0001] The present invention relates to the field of face detection, and more specifically, to a non-cooperative face liveness detection method and a computer storage medium. Background Art

[0002] Face recognition has gradually become an important encryption and decryption method due to its speed, effectiveness and user-friendliness. However, many face recognition systems are currently unable to distinguish the authenticity of faces. Therefore, introducing liveness detection methods into face recognition systems can help improve the practicality and security of face recognition. Currently, there are mainly the following methods for face liveness detection:

[0003] 1) Active liveness detection based on interactive video streams. Main technical means: First, the system detects faces and locates key points of faces. If there is a face in the video, it randomly generates several actions. If the tester completes the specified action within the specified time, the system determines that the tester is alive, otherwise it is determined to be non-live. The problems and defects of this method are: it requires user cooperation and takes a long time.

[0004] 2) Face liveness detection method based on bright pupil effect. Main technical means: distinguish live faces from non-live faces by detecting whether there is a bright pupil effect in the eye area of ​​the face. The problems and defects of this method are: it requires additional light source equipment, is expensive, and is cumbersome to operate.

[0005] 3) Face liveness detection method with image distortion analysis. Main technical means: First, the system performs face detection and facial key point positioning. If there is a face in the image, four features in the image (mirror reflection feature, blur feature, moment feature, color diversity feature) are extracted, and support vector machine is used for training and prediction. The existing problems and defects are: the features extracted by this method are relatively simple, the discrimination ability is not strong and the generalization ability is not strong, and it cannot be well applied in real scenes.

[0006] Therefore, based on the existing technology, the above problems still need to be solved, and it is urgent to develop a new type of non-cooperative face liveness detection method. Summary of the invention

[0007] In view of this, the present invention provides a non-cooperative human face liveness detection method and a computer storage medium, which can effectively improve the accuracy and speed of human face liveness detection.

[0008] To solve the above technical problems, on the one hand, the present invention provides a non-cooperative human face liveness detection method, the method comprising the following steps: S1, obtaining a liveness detection image set for model training, and processing the image set; S2, designing an Inception Net classification network; S3, designing an SSD detection network; S4, performing model training on the image set through the Inception Net classification network and the SSD detection network; S5, obtaining a real-time image, and predicting whether the real-time image is a live human face according to the result of the model training.

[0009] According to the non-cooperative face liveness detection method of the embodiment of the present invention, the accuracy and speed of face liveness detection are greatly improved by using the Inception Net classification network in conjunction with the SSD detection network. The SSD detection network can adapt to the training and detection tasks of electronic device frames of various scales, and can detect targets in real time with high precision. The liveness detection method used in the present invention does not require the addition of hardware equipment other than the camera, and does not require the cooperation of the person to be tested, can achieve ultra-real-time face liveness detection speed, is low-cost, and is easy to use.

[0010] According to some embodiments of the present invention, step S1 includes: S11, batch saving a living body detection image set Q through a camera device; S12, annotating borders of images of a first set in the image set Q, and obtaining images G with border annotation information and images P without annotation information, wherein the first set of images is a subset of the image set Q.

[0011] According to some embodiments of the present invention, in step S2, the designed Inception Net classification network is denoted as M, and the Inception Net classification network includes a deep neural network A and 3 groups of Inception structures C. The Inception structure C includes four branches, and each of the branches is composed of 1*1 convolution, 3*3 convolution, 5*5 convolution, and 3*3 maximum pooling.

[0012] According to some embodiments of the present invention, in step S3, the designed SSD detection network is denoted as S, and the main network of the SSD detection network is a VGG16 model with 4 convolutional layers added to the convolutional layer.

[0013] According to some embodiments of the present invention, step S3 also includes: convolving the outputs of the five different convolutional layers with two 3*3 convolution kernels, respectively, wherein the output of one convolution kernel is the classification confidence, and each default box generates three confidences, corresponding to the background, real person, and non-living face in the living body detection task, respectively; the output of the other convolution kernel is the positioning of the target position, and each default box generates four coordinate values ​​of x, y, w, and h; the five convolutional layers generate a default box through the priorBox layer; and the above three calculation results are merged and passed to the loss layer.

[0014] According to some embodiments of the present invention, step S4 comprises:

[0015] S41, dividing the image set Q into a training set T and a verification set V, so that both the training set T and the verification set V contain images G with border annotation information and images P without annotation information;

[0016] S42, forward calculation is performed on some face images P in the training set T through the Inception Net classification network M, and the recognition results are output after passing through each layer of the model. The batch stochastic gradient descent algorithm is applied to perform model training according to the label difference between the current network output and the input feature, and each weight in the Inception Net classification network M is continuously adjusted;

[0017] S43, using the image P without annotation information in the verification set V to verify the model training effect, when the accuracy of the model M on the verification set V cannot continue to improve with the training time, the model training stops, and the model M' is obtained;

[0018] S44, forward calculation is performed on some images G with border annotation information in the training set T through the SSD detection network S, and after passing through each layer of the model, the target border detected by the model and the category corresponding to each border are output;

[0019] S45. Compare the current network output result with the target’s labeled information and calculate the positioning loss L loc With confidence loss L conf ; The overall objective loss function of the network L(x, c, l, g) is L loc With L conf The weighted sum of is shown in formula (1):

[0020]

[0021] Where x represents the matching of the prior box and the annotation information; c is the confidence of the predicted box output by the model, l is the predicted box of the model, g is the position of the annotated box, a is a parameter used to adjust the ratio between the confidence loss and the positioning loss, and N is the number of predicted boxes that match the annotated box information; the batch stochastic gradient descent method algorithm is used to train the model of the objective function in formula (1), and the weights in the SSD detection network S are continuously adjusted;

[0022] S46, using the image G with border annotation information in the verification set V to verify the model training effect, when the accuracy of the model S on the verification set V cannot continue to improve with the training time, the model training stops and the model S' is obtained;

[0023] S47. After the image passes through M' and S', the confidence scores of the real person category input into the two models are added together and recorded as the confidence score of model D.

[0024] According to some embodiments of the present invention, step S5 includes: S51, obtaining a real-time RGB image A through a camera device, inputting image A into a cascaded convolutional neural network model to perform face detection; if a face is detected, normalizing the face to obtain a normalized face image B; and recording the position k of the face in image A; S52, inputting image A into the model M' obtained in step S4 to obtain a result x; S53, inputting image B into the model S' obtained in step S4 to obtain a border set Y; S54, judging the overlap rate between the face position k and each border in the border set Y; if there is a border whose overlap rate exceeds a set value, recording the confidence u of the border; among all u that meet the conditions, selecting the maximum value v; S55, inputting the result x and the maximum confidence value v into the fusion model D obtained in step S4, and giving the face liveness detection prediction result of image A through the fusion model D.

[0025] According to some embodiments of the present invention, in step S51, the step of normalizing the face is: inputting the face area image in image A into the face key point detection model to locate the face key points, and transforming the face images in different postures into face images in standard postures by calculating the affine transformation from the key points to the standard key points, so as to obtain a normalized face image B.

[0026] According to some embodiments of the present invention, in step S53, overlapping or erroneous predicted borders are removed by a non-maximum suppression algorithm to obtain a border set W, and subsequent operations are performed using the border set W.

[0027] In a second aspect, an embodiment of the present invention provides a computer storage medium, comprising one or more computer instructions, wherein the one or more computer instructions implement the method described in the above embodiment when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a general flow chart of the non-cooperative face liveness detection method according to an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of the composition structure of the Inception structure C in the non-cooperative face liveness detection method according to an embodiment of the present invention;

[0030] Figure 3 It is a flowchart of real-time prediction in the non-cooperative human face liveness detection method according to an embodiment of the present invention;

[0031] Figure 4 Schematic diagram of an electronic device according to an embodiment of the present invention.

[0032] Reference numerals:

[0033] Non-cooperative face liveness detection method 100;

[0034] Electronic device 300;

[0035] Memory 310; operating system 311; application program 312;

[0036] Processor 320 ; network interface 330 ; input device 340 ; hard disk 350 ; display device 360 ​​. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention will be further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0038] The following first describes in detail the non-cooperative face liveness detection method 100 according to an embodiment of the present invention with reference to the accompanying drawings.

[0039] like Figure 1 As shown, the non-cooperative face liveness detection method according to an embodiment of the present invention comprises the following steps:

[0040] S1. Obtain a set of liveness detection images for model training, and process the set of images.

[0041] S2. Design the Inception Net classification network.

[0042] S3. Design SSD detection network.

[0043] S4. Performing model training on the image set through the Inception Net classification network and the SSD detection network.

[0044] S5. Acquire a real-time image, and predict whether the real-time image is a living human face according to the model training result.

[0045] In other words, the non-cooperative face liveness detection method according to an embodiment of the present invention first collects a set of liveness detection images for model training and marks these image sets before acquiring real-time images of faces. Then, an Inception Net classification network and an SSD detection network are designed, and model training is performed on the image sets through the Inception Net classification network and the SSD detection network. After the model training, real-time images are acquired, and a fusion model of the model training is used to predict and determine whether the real-time images are live faces.

[0046] This method uses a classifier that combines the SSD detection network with the face Inception net classification network to classify the authenticity of the face. The technical advantages of this method are: using the SSD detection network to detect the environmental information of the face, and using the face Inception net to classify the partial content of the face. The fusion of the two models has better liveness detection effect than a single classifier or detector.

[0047] Therefore, according to the non-cooperative face liveness detection method of the embodiment of the present invention, by cooperating the Inception Net classification network with the SSD detection network, the accuracy and speed of face liveness detection are greatly improved. The SSD detection network can adapt to the training and detection tasks of electronic device frames of various scales, and can detect targets in real time with high precision. The liveness detection method used in the present invention does not require the addition of hardware equipment other than the camera, and does not require the cooperation of the person to be tested, can achieve ultra-real-time face liveness detection speed, is low-cost, and is easy to use.

[0048] The following describes in detail the steps of the non-cooperative face liveness detection method according to an embodiment of the present invention.

[0049] In some specific embodiments of the present invention, step S1 includes:

[0050] S11. Batch save a set of living body detection images Q using a camera device.

[0051] S12: perform border annotation on images of a first set in the image set Q to obtain images G with border annotation information and images P without annotation information, wherein the first set of images is a subset of the image set Q.

[0052] Specifically, borders of some images in the image set Q may be manually annotated, wherein images with border annotation information may be denoted as G, and images without annotation information may be denoted as P.

[0053] Therefore, by classifying and marking a collection of liveness detection images saved in batches, the authenticity of the face can be classified.

[0054] According to one embodiment of the present invention, in step S2, the designed Inception Net classification network is denoted as M, and the Inception Net classification network includes a deep neural network A and 3 groups of Inception structures C. The Inception structure C includes four branches, and each of the branches is composed of 1*1 convolution, 3*3 convolution, 5*5 convolution, and 3*3 maximum pooling.

[0055] Specifically, the Inception Net network structure, also known as Google Net, is a deeper network structure than the VGG deep learning convolutional network, but with fewer parameters and higher computational efficiency. The newly designed Inception Net classification network is denoted as M, which includes a deep neural network A (8-layer convolutional neural network) and 3 groups of Inception structures C; the Inception structure C uses four branches, each of which is composed of 1*1 convolution, 3*3 convolution, 5*5 convolution, and 3*3 maximum pooling; the feature map of the previous layer generates four outputs after passing through the four branches, and the outputs of these four branches are superimposed in the channel dimension as the input of the next layer.

[0056] Adding multiple Inception structures C after the last convolutional layer of the deep neural network A not only increases the width of the network, but also increases the adaptability of the network to different scales. Therefore, compared with the deep neural network A, the Inception Net classification network M has a better classification effect of the live face classifier.

[0057] Generally speaking, the most direct way to improve the performance of a deep neural network A is to increase the depth and width of the network, but this also means a massive number of parameters, which can easily lead to overfitting and greatly increase the amount of calculation. Figure 2 As shown in the figure, the Inception structure C can perform multiple convolution operations or pooling operations on the input image in parallel and concatenate all the output results into a very deep feature map. Because different convolution operations and pooling operations such as 1*1, 3*3 or 5*5 can obtain different information of the input image, processing these operations in parallel and combining all the results will obtain a better image representation. Therefore, the newly designed Inception Net classification network can not only maintain the sparsity of the network structure, but also utilize the high computing performance of dense matrices, thereby improving the training effect.

[0058] Therefore, according to the non-cooperative face liveness detection method of the embodiment of the present invention, the face Inception net classification network adds an Inception structure on the basis of the traditional convolutional neural network. The technical advantage of this method is that it can greatly improve the classification effect of the face liveness classifier.

[0059] Optionally, in some specific embodiments of the present invention, in step S3, the designed SSD detection network is denoted as S, and the main network of the SSD detection network is a VGG16 model with 4 convolutional layers added to the convolutional layer.

[0060] Furthermore, step S3 further includes:

[0061] The outputs of the five different convolutional layers are convolved with two 3*3 convolution kernels, one of which outputs the classification confidence. Each default box generates three confidences, corresponding to the background, real person, and non-living face in the liveness detection task, respectively. The output of the other convolution kernel is the positioning of the target position, and each default box generates four coordinate values ​​of x, y, w, and h.

[0062] Five convolutional layers pass through the priorBox layer to generate the default box.

[0063] The above three calculation results are combined and passed to the loss layer.

[0064] In other words, the SSD (Single-Shot MultiBox Detector) detection network is an object detection algorithm that directly predicts the coordinates and categories of the bounding box without generating candidate regions. For object detection of different sizes, the traditional approach is to convert the image into different sizes, process them separately, and finally combine the results.

[0065] The newly designed SSD detection network is denoted as S. Its main network structure is a VGG16 model with two fully connected layers changed into convolutional layers and four more convolutional layers. The outputs of five different convolutional layers (conv4_3, conv7, conv9_2, conv10_2, conv11_2) are convolved with two 3*3 convolutional kernels, one of which outputs the classification confidence, and each default box generates three confidences (corresponding to the background, real person, and non-living face in the live detection task); the other convolutional kernel outputs the location of the target, and each default box generates four coordinate values ​​(x, y, w, h); in addition, these five convolutional layers also pass through the priorBox layer to generate a default box; finally, the previous three calculation results are merged and passed to the loss layer. Among them, the default box refers to a series of fixed-size default boxes in each small grid of the feature map. The PriorBox layer is a network structure layer used to deploy the default box at each position (pixel) in the feature map.

[0066] Generally speaking, the object detector will first generate candidate regions using some method, and then classify and accurately locate these regions, but this repeated sampling process will take a lot of time and slow down the speed. The newly designed SSD detection network of the present invention is a network that can directly locate and classify objects in images. The network will generate a series of fixed-size borders on each cell in several specific layers of feature maps, so the network can detect objects with different aspect ratios.

[0067] Therefore, according to the non-cooperative face liveness detection method of the embodiment of the present invention, the SSD detection network adopts the onestage idea to improve the detection speed; and the anchors idea in Faster R-CNN is integrated into the network, and feature hierarchical extraction is performed and the border regression and classification operations are calculated in sequence. The technical advantages of this method are: the SSD detection network can adapt to the training and detection tasks of electronic device borders of various scales, and can detect targets in real time with high precision.

[0068] According to one embodiment of the present invention, the specific steps of step S4 are as follows:

[0069] S41, dividing the image set Q into a training set T and a verification set V, so that both the training set T and the verification set V contain images G with bounding box annotation information and images P without annotation information.

[0070] S42. Perform forward calculations on some face images P in the training set T through the Inception Net classification network M, output recognition results after passing through each layer of the model, apply the batch stochastic gradient descent algorithm to perform model training based on the label difference between the current network output and input features, and continuously adjust the weights in the Inception Net classification network M.

[0071] S43. Use the image P without labeled information in the verification set V to verify the model training effect. When the accuracy of the model M on the verification set V cannot continue to improve with the training time, the model training stops and the model M' is obtained.

[0072] S44. Perform forward calculation on some images G with bounding box annotation information in the training set T through the SSD detection network S. After passing through each layer of the model, the target bounding box detected by the model and the category corresponding to each bounding box are output.

[0073] S45. Compare the current network output result with the target’s labeled information and calculate the positioning loss L loc With confidence loss L conf ; The overall objective loss function of the network L(x, c, l, g) is L loc With L conf The weighted sum of is shown in formula (1):

[0074]

[0075] Among them, x represents the matching situation between the prior box and the annotated information; c is the confidence of the predicted box output by the model, l is the predicted box of the model, g is the position of the annotated box, a is a parameter used to adjust the ratio between the confidence loss and the positioning loss, and N is the number of predicted boxes that match the annotated box information; the batch stochastic gradient descent method algorithm is used to train the model of the objective function in formula (1), and the weights in the SSD detection network S are continuously adjusted.

[0076] S46. Use the image G with border annotation information in the verification set V to verify the model training effect. When the accuracy of the model S on the verification set V cannot continue to improve with the training time, the model training stops and the model S' is obtained.

[0077] S47. After the image passes through M' and S', the confidence scores of the real person category input into the two models are added together and recorded as the confidence score of model D.

[0078] Among them, in step S42, forward calculation is performed on some face images P in the training set T through the Inception Net classification network M, and the recognition results are output after passing through each layer of the model (there are n layers in total, namely M1,...Mn). Each layer of the network is composed of many neurons, and each neuron has a preset weight. Then, according to the label difference between the current network output and the input feature, the batch stochastic gradient descent algorithm is applied to perform model training, and the weights in the Inception Net classification network M are continuously adjusted.

[0079] In step S44, the model has k layers, namely S1, ...Sk, and each layer of the network is composed of many neurons, and each neuron has a preset weight.

[0080] In step S47, the trained Inception Net classification network M' and SSD detection network S' will have a good recognition effect on the pictures in the training set; the result of fusing the two models will have a stronger discrimination ability than a single model, which is recorded as model D. The fusion method used is: after the picture passes through M' and S', the confidence of the two models input into the real person category is added, which is recorded as the confidence score of model D.

[0081] Therefore, according to the non-cooperative face liveness detection method of the embodiment of the present invention, a classifier that integrates the SSD detection network and the face Inception net classification network is used to classify the authenticity of the face. The technical advantages of using this method are: using the SSD detection network to detect the environmental information of the face, and using the face Inception net to classify the partial content of the face. The fusion of the two models has a better liveness detection effect than a single classifier or detector.

[0082] like Figure 3 As shown, according to one embodiment of the present invention, step S5 includes:

[0083] S51. Obtain a real-time RGB image A through a camera device, input image A into a cascade convolutional neural network model, and perform face detection; if a face is detected, normalize the face to obtain a normalized face image B; and record the position k of the face in image A.

[0084] S52 , input the image A into the model M' obtained in step S4 to obtain a result x.

[0085] S53 , input the picture B into the model S′ obtained in step S4 to obtain a bounding box set Y.

[0086] S54, determine the overlap rate between the face position k and each frame in the frame set Y; if there is a frame whose overlap rate exceeds the set value, record the confidence u of the frame; and select the maximum value v from all u that meet the conditions.

[0087] S55. Input the result x and the maximum confidence value v into the fusion model D obtained in step S4, and use the fusion model D to give the face liveness detection prediction result of image A.

[0088] Specifically, in step S51, a real-time RGB image A is obtained through a camera device, and image A is input into a cascade convolutional neural network (Cascade CNN) model to perform face detection; if a face is detected, the face area picture in image A is input into the face key point detection model to locate the face key points, and by calculating the affine transformation from the key points to the standard key points, the face pictures in different postures are transformed into the face pictures in the standard posture to obtain a normalized face image B; and the position k of the face in image A is recorded.

[0089] In step S53, since the model S' will generate a large number of predicted borders, but these predicted borders may have a large number of errors and overlaps, the overlapping or erroneous predicted borders are removed through the non-maximum suppression algorithm to obtain the border set W.

[0090] In other words, after the above training model is established, when the real-time image is detected, the real-time image is first obtained through the camera device to determine whether a face is detected. If a face is detected, normalization processing is performed to obtain a normalized face image B, and the position k of the face in image A is recorded. Image A and image B are respectively input into the model obtained in the above step S4 to obtain different results. Then, the overlap rate between the face position k and each frame in the frame set Y is determined. Finally, the result and the maximum confidence value are input into the fusion model to predict the face liveness detection result.

[0091] In summary, the non-cooperative face liveness detection method according to the embodiment of the present invention greatly improves the accuracy and speed of face liveness detection by using the InceptionNet classification network in conjunction with the SSD detection network. The SSD detection network can adapt to the training and detection tasks of electronic device frames of various scales, and can detect targets in real time and with high precision. The liveness detection method used in the present invention does not require the addition of hardware equipment other than the camera, and does not require the cooperation of the person to be tested, can achieve ultra-real-time face liveness detection speed, is low-cost, and is easy to use.

[0092] In addition, the present invention also provides a computer storage medium, which includes one or more computer instructions, and the one or more computer instructions implement any of the above-mentioned non-cooperative face liveness detection methods 100 when executed.

[0093] That is, the computer storage medium stores a computer program, and when the computer program is executed by the processor, the processor executes any of the non-cooperative face liveness detection methods 100 described above.

[0094] like Figure 4 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310 and a processor 320, wherein the memory 310 is used to store one or more computer instructions, and the processor 320 is used to call and execute the one or more computer instructions, so as to implement any of the methods described above.

[0095] That is, the electronic device 300 includes: a processor 320 and a memory 310, wherein the memory 310 stores computer program instructions, wherein when the computer program instructions are executed by the processor, the processor 320 executes any of the above-mentioned methods 100.

[0096] Furthermore, if Figure 4 As shown, the electronic device 300 further includes a network interface 330 , an input device 340 , a hard disk 350 , and a display device 360 ​​.

[0097] The above-mentioned interfaces and devices can be interconnected through a bus architecture. The bus architecture can be a bus and bridge that can include any number of interconnected buses. Specifically, one or more central processing units (CPUs) represented by processor 320 and various circuits of one or more memories represented by memory 310 are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together. It can be understood that the bus architecture is used to achieve connection and communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are all well known in the art, so they will not be described in detail herein.

[0098] The network interface 330 can be connected to a network (such as the Internet, a local area network, etc.), obtain relevant data from the network, and save it in the hard disk 350.

[0099] The input device 340 can receive various instructions input by the operator and send them to the processor 320 for execution. The input device 340 can include a keyboard or a pointing device (for example, a mouse, a trackball, a touch pad or a touch screen, etc.).

[0100] The display device 360 ​​can display the result obtained by the processor 320 executing the instruction.

[0101] The memory 310 is used to store programs and data necessary for the operation of the operating system, as well as data such as intermediate results during the calculation process of the processor 320.

[0102] It is understood that the memory 310 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. The memory 310 of the apparatus and method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0103] In some implementations, the memory 310 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 311 and application programs 312 .

[0104] The operating system 311 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 312 includes various application programs, such as a browser, etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 312.

[0105] The method disclosed in the above embodiment of the present invention can be applied to the processor 320, or implemented by the processor 320. The processor 320 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 320 or the instruction in the form of software. The above processor 320 can be a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 310, and the processor 320 reads the information in the memory 310 and completes the steps of the above method in combination with its hardware.

[0106] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0107] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0108] Specifically, the processor 320 is further configured to read the computer program and execute any of the above methods.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0111] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0112] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A non-cooperative face liveness detection method, characterized in that: The method comprises the following steps: S1. Obtain a set of liveness detection images for model training, and process the set of images; S2. Design the Inception Net classification network; S3, design SSD detection network; S4, performing model training on the image set through the Inception Net classification network and the SSD detection network; S5, acquiring a real-time image, and predicting whether the real-time image is a living human face according to the model training result; Step S1 includes: S11, batch saving of a living body detection image set Q by a camera device; S12, performing border annotation on a first set of images in the image set Q to obtain an image G with border annotation information and an image P without annotation information, wherein the first set of images is a subset of the image set Q; Step S4 includes: S41, dividing the image set Q into a training set T and a verification set V, so that both the training set T and the verification set V contain images G with border annotation information and images P without annotation information; S42, forward calculation is performed on some face images P in the training set T through the Inception Net classification network M, and the recognition results are output after passing through each layer of the model. The batch stochastic gradient descent algorithm is applied to perform model training according to the label difference between the current network output and the input feature, and each weight in the Inception Net classification network M is continuously adjusted; S43, using the image P without annotation information in the verification set V to verify the model training effect, when the accuracy of the model M on the verification set V cannot continue to improve with the training time, the model training stops, and the model M' is obtained; S44, forward calculation is performed on some images G with border annotation information in the training set T through the SSD detection network S, and after passing through each layer of the model, the target border detected by the model and the category corresponding to each border are output; S45. Compare the current network output result with the target’s labeled information and calculate the positioning loss L loc With confidence loss L conf ; The overall objective loss function of the network L(x, c, l, g) is L loc With L conf The weighted sum of is shown in formula (1): Where x represents the matching of the prior box and the annotation information; c is the confidence of the predicted box output by the model, l is the predicted box of the model, g is the position of the annotated box, a is a parameter used to adjust the ratio between the confidence loss and the positioning loss, and N is the number of predicted boxes that match the annotated box information; the batch stochastic gradient descent method algorithm is used to train the model of the objective function in formula (1), and the weights in the SSD detection network S are continuously adjusted; S46, using the image G with border annotation information in the verification set V to verify the model training effect, when the accuracy of the model S on the verification set V cannot continue to improve with the training time, the model training stops and the model S' is obtained; S47, after the image passes through M' and S', the confidence scores of the real person category input into the two models are added together, and recorded as the confidence score of model D; The Inception Net classification network includes a deep neural network A and three groups of Inception structures C. The Inception structure C includes four branches, each of which is composed of 1*1 convolution, 3*3 convolution, 5*5 convolution, and 3*3 maximum pooling. In the step S3, the designed SSD detection network is denoted as S, and the main network of the SSD detection network is a VGG16 model with 4 convolutional layers added to the convolutional layer; The step S3 further comprises: The outputs of the five different convolutional layers are convolved with two 3*3 convolution kernels, one of which outputs the classification confidence, and each default box generates three confidences, corresponding to the background, real person, and non-living face in the liveness detection task, respectively. The output of the other convolution kernel is the location of the target position, and each default box generates four coordinate values ​​of x, y, w, and h; 5 convolutional layers pass through the priorBox layer to generate the default box; Combine the above three calculation results and pass them to the loss layer; The step S5 comprises: S51, obtaining a real-time RGB image A through a camera device, inputting the image A into a cascade convolutional neural network model to perform face detection; if a face is detected, normalizing the face to obtain a normalized face image B; and recording the position k of the face in the image A; S52, inputting the image A into the model M' obtained in step S4 to obtain a result x; S53, inputting the picture B into the model S' obtained in step S4 to obtain a bounding box set Y; S54, determining the overlap rate between the face position k and each frame in the frame set Y; if there is a frame whose overlap rate exceeds a set value, recording the confidence u of the frame; and selecting the maximum value v from all u that meet the conditions; S55. Input the result x and the maximum confidence value v into the fusion model D obtained in step S4, and use the fusion model D to give the face liveness detection prediction result of image A.

2. The method according to claim 1, characterized in that In step S51, the steps of normalizing the face are: The face area image in image A is input into the face key point detection model to locate the face key points, and the face images in different postures are transformed into the face images in the standard posture by calculating the affine transformation from the key points to the standard key points, so as to obtain the normalized face image B.

3. The method according to claim 1, characterized in that In step S53, overlapping or erroneous predicted borders are removed by a non-maximum suppression algorithm to obtain a border set W, and subsequent operations are performed using the border set W.

4. A computer storage medium, characterized in that: The method comprises one or more computer instructions, which implement the method according to any one of claims 1 to 3 when executed.

Images