Living body detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a live body detection method and device, electronic equipment and storage medium. The live body detection method comprises: acquiring a to-be-detected image; inputting the to-be-detected image into a pre-trained live body detection model to acquire a face feature and a live body feature corresponding to the to-be-detected image output by the live body detection model; acquiring a preset live body feature corresponding to the face feature; and determining a live body detection result corresponding to the to-be-detected image based on the live body feature and the preset live body feature. Through the above method, the face feature and the live body feature are obtained by inputting the to-be-detected image into the live body detection model, face recognition is performed on the face feature, the preset live body feature is obtained if the face recognition is successful, and the preset live body feature is compared with the live body feature to determine whether it is a live body, thereby improving the live body detection accuracy and improving the user experience.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to a liveness detection method, device, electronic device, and readable storage medium. Background Technology

[0002] With the continuous development of computer technology, facial recognition technology has also advanced, finding widespread application in scenarios such as financial payments, identity verification, and location tracking. Furthermore, when combined with liveness detection, it further enhances the authenticity and security of these applications. However, the accuracy of liveness detection methods still needs improvement when processing liveness data. Summary of the Invention

[0003] In view of the above problems, this application proposes a liveness detection method, apparatus, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, embodiments of this application provide a liveness detection method applied to an electronic device. The method includes: firstly acquiring an image to be detected; then inputting the image to be detected into a pre-trained liveness detection model to acquire facial features and liveness features corresponding to the image to be detected, output by the liveness detection model; then acquiring preset liveness features corresponding to the facial features; and finally determining the liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features.

[0005] Secondly, embodiments of this application provide a liveness detection device, operating in an electronic device. The device includes: a target image acquisition unit, a face feature and liveness feature acquisition unit, a preset liveness feature acquisition unit, and a liveness detection result determination unit. The target image acquisition unit is used to acquire a target image; the face feature and liveness feature acquisition unit is used to input the target image into a pre-trained liveness detection model to acquire the face features and liveness features corresponding to the target image output by the liveness detection model; the preset liveness feature acquisition unit is used to acquire preset liveness features corresponding to the face features; and the liveness detection result determination unit is used to determine the liveness detection result corresponding to the target image based on the liveness features and the preset liveness features.

[0006] Thirdly, embodiments of this application provide an electronic device for a liveness detection method, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.

[0008] This application provides a liveness detection method, apparatus, electronic device, and storage medium. The liveness detection method includes: acquiring an image to be detected; inputting the image to be detected into a pre-trained liveness detection model to acquire facial features and liveness features corresponding to the image output by the liveness detection model; acquiring preset liveness features corresponding to the facial features; and determining a liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features. Through this method, by inputting the image to be detected into a liveness detection model to obtain facial features and liveness features, performing face recognition on the facial features, obtaining preset liveness features if face recognition is successful, and comparing the preset liveness features with the liveness features to determine whether a person is alive, the accuracy of liveness detection is improved, and the user experience is enhanced. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart of a liveness detection method according to an embodiment of this application is shown;

[0011] Figure 2 A model training diagram of a liveness detection method according to another embodiment of this application is shown;

[0012] Figure 3 This invention illustrates the structure of an image feature extraction module for a liveness detection method according to another embodiment of the present application;

[0013] Figure 4 A flowchart of a liveness detection method according to another embodiment of this application is shown;

[0014] Figure 5 A flowchart of a liveness detection method according to another embodiment of this application is shown;

[0015] Figure 6 A flowchart of a liveness detection method according to another embodiment of this application is shown;

[0016] Figure 7 This paper shows a structural block diagram of a liveness detection device according to another embodiment of the present application;

[0017] Figure 8 This paper shows a structural block diagram of a liveness detection device according to another embodiment of the present application;

[0018] Figure 9 This diagram shows a structural block diagram of an electronic device used to perform the liveness detection method of the embodiments of this application in real time;

[0019] Figure 10 The present application shows a storage unit for storing or carrying program code that implements the liveness detection method according to the embodiments of the present application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] In facial recognition, liveness detection is often used to verify the identity of a user by analyzing their facial features. Liveness detection verifies whether a user is a real, living person by using techniques such as facial landmark localization and face tracking, based on combined actions like blinking, opening their mouth, shaking their head, and nodding. The system captures video or images containing faces using a camera, automatically monitors and tracks faces within the video or images, performs face image matching and recognition, and then identifies the detected faces.

[0022] The inventors, in their research on related liveness detection methods, discovered that these methods typically add a style transfer module to the overall process. After the electronic device's camera acquires a facial image, the image is first input into the style transfer module to obtain a style-transferred facial image. This style-transferred image is then input into a multi-task application network to extract shared features. These shared features are then input into separate liveness detection and face recognition branch networks to obtain liveness classification and face classification results, respectively. Finally, the face recognition result is obtained based on the liveness and face classification results. However, in these methods, the addition of the style transfer module significantly increases the computational load. Furthermore, the system directly outputs the liveness detection result without utilizing user registration information, which negatively impacts the accuracy of liveness detection.

[0023] Therefore, the inventors have proposed a liveness detection method, apparatus, electronic device, and storage medium according to embodiments of this application. First, an image to be detected is acquired; then, the image to be detected is input into a pre-trained liveness detection model to obtain facial features and liveness features corresponding to the image output by the liveness detection model; next, a preset liveness feature corresponding to the facial features is acquired; finally, based on the liveness feature and the preset liveness feature, the liveness detection result corresponding to the image to be detected is determined. Through the above method, by inputting the image to be detected into a liveness detection model to obtain facial features and liveness features, obtaining preset liveness features based on the recognition results of the facial features, and comparing the preset liveness feature with the liveness feature to determine whether a person is alive, the accuracy of liveness detection is improved, and the user experience is enhanced.

[0024] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0025] Please see Figure 1 This application provides a liveness detection method applied to electronic devices, the method comprising:

[0026] Step S110: Obtain the image to be detected.

[0027] In this embodiment of the application, the image to be detected can be acquired by an image acquisition device, which can be a camera.

[0028] In one approach, when face recognition is required, the image acquisition device is activated and acquires the current image. The system detects the current image. If the current image contains a face image, the system crops the current image until a face image is obtained and uses it as the image to be detected. If the system detects that the current image does not contain a face image, the image acquisition device re-acquires the image until the system detects that the current image contains a face image.

[0029] Step S120: Input the image to be detected into a pre-trained liveness detection model to obtain the facial features and liveness features corresponding to the image to be detected output by the liveness detection model.

[0030] In this embodiment, facial features include the structural relationship and geometric description between the eyes, nose, and mouth; liveness features include actions such as blinking, opening the mouth, nodding, and shaking the head. The liveness detection model may include an image feature extraction module, a facial feature extraction module, and a liveness feature extraction module. The image feature extraction module is used to extract image features from the image to be detected, the facial feature extraction module is used to extract facial features included in the image features of the image to be detected, and the liveness feature extraction module is used to extract liveness features included in the image features of the image to be detected.

[0031] In this embodiment, the system uses the acquired face image as the image to be detected and inputs it into a pre-trained liveness detection model. First, the image feature extraction module extracts the image features of the image to be detected. Then, the face feature extraction module and the liveness feature extraction module extract the image features respectively, obtaining face feature data and liveness feature data. Specifically, the face feature data corresponds to the face feature extraction module, and the liveness data corresponds to the liveness feature extraction module.

[0032] In this embodiment, the liveness detection model can be a pre-trained detection model based on a CNN neural network. The training process for the liveness detection model includes:

[0033] Step S121: Obtain a training image set, which includes positive sample images and negative sample images.

[0034] In the embodiments of this application, positive samples are used to characterize live individuals in the liveness binary classification, and negative samples are used to characterize non-live individuals in the liveness binary classification.

[0035] Step S122: Input the positive sample image into the model to be trained to obtain the face features and liveness features corresponding to the positive sample image.

[0036] In this embodiment, the model to be trained comprises three parts: an image feature extractor, a face feature extractor, and a liveness feature extractor. The system acquires positive sample images from the training image set and inputs them into the model to be trained. The model then extracts features from the positive sample images. The face feature extractor extracts facial features from the positive sample images, and the liveness feature extractor extracts liveness features from the positive sample images. The model to be trained is based on a CNN neural network.

[0037] In step S123: The negative sample image is input into the model to be trained to obtain the face features and liveness features corresponding to the negative sample image.

[0038] In this embodiment of the application, the system acquires negative sample images from the training image set and inputs them into the model to be trained. The model to be trained performs feature extraction on the negative sample images. The face feature extractor can extract face features from the negative sample images, and the liveness feature extractor can extract liveness features from the negative sample images.

[0039] Step S124: Based on the preset loss function, the face features and liveness features corresponding to the positive sample image, and the face features and liveness features corresponding to the negative sample image, obtain the loss function value, wherein the preset loss function includes the triplet loss function, the adversarial loss function, and the binary classification loss function.

[0040] In this embodiment, when the system acquires the facial features and liveness features corresponding to the positive and negative sample images, it obtains the loss function value through a loss function. The loss function can include a triplet loss function, an adversarial loss function, and a binary classification loss function. The triplet loss function corresponds to the triplet loss function value, the adversarial loss function corresponds to the adversarial loss function value, and the binary classification loss function corresponds to the liveness binary classification loss function value and the facial recognition loss function value. The loss function value is a linear sum of the corresponding function values. Based on the function value of each loss function, corresponding weights are assigned, i.e., loss function value = a * triplet loss function value + b * adversarial loss function value + c * liveness binary classification loss function value + d * facial recognition loss function value, where a + b + c + d = 1, and the values ​​of a, b, c, and d can be dynamically adjusted. For example, we can set a = b = c = d = 0.25, that is, the loss function value = 0.25 * triplet loss function value + 0.25 * adversarial loss function value + 0.25 * liveness binary classification loss function value + 0.25 * face recognition loss function value.

[0041] As one approach, when the system obtains facial features through positive and negative sample images, it also obtains the facial recognition loss function value through the facial recognition loss function; when the system obtains liveness features through positive and negative sample images, it also obtains the corresponding binary classification loss function value, triplet loss function value, and adversarial loss function value through the liveness binary classification loss function, triplet loss function, and adversarial loss function.

[0042] Step S125: Iteratively train the model to be trained according to the loss function value until the training termination condition is met, and obtain the liveness detection model.

[0043] In this embodiment, during the iterative training of the model to be trained, the loss function value gradually decreases. When the loss function value decreases to a certain level, the model training is considered complete. This can be achieved by using the decrease in the value of any loss function to a certain level as the termination condition for model training, or by using the decrease in the loss function value itself as the termination condition. For example, based on the loss function value, the model to be trained is iteratively trained. When the liveness detection binary classification loss function value decreases to 0.05, the model training ends, and a liveness detection model is obtained.

[0044] For training the model to be trained, the training process described in steps S121, S122, S123, S124, and S125 can be as follows: Figure 2 As shown, the model to be trained may include an image feature extractor, a face feature extractor, and a liveness feature extractor. The image feature extractor may include five sub-modules, each of which may include a convolutional layer, a batch normalization module, and a nonlinear activation function. The specific structure of each sub-module can be as follows: Figure 3 As shown, the face feature extractor may include convolutional layers, tensor flattening modules, neural network overfitting prevention modules, and fully connected layers; the liveness feature extractor may include feature fusion layers, convolutional layers, pooling layers, and fully connected layers. When the built-in camera of an electronic device captures the current image, it determines that the current image contains a face image. The background of the current image is cropped, retaining the face image. The acquired face image is used as the input image for the model to be trained. The input image is first input into the first sub-module of the image feature extractor to extract image features from the input image. The output of the first sub-module is used as the input of the second sub-module, and so on. The liveness feature extractor acquires image features from the third, fourth, and fifth sub-modules. These features are then fused through a feature fusion layer, and the fused features are further processed through a convolutional layer. The result is then input into a pooling layer for data compression, followed by a fully connected layer to obtain the triplet loss. The result from the fully connected layer is then input into another fully connected layer to obtain the liveness feature and liveness binary classification loss. Simultaneously, the output of the first fully connected layer is input into a gradient inversion layer, and the output of the gradient inversion layer is input into a discriminator to obtain the adversarial loss. The face feature extractor acquires image features from the last sub-module of the image feature extractor. These features are then input into a convolutional layer for convolution, and the result is processed through a tensor flattening module. This process is then further processed by a neural network overfitting prevention module, and the result is transmitted to a fully connected layer. The fully connected layer outputs the face recognition loss and face features. The face recognition loss can include cosface and arcface, which are not specifically limited here. The tensor flattening module flattens the convolutional layer outputs passed to the fully connected layer between inputs. The neural network overfitting prevention module randomly selects some neurons and temporarily hides them for a certain layer in an iteration during neural network training, and then performs the current training and optimization. In the next iteration, some neurons are randomly hidden again, and so on until the training ends. The gradient reversal layer multiplies the gradient passed to the gradient reversal layer by a negative number, so that the training objectives of the network before and after the gradient reversal layer are opposite, thus achieving an adversarial effect.

[0045] Step S130: Obtain the preset liveness feature corresponding to the facial feature.

[0046] In this embodiment of the application, after the system obtains the facial features and liveness features output by the pre-trained liveness detection model, the system performs facial recognition on the facial features. When the system successfully determines the facial features, the system obtains the preset liveness features, which are the liveness features that the system has saved in advance.

[0047] Step S140: Based on the liveness features and the preset liveness features, determine the liveness detection result corresponding to the image to be detected.

[0048] In this embodiment of the application, after the system successfully recognizes the facial features, the system obtains the preset liveness feature and compares the obtained liveness feature with the preset liveness feature. If the liveness feature is the same as the preset liveness feature, the liveness detection result is live; if the liveness feature is different from the preset liveness feature, the liveness detection result is not live.

[0049] As a method, liveness detection is generally divided into cooperative and non-cooperative liveness detection. Cooperative liveness detection is the most common method. It verifies whether the user is a real, living person by using a combination of cooperative actions such as blinking, opening their mouth, shaking their head, nodding, or reading out random numbers, and employing technologies such as facial landmark localization and face tracking. Non-cooperative liveness detection does not require any additional actions from the user and can directly identify spoofed faces such as paper photos, screen images, and face masks. It generally uses dual cameras for liveness detection and does not require users to perform actions such as blinking or nodding based on prompts. It has higher requirements for algorithms but is faster and provides a more user-friendly experience.

[0050] This application provides a liveness detection method. First, an image to be detected is acquired. Then, the image is input into a pre-trained liveness detection model to obtain facial features and liveness features corresponding to the image output by the liveness detection model. Next, a preset liveness feature corresponding to the facial features is acquired. Finally, based on the liveness feature and the preset liveness feature, the liveness detection result corresponding to the image to be detected is determined. Through this method, by inputting the image to be detected into the liveness detection model to obtain facial features and liveness features, performing face recognition on the facial features, and obtaining the preset liveness feature if the face recognition is successful, the preset liveness feature is compared with the liveness feature to determine whether the person is alive. This improves the accuracy of liveness detection and enhances the user experience.

[0051] Please see Figure 4 This application provides a liveness detection method applied to electronic devices, the method comprising:

[0052] Step S210: Obtain the image to be detected.

[0053] Step S210 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0054] Step S220: Input the image to be detected into the image feature extraction module to obtain the image features corresponding to the image to be detected output by the image feature extraction module.

[0055] In this embodiment of the application, the image feature extraction module includes multiple sub-modules, wherein each sub-module is connected sequentially, and in adjacent sub-modules, the preceding sub-module is the input of the following sub-module.

[0056] In one approach, when the system crops the image acquired by the image acquisition device to obtain the face image in the image, which is the image to be detected, the system inputs the image to be detected into the image feature extraction module. First, the image to be detected is input into the first sub-module of the image feature extraction module. After the image to be detected is processed by the first sub-module, it is transmitted as the output data of the first sub-module to the second sub-module as input. Subsequent sub-modules perform calculations and processing according to the above method. In the last sub-module, the image features corresponding to the image to be detected can be output.

[0057] As one approach, each submodule includes convolutional layers, a batch normalization module, and a nonlinear activation function. The convolutional layers are used to perform convolution calculations on the input features; the batch normalization module is used to normalize the input features of each layer, thereby accelerating the learning of the neural network and improving its learning efficiency; the nonlinear activation function is used to enhance the nonlinearity of the neural network. Without the nonlinear activation function, the output of each layer is the input of the previous layer, meaning that no matter how many layers the neural network has, the output is always a linear combination of the inputs.

[0058] Step S230: Input the image features into the face feature extraction module to obtain the face features corresponding to the image to be detected output by the face feature extraction module.

[0059] In this embodiment of the application, after the image to be detected is processed by multiple sub-modules in the image feature extractor to obtain image features, the system transmits the obtained image features as input to the face feature extraction module, and obtains face features by calculating and processing the image features through the face feature extraction module.

[0060] Step S240: Input the image features into the liveness feature extraction module, and obtain the liveness features corresponding to the image to be detected output by the liveness feature extraction module.

[0061] In this embodiment, the liveness feature extraction module acquires the output data of a preset number of sub-modules at the end of the image feature extraction module. The acquired output data packets contain partial image features. The liveness feature extraction module performs feature fusion on the acquired output data and then processes the fused data to obtain liveness features. For example, the preset number can be 3.

[0062] Step S250: Perform face recognition on the facial features to obtain the corresponding face recognition result.

[0063] In this embodiment of the application, facial features are obtained by processing image features through a facial feature extraction module. The system then performs facial recognition on the facial features to determine whether the facial recognition is successful.

[0064] As a method, face recognition comprises four components: face image acquisition and detection, face image preprocessing, face image feature extraction, and face image matching and recognition. Face image acquisition and detection uses a camera to capture images and accurately mark the position and size of faces within the images. Face detection algorithms can utilize the Adaboost learning algorithm. Face image preprocessing, based on the face detection results, preprocesses the original images due to various interferences. Preprocessing may include lighting compensation, grayscale compensation, normalization, and filtering, without specific limitations. Face image feature extraction extracts features from the face image, including visual features, pixel statistical features, and algebraic features. This process of face feature modeling is essentially a process of creating a face feature set. Face image matching and recognition searches and matches the extracted face image feature data against feature templates stored in a database. By setting a similarity threshold, the matching result is output when the similarity exceeds the threshold.

[0065] Step S260: If the face recognition result indicates that the face recognition is successful, obtain the preset liveness feature corresponding to the face feature.

[0066] In this embodiment of the application, the system performs face recognition on the face features output by the face feature extraction module. If the face feature recognition is successful, the system obtains the preset liveness feature.

[0067] As another approach, if face recognition fails, the image to be detected is reacquired, and the image is input into a pre-trained liveness detection model to obtain face features and liveness features. Then, the obtained face features are used for face recognition.

[0068] Step S270: Obtain the similarity between the live feature and the preset live feature.

[0069] In this embodiment of the application, after face recognition is successful, the system obtains a preset liveness feature and compares the liveness feature with the preset liveness feature to obtain the similarity between the liveness feature and the preset liveness feature.

[0070] Step S280: If the similarity exceeds the preset similarity, determine that the liveness detection result corresponding to the image to be detected is a live body.

[0071] In this embodiment, after the system obtains the similarity between the liveness feature and the preset liveness feature, it compares the obtained similarity with the preset similarity. If the similarity is greater than the preset similarity, the system determines that the liveness detection result is a live body. The preset similarity is a threshold characterizing the similarity of the liveness detection result to a live body. When the similarity between the liveness feature and the preset liveness feature is greater than the preset similarity, the system determines that the liveness detection result is a live body; when the similarity between the liveness feature and the preset liveness feature is less than the preset similarity, the system determines that the liveness detection result is a non-live body.

[0072] For example, the system can set the preset similarity of liveness detection to 90%. If the system detects that the similarity between the liveness feature and the preset liveness feature is greater than or equal to 90%, the system determines that the liveness detection is a liveness detection; if the system detects that the similarity between the liveness feature and the preset liveness feature is less than 90%, the system determines that the liveness detection is a non-liveness detection.

[0073] This application provides a liveness detection method, which first acquires an image to be detected, then inputs the image to be detected into an image feature extraction module to acquire image features corresponding to the image to be detected, output by the image feature extraction module, then inputs the image features into a face feature extraction module to acquire face features corresponding to the image to be detected, output by the face feature extraction module, then inputs the image features into a liveness feature extraction module to acquire liveness features corresponding to the image to be detected, output by the liveness feature extraction module, then performs face recognition on the face features to acquire the corresponding face recognition result, if the face recognition result indicates successful face recognition, acquires a preset liveness feature corresponding to the face feature, then acquires the similarity between the liveness feature and the preset liveness feature, and finally, if the similarity exceeds a preset similarity, the liveness detection result corresponding to the image to be detected is determined to be live. The above method obtains facial features and liveness features by inputting the image to be detected into the liveness detection model. Based on the recognition results of the facial features, a preset liveness feature is obtained. The preset liveness feature is compared with the liveness feature to determine whether the person is alive, thereby improving the accuracy of liveness detection and enhancing the user experience.

[0074] Please see Figure 5 This application provides a liveness detection method applied to electronic devices, the method comprising:

[0075] Step S301: Obtain the image to be detected.

[0076] Step S301 can be specifically explained in the detailed explanation of the above embodiments, and therefore will not be repeated in this embodiment.

[0077] Step S302: Input the image to be detected into the image feature extraction module to obtain the image features corresponding to the image to be detected output by the image feature extraction module.

[0078] Step S302 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0079] Step S303: Obtain the output of the last submodule among the plurality of submodules.

[0080] In this embodiment, the system inputs the image to be detected into the image feature extraction module. After data processing by multiple modules in the image extraction module, the last sub-module outputs the image features of the image to be detected. At the same time, the face feature extraction module obtains the image features output by the last sub-module.

[0081] Step S304: Input the output of the last sub-module into the face feature extraction module to obtain the face features corresponding to the image to be detected output by the face feature extraction module.

[0082] In this embodiment of the application, after the face feature extraction module obtains the image features output by the last sub-module of the image feature extraction module, the image features are used as the input of the face feature extraction module, and the face features are obtained after the image features are processed by the face feature extraction module.

[0083] Step S305: Obtain the output of a preset number of sub-modules among the plurality of sub-modules.

[0084] In this embodiment, the face feature extraction module obtains the output of a preset number of sub-modules located at the end of the multiple sub-modules in the image feature extraction module, and performs feature fusion on them.

[0085] Step S306: Input the output of the preset number of sub-modules into the liveness feature extraction module, and obtain the liveness features corresponding to the image to be detected output by the liveness feature extraction module.

[0086] In this embodiment, after the liveness feature extraction module obtains the output of a preset number of sub-modules, it performs feature fusion on them. The liveness feature extraction module then processes the fused features to obtain the liveness features.

[0087] Step S307: Perform face recognition on the facial features to obtain the corresponding face recognition result.

[0088] Step S307 can be found in the detailed explanation in the above embodiments, and therefore will not be repeated in this embodiment.

[0089] Step S308: If the face recognition result indicates that the face recognition is successful, obtain the preset liveness feature corresponding to the face feature.

[0090] Step S308 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0091] Step S309: Obtain the similarity between the live feature and the preset live feature.

[0092] Step S309 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0093] Step S310: If the similarity exceeds the preset similarity, determine that the liveness detection result corresponding to the image to be detected is a live body.

[0094] Step S310 can be referred to in detail in the above embodiments, and therefore will not be repeated in this embodiment.

[0095] This application provides a liveness detection method. First, an image to be detected is acquired. Then, the image to be detected is input into an image feature extraction module to acquire image features corresponding to the image to be detected, output by the image feature extraction module. Next, the output of the last sub-module among multiple sub-modules is acquired and input into a face feature extraction module to acquire face features corresponding to the image to be detected, output by the face feature extraction module. Then, the outputs of a preset number of sub-modules among the multiple sub-modules are acquired and input into a liveness feature extraction module to acquire liveness features corresponding to the image to be detected, output by the liveness feature extraction module. Finally, face recognition is performed on the face features to acquire the corresponding face recognition result. If the face recognition result indicates successful face recognition, a preset liveness feature corresponding to the face feature is acquired. Then, the similarity between the liveness feature and the preset liveness feature is acquired. If the similarity exceeds a preset similarity, the liveness detection result corresponding to the image to be detected is determined to be live. The above method obtains facial features and liveness features by inputting the image to be detected into the liveness detection model. Facial recognition is then performed on the facial features. If the facial recognition is successful, a preset liveness feature is obtained. The preset liveness feature is compared with the liveness feature to determine whether the person is alive, thereby improving the accuracy of liveness detection and enhancing the user experience.

[0096] Please see Figure 6 This application provides a liveness detection method applied to electronic devices, the method comprising:

[0097] Step S410: Input image.

[0098] In this embodiment of the application, the electronic device turns on the camera, acquires the image at the current moment, and crops the image at the current moment through the system to obtain a face image, and then inputs the face image.

[0099] Step S420: CNN neural network.

[0100] In this embodiment, after the electronic device acquires the cropped face image, it inputs the face image into a CNN neural network for feature extraction. The CNN neural network extracts features from the face image and outputs the corresponding liveness features and face features.

[0101] As a method, CNN (Convolutional Neural Network) is a type of feedforward neural network that includes convolutional computations and is designed based on the translation invariance of image tasks. The hierarchical structure of a CNN consists of five parts: 1) Input layer, which transforms the input data into a four-dimensional array during image recognition; 2) Convolutional layer, composed of input data and convolutional kernels, where the kernels sample feature values ​​from the image array; 3) Activation layer, which performs a non-linear mapping on the output of the convolutional layer; 4) Pooling layer, sandwiched between consecutive convolutional layers, used to compress the amount of data and parameters, reducing overfitting. If the input is an image, its main function is to further compress the image and further amplify the feature values; 5) Fully connected layer, which connects all neurons between two layers with weights, typically located at the end of the convolutional neural network.

[0102] Step S430: Facial features.

[0103] In this embodiment of the application, after the CNN neural network extracts the features of the face image, some features in the face image features are extracted by the face feature extractor to obtain the face features, and at the same time, the face recognition loss is obtained.

[0104] Step S440: Liveness characteristics.

[0105] In this embodiment, after the CNN neural network extracts the features of the face image, some features in the face image features are extracted by a liveness feature extractor to obtain liveness features, and at the same time, liveness binary classification loss, triplet loss and adversarial loss are obtained.

[0106] Step S450: Perform face recognition. If the recognition is successful, proceed to step S460; if the recognition fails, proceed to step S410.

[0107] In this embodiment of the application, the system performs face recognition on the acquired facial features and detects whether the face recognition is successful.

[0108] Step S460: Obtain registered liveness features.

[0109] In this embodiment of the application, after successful face recognition, the system obtains the liveness features used by the user during registration.

[0110] Step S470: Compare the registered liveness feature with the liveness feature. If they are the same, proceed to step S480; if they are different, proceed to step S490.

[0111] In this embodiment of the application, after the system obtains the registered liveness data, it compares the registered liveness features with the liveness features to determine whether the liveness features are the same as the registered liveness features.

[0112] Step S480: Determine if the organism is alive.

[0113] In this embodiment of the application, if the system detects that the liveness feature is the same as the registered liveness feature, it determines that the person is alive.

[0114] Step S490: Determined to be non-living.

[0115] In this embodiment of the application, if the system detects that the liveness feature is different from the registered liveness feature, it determines that the person is not alive.

[0116] In this embodiment, an image is first input, then facial features and liveness features are obtained through a CNN neural network, followed by face recognition. If recognition fails, the image is input again; if recognition is successful, registered liveness features are obtained, and then compared with the liveness feature. If they are the same, the person is determined to be alive; if they are different, the person is determined to be not alive. Through this method, by inputting the image to be detected into the liveness detection model to obtain facial features and liveness features, and performing face recognition based on the facial features, a preset liveness feature is obtained if the face recognition is successful. The preset liveness feature is then compared with the liveness feature to determine whether the person is alive, thereby improving the accuracy of liveness detection and enhancing the user experience.

[0117] Please see Figure 7 This application provides a liveness detection device 500, which operates in an electronic device. The device 500 includes:

[0118] The image acquisition unit 510 is used to acquire the image to be detected.

[0119] The face feature and liveness feature acquisition unit 520 is used to input the image to be detected into a pre-trained liveness detection model and acquire the face features and liveness features corresponding to the image to be detected output by the liveness detection model.

[0120] In one embodiment, the face feature and liveness feature acquisition unit 520 is further configured to input the image to be detected into the image feature extraction module to acquire the image features corresponding to the image to be detected output by the image feature extraction module; input the image features into the face feature extraction module to acquire the face features corresponding to the image to be detected output by the face feature extraction module; and input the image features into the liveness feature extraction module to acquire the liveness features corresponding to the image to be detected output by the liveness feature extraction module.

[0121] Optionally, the face feature and liveness feature acquisition unit 520 is further configured to acquire the output of the last sub-module among the plurality of sub-modules; input the output of the last sub-module into the face feature extraction module to acquire the face features corresponding to the image to be detected output by the face feature extraction module.

[0122] Optionally, the face feature and liveness feature acquisition unit 520 is further configured to acquire the output of a preset number of sub-modules among the plurality of sub-modules; input the output of the preset number of sub-modules into the liveness feature extraction module; and acquire the liveness features corresponding to the image to be detected output by the liveness feature extraction module.

[0123] A preset liveness feature acquisition unit 530 is used to acquire preset liveness features corresponding to the facial features;

[0124] As one method, the preset liveness feature acquisition unit 530 is also used to perform face recognition on the face features and obtain the corresponding face recognition result; if the face recognition result indicates that the face recognition is successful, the preset liveness feature corresponding to the face features is obtained.

[0125] The liveness detection result determination unit 540 is used to determine the liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features;

[0126] In one manner, the liveness detection result determination unit 540 is also used to obtain the similarity between the liveness feature and the preset liveness feature; if the similarity exceeds the preset similarity, the liveness detection result corresponding to the image to be detected is determined to be live.

[0127] Please see Figure 8 The device 500 further includes:

[0128] The model training unit 550 is used to acquire a training image set, which includes positive sample images and negative sample images; input the positive sample images into the model to be trained to obtain the facial features and liveness features corresponding to the positive sample images; input the negative sample images into the model to be trained to obtain the facial features and liveness features corresponding to the negative sample images; obtain a loss function value based on a preset loss function, the facial features and liveness features corresponding to the positive sample images, and the facial features and liveness features corresponding to the negative sample images, wherein the preset loss function includes a triplet loss function, an adversarial loss function, and a binary classification loss function; iteratively train the model to be trained according to the loss function value until the training termination condition is met to obtain the liveness detection model.

[0129] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0130] The following will combine Figure 9 This application describes an electronic device.

[0131] Please see Figure 9 Based on the aforementioned liveness detection method and apparatus, this application also provides another electronic device 600 capable of performing the aforementioned liveness detection method. The electronic device 600 includes one or more (only one shown in the figure) processors 602, a memory 604, and a network module 606 coupled together. The memory 604 stores programs capable of executing the contents of the aforementioned embodiments, and the processors 602 can execute the programs stored in the memory 604.

[0132] The processor 602 may include one or more processing cores. The processor 602 connects to various parts within the electronic device 600 using various interfaces and lines, and executes various functions of the server 600 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 604, and by calling data stored in the memory 604. Optionally, the processor 602 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 602 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 602 and may be implemented separately using a communication chip.

[0133] The memory 604 may include random access memory (RAM) or read-only memory (ROM). The memory 604 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 604 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 600 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0134] The network module 606 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby communicating with communication networks or other devices, such as audio playback devices. The network module 606 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM cards), memory, etc. The network module 606 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices through wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). For example, the network module 606 can interact with base stations.

[0135] Please refer to Figure 10 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 700 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0136] The computer-readable storage medium 700 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has storage space for program code 710 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 710 may be compressed, for example, in a suitable form.

[0137] This application provides a liveness detection method, apparatus, electronic device, and storage medium. The liveness detection method includes: acquiring an image to be detected; inputting the image to be detected into a pre-trained liveness detection model to acquire facial features and liveness features corresponding to the image output by the liveness detection model; acquiring preset liveness features corresponding to the facial features; and determining a liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features. Through this method, by inputting the image to be detected into a liveness detection model to obtain facial features and liveness features, performing face recognition on the facial features, obtaining preset liveness features if face recognition is successful, and comparing the preset liveness features with the liveness features to determine whether a person is alive, the accuracy of liveness detection is improved, and the user experience is enhanced.

[0138] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other modifications under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these modifications are protected by the present invention.

Claims

1. A method for detecting liveness, characterized in that, Applied to electronic devices, the method includes: Acquire the image to be detected; The process of acquiring the image to be detected includes: Train the liveness detection model to be trained; The liveness detection model to be trained includes an image feature extractor, a face feature extractor, and a liveness feature extractor. The image feature extractor includes five sub-modules connected sequentially, with the output of the preceding sub-module serving as the input of the following sub-module. The face feature extractor includes a convolutional layer, a tensor flattening module, a neural network overfitting prevention module, and a fully connected layer. The liveness feature extractor includes a feature fusion layer, a convolutional layer, a pooling layer, a first fully connected layer, and a second fully connected layer. The liveness feature extractor acquires image features from the third, fourth, and fifth sub-modules. These features are then fused through a feature fusion layer, and the fused features are convolved through a convolutional layer. The calculated result is then input into a pooling layer for data compression. The compressed result is then input into a first fully connected layer to obtain the triplet loss. The result from the first fully connected layer is then input into a second fully connected layer to obtain the acquired features and the liveness binary classification loss. Simultaneously, the data output from the first fully connected layer is input into a gradient inversion layer, and the output data from the gradient inversion layer is input into a discriminator to obtain the adversarial loss. The face feature extractor obtains the image features from the last sub-module of the image feature extractor, inputs the obtained image features into the convolutional layer for convolution calculation, performs tensor flattening operation on the convolution calculation result through the tensor flattening module, processes it through the neural network overfitting prevention module, and transmits the processing result to the fully connected layer. The fully connected layer outputs the face recognition loss and face features. The image to be detected is input into a pre-trained liveness detection model to obtain the facial features and liveness features corresponding to the image to be detected output by the liveness detection model. Obtain the preset liveness features corresponding to the facial features; Based on the liveness features and the preset liveness features, the liveness detection result corresponding to the image to be detected is determined.

2. The method according to claim 1, characterized in that, The process involves inputting the image to be detected into a pre-trained liveness detection model to obtain the facial features and liveness features corresponding to the image to be detected, as output by the liveness detection model, including: The image to be detected is input into the image feature extraction module to obtain the image features corresponding to the image to be detected output by the image feature extraction module; The image features are input into the face feature extraction module to obtain the face features corresponding to the image to be detected, which are output by the face feature extraction module. The image features are input into the liveness feature extraction module, and the liveness features corresponding to the image to be detected are obtained from the output of the liveness feature extraction module.

3. The method according to claim 2, characterized in that, The step of inputting the image features into the face feature extraction module to obtain the face features corresponding to the image to be detected output by the face feature extraction module includes: Obtain the output of the last submodule among the five submodules; The output of the last submodule is input to the face feature extraction module to obtain the face features corresponding to the image to be detected output by the face feature extraction module.

4. The method according to claim 3, characterized in that, The step of inputting the image features into the liveness feature extraction module and obtaining the liveness features corresponding to the image to be detected output by the liveness feature extraction module includes: Obtain the output of a preset number of submodules from the five submodules; The outputs of the preset number of sub-modules are input to the liveness feature extraction module, and the liveness features corresponding to the image to be detected are obtained from the output of the liveness feature extraction module.

5. The method according to claim 1, characterized in that, The step of obtaining the preset liveness feature corresponding to the facial feature includes: Perform facial recognition on the facial features to obtain the corresponding facial recognition results; If the face recognition result indicates that the face recognition was successful, the preset liveness feature corresponding to the face feature is obtained.

6. The method according to claim 1, characterized in that, The step of determining the liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features includes: Obtain the similarity between the liveness feature and the preset liveness feature; If the similarity exceeds a preset similarity, the liveness detection result corresponding to the image to be detected is determined to be a live object.

7. A liveness detection device, characterized in that, Operating in an electronic device, the device includes: A model training unit is used to train a liveness detection model to be trained. The liveness detection model to be trained includes an image feature extractor, a face feature extractor, and a liveness feature extractor. The image feature extractor includes five sub-modules connected sequentially, with the output of the preceding sub-module serving as the input of the following sub-module. The face feature extractor includes a convolutional layer, a tensor flattening module, a neural network overfitting prevention module, and a fully connected layer. The liveness feature extractor includes a feature fusion layer, a convolutional layer, a pooling layer, a first fully connected layer, and a second fully connected layer. The liveness feature extractor acquires image features from the third, fourth, and fifth sub-modules, fuses these features through the feature fusion layer, and then processes the fused features... The convolutional layer performs convolution calculations, and the results are then input into the pooling layer for data compression. The compressed results are then input into the fully connected layer to obtain the triplet loss. The results from the fully connected layer are then input into the fully connected layer to obtain the feature acquisition and liveness binary classification loss. Simultaneously, the data output from the first fully connected layer is input into the gradient inversion layer, and the output data from the gradient inversion layer is input into the discriminator to obtain the adversarial loss. The face feature extractor acquires the image features from the last submodule of the image feature extractor, inputs the acquired image features into the convolutional layer for convolution calculations, and performs tensor flattening operations on the convolution calculation results through the tensor flattening module. Then, it is processed by the neural network overfitting prevention module, and the processed results are transmitted to the fully connected layer. The fully connected layer outputs the face recognition loss and face features. The image acquisition unit is used to acquire the image to be detected. The face feature and liveness feature acquisition unit is used to input the image to be detected into a pre-trained liveness detection model and acquire the face features and liveness features corresponding to the image to be detected output by the liveness detection model. A preset liveness feature acquisition unit is used to acquire preset liveness features corresponding to the facial features; The liveness detection result determination unit is used to determine the liveness detection result corresponding to the image to be detected based on the liveness features and the preset liveness features.

8. An electronic device, characterized in that, It includes one or more processors and memory, wherein one or more programs are stored in the memory and configured to be executed by one or more processors according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, which includes instructions for performing the method as claimed in any one of claims 1-6.

Images