Living body detection model training method, application method, equipment and medium

Abstract

The invention provides a living body detection model training method, an application method, equipment and a medium. According to the embodiment of the invention, the method comprises the steps: carrying out the enhancement processing of a plurality of different data according to a living body training picture, obtaining a plurality of enhanced images, carrying out the image feature extraction of the plurality of enhanced images, obtaining a plurality of image features, determining the image feature invariance loss function value between the plurality of image features, and obtaining the image feature invariance loss function value. The loss is added in the loss calculation of the whole model, so that the point that the living body judgment result is irrelevant to the change of the external environment such as illumination change and shooting environment is considered, the influence of the change of the external environment such as illumination or shooting environment on model training is reduced, the accuracy of living body detection is improved, and the stability of the detection result is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of liveness detection, and in particular to a training method, application method, device and medium for a liveness detection model. Background Art

[0002] Face liveness detection plays a crucial role in ensuring facial recognition security. Related technologies can directly use the RGB sensor used to capture facial images for liveness detection (i.e., RGB liveness detection), or they can be based on additional sensor information, such as infrared information and depth information.

[0003] However, due to the influence of lighting changes and changing shooting environments during the actual face collection process, the accuracy of RGB liveness detection will be reduced and the stability of the detection results cannot be guaranteed. Summary of the Invention

[0004] This application provides a training method, application method, device and medium for a liveness detection model, aiming to improve the accuracy of liveness detection and ensure the stability of detection results.

[0005] In one aspect, the present application provides a method for training a liveness detection model, the method comprising:

[0006] Based on the in vivo training images, multiple different data enhancement processes are performed to obtain multiple enhanced images;

[0007] Extracting image features from the multiple enhanced images to obtain multiple image features;

[0008] Determining a picture feature invariance loss function value between the plurality of picture features;

[0009] Based on the image feature invariance loss function value and the loss function value of the liveness judgment result, the model parameters of the to-be-trained model are tuned to obtain a liveness detection model, wherein at least one of the image features is input into the liveness detection classification layer in the to-be-trained model to obtain the liveness judgment result output by the liveness detection classification layer.

[0010] In a possible implementation of the present application, the model parameter tuning of the training model based on the image feature invariance loss function value and the liveness judgment result loss function value includes:

[0011] Inputting at least one of the image features into a non-live attack type classification layer in the model to be trained, and obtaining a non-live attack type determination result output by the non-live attack type classification layer; and / or inputting at least one of the image features into a facial attribute analysis layer in the model to be trained, and obtaining a facial attribute determination result output by the facial attribute analysis layer;

[0012] Based on the loss function value of the non-live attack type judgment result and / or the face attribute judgment result, as well as the image feature invariance loss function value and the loss function value of the liveness judgment result, the model parameters of the training model are tuned.

[0013] In a possible implementation of the present application, the plurality of different data enhancement processes are performed on the living body training images, including:

[0014] Determine the face area in the live training image;

[0015] Expanding the face area to obtain an expanded area;

[0016] A plurality of different data enhancement processes are performed on the expanded areas.

[0017] In a possible implementation of the present application, the model parameter tuning of the training model based on the image feature invariance loss function value and the liveness judgment result loss function value includes:

[0018] Inputting at least one of the image features into a non-living attack visible to human eye clue classification layer in the to-be-trained model, and obtaining a visible to human eye clue judgment result output by the non-living attack visible to human eye clue classification layer; and / or inputting at least one of the image features into a facial attribute analysis layer in the to-be-trained model, and obtaining a facial attribute judgment result output by the facial attribute analysis layer;

[0019] Based on the loss function value of the judgment result of whether there are visible clues to the human eye and / or the loss function value of the facial attribute judgment result, as well as the loss function value of the image feature invariance and the loss function value of the liveness judgment result, the model parameters of the training model are tuned.

[0020] In a possible implementation of the present application, each data enhancement process includes a random brightness adjustment process and / or a random motion blur process;

[0021] In a plurality of different data enhancement processes, the brightness adjustment range of the random brightness adjustment process is different, and / or the motion blur degree of the random motion blur process is different.

[0022] On the other hand, the present application provides an application method of a liveness detection model, the application method of the liveness detection model comprising:

[0023] Inputting the image to be detected into any one of the liveness detection models, and receiving the liveness determination result of the image to be detected output by the liveness detection model;

[0024] According to the liveness judgment result of the image to be detected, it is determined whether there is a live human face in the image to be detected.

[0025] In a possible implementation of the present application, the liveness determination result of the image to be detected includes the first liveness determination result of the image to be detected output by any one of the liveness detection models and the second liveness determination result of the image to be detected output by the other one of the liveness detection models;

[0026] The determining whether there is a living face in the image to be detected according to the liveness determination result of the image to be detected includes:

[0027] performing weighted fusion processing on the first liveness determination result and the second liveness determination result of the image to be detected to obtain a third liveness determination result of the image to be detected;

[0028] Determine whether there is a living human face in the image to be detected according to the third living body judgment result of the image to be detected.

[0029] In a possible implementation of the present application, there are multiple images to be detected that are collected continuously at the same time. After determining whether there is a living face in the image to be detected based on the liveness determination result of the image to be detected, the method further includes:

[0030] Determine a comprehensive result of living faces based on whether there are living faces in the multiple continuously collected pictures to be detected.

[0031] On the other hand, the present application provides a training device for a liveness detection model, comprising:

[0032] The data enhancement module is used to perform multiple different data enhancement processes based on the living body training images to obtain multiple enhanced images;

[0033] A feature extraction module is used to extract image features from the multiple enhanced images to obtain multiple image features;

[0034] A loss function module, configured to determine a picture feature invariance loss function value between the plurality of picture features;

[0035] A parameter tuning module is used to tune the model parameters of the to-be-trained model based on the image feature invariance loss function value and the loss function value of the liveness judgment result to obtain a liveness detection model, wherein the liveness judgment result output by the liveness detection classification layer is obtained by inputting at least one of the image features into the liveness detection classification layer in the to-be-trained model.

[0036] On the other hand, the present application provides an application device of a liveness detection model, comprising:

[0037] An image input module, configured to input an image to be detected into any one of the liveness detection models, and receive a liveness determination result of the image to be detected output by the liveness detection model;

[0038] The result determination module is used to determine whether there is a living face in the image to be detected according to the liveness judgment result of the image to be detected.

[0039] On the other hand, the present application also provides a liveness detection device, the liveness detection device comprising:

[0040] one or more processors;

[0041] Memory; and

[0042] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement any of the training methods for the liveness detection model or any of the application methods for the liveness detection model.

[0043] On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in any one of the training methods for the liveness detection model or any one of the application methods for the liveness detection model.

[0044] On the other hand, an embodiment of the present application provides a computer program product, including a computer program or instructions, which are executed by a processor to implement any of the training methods for the liveness detection model or any of the application methods for the liveness detection model.

[0045] The training method, application method, device and medium of the liveness detection model provided in the embodiments of the present application perform multiple different data enhancement processing on the liveness training images to obtain multiple enhanced images, then perform image feature extraction on the multiple enhanced images to obtain multiple image features, and then determine the image feature invariance loss function value between the multiple image features, and add this loss to the overall loss calculation of the model, thereby taking into account the fact that the liveness judgment result is unrelated to changes in external environments such as lighting changes and shooting environment, reducing the impact of changes in external environments such as lighting or shooting environment on model training, improving the accuracy of liveness detection, and ensuring the stability of detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 This is a flow chart of an embodiment of a method for training a liveness detection model provided in an embodiment of the present application;

[0048] Figure 2 This is a flow chart of an embodiment of the method for applying the liveness detection model provided in the embodiments of the present application;

[0049] Figure 3 Schematic diagram of the comprehensive process of the training method and application method of the liveness detection model provided in the embodiments of the present application;

[0050] Figure 4 Schematic diagram of the liveness detection model provided in the embodiments of the present application;

[0051] Figure 5 This is a schematic diagram of the structure of an embodiment of a liveness detection device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0053] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically specified.

[0054] In this application, the phrase "in some embodiments" is used to mean "serving as an example, illustration or illustration". Any embodiment described in this application as "in some embodiments" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0055] It should be noted that since the system of the embodiment of the present application is executed in a liveness detection device, the processing objects of the liveness detection device are all in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, the corresponding data exist so that the liveness detection device can process them. The details will not be repeated here.

[0056] In related technologies, face liveness detection plays a crucial role in ensuring facial recognition security. For example, when using facial recognition in access control systems for unlocking doors or clocking in attendance, forgeries (non-liveness attacks) can occur. RGB liveness detection requires no additional sensor hardware and is relatively low-cost. It can independently determine facial liveness or combine it with infrared or depth information to ensure facial recognition security.

[0057] However, due to the influence of lighting variations, changing shooting environments, and varying attack types during actual face acquisition, the accuracy of RGB face liveness detection needs to be improved. Furthermore, when capturing videos, the effects of shooting angle, lighting variations, and slight facial rotation can affect the stability of liveness detection results for faces in the same video using the same detection method.

[0058] In this regard, the embodiments of the present application provide a training method, application method, device and medium for a liveness detection model, which can be applied to scenarios such as identity authentication in the financial payment field, security authentication in the public field (such as security authentication when riding a high-speed rail, security authentication when checking in a hotel, etc.), and face recognition authentication in the smart terminal field (such as face unlocking of smart phones, face recognition authentication of access control products, etc.). Access control products include access control machines, so the liveness detection device may include devices in any of the above scenarios. Detailed descriptions are given below.

[0059] In one embodiment, referring to Figure 1 ,The training methods of the liveness detection model include:

[0060] 101. Based on the living body training images, multiple different data enhancement processes are performed to obtain multiple enhanced images.

[0061] In an embodiment of the present application, a live training image is an image in a live training data set, which is used for model training to obtain a live detection model. A plurality of different enhanced images are obtained by performing a plurality of different data augmentation processes on the live training images. The live training images may be RGB (red, green, blue) images, and RGB may be acquired by a corresponding RGB sensor. The live body may include a live human face, that is, the live detection model is used to detect a live human face.

[0062] 102. Perform image feature extraction on the multiple enhanced images to obtain multiple image features.

[0063] In an embodiment of the present application, image feature extraction can be implemented using a feature extraction network in a model to be trained. The feature extraction network may include structures such as convolutional layers, thereby implementing the feature extraction function using structures such as convolutional layers.

[0064] 103. Determine a value of an image feature invariance loss function between multiple image features.

[0065] In the embodiments of this application, the image feature invariance loss function refers to a loss function used to measure the degree of image feature variation between multiple image features. Considering that the liveness determination result is unrelated to changes in external conditions such as illumination and the shooting environment, the image features between multiple image features should also be the same. Therefore, determining the value of the image feature invariance loss function between multiple image features is equivalent to determining the error in the liveness determination result caused by changes in external conditions such as illumination and the shooting environment.

[0066] 104. Based on the image feature invariance loss function value and the liveness judgment result loss function value, the model parameters of the to-be-trained model are tuned to obtain a liveness detection model, wherein at least one image feature is input into the liveness detection classification layer in the to-be-trained model to obtain the liveness judgment result output by the liveness detection classification layer.

[0067] In the embodiments of the present application, the loss function of the liveness judgment result refers to a loss function used to measure the degree of difference between the liveness judgment result and the corresponding label information of the liveness training picture. Among them, the liveness judgment result is the result actually output by the model to be trained, and the corresponding label information of the liveness training picture is the result expected to be output by the model to be trained (i.e., the standard result). When tuning the model parameters of the training model, the loss function value of the image feature invariance and the loss function value of the liveness judgment result are comprehensively considered to eliminate the influence of changes in external environment such as lighting or shooting environment on model training, thereby improving the accuracy of the results of the liveness detection model.

[0068] In some embodiments of the present application, based on the image feature invariance loss function value and the loss function value of the liveness judgment result, the model parameters of the training model are tuned, which can include: weighted summing the image feature invariance loss function value and the loss function value of the liveness judgment result to obtain a comprehensive loss function value, wherein the weights in the weighted summation can be set based on actual needs; based on the comprehensive loss function value, the model parameters of the training model are tuned, and the model parameter tuning strategy can be, for example, minimizing the comprehensive loss function value.

[0069] It can be seen that in the technical solution disclosed in this embodiment, multiple different data enhancement processes are performed on the living body training pictures to obtain multiple enhanced images, and then image feature extraction is performed on the multiple enhanced images to obtain multiple image features. Then, the image feature invariance loss function value between the multiple image features is determined, and the loss is added to the overall loss calculation of the model, thereby taking into account the fact that the living body judgment result is unrelated to changes in external environments such as lighting changes and shooting environment, reducing the impact of changes in external environments such as lighting or shooting environment on model training, improving the accuracy of living body detection, and ensuring the stability of detection results.

[0070] In some embodiments of the present application, based on the image feature invariance loss function value and the liveness judgment result loss function value, the model parameter tuning of the to-be-trained model may include: inputting at least one image feature into the non-liveness attack type classification layer in the to-be-trained model to obtain the non-liveness attack type judgment result output by the non-liveness attack type classification layer, and / or, inputting at least one image feature into the face attribute analysis layer in the to-be-trained model to obtain the face attribute judgment result output by the face attribute analysis layer, wherein the non-liveness attack type may include, for example, photo printing, video playback, masks, etc., and the face attributes may include, for example, age, gender, hairstyle, expression, etc.; based on the loss function value of the non-liveness attack type judgment result and / or the face attribute judgment result, as well as the image feature invariance loss function value and the liveness judgment result loss function value, the model parameter tuning of the to-be-trained model may be performed. It can be seen that when tuning the model parameters of the training model, the embodiment of the present application not only combines the image feature invariance loss function value and the loss function value of the liveness judgment result, but also combines the loss function value of the non-liveness attack type judgment result and / or the face attribute judgment result, so that the result accuracy of the liveness detection model is higher.

[0071] In addition, when combining the image feature invariance loss function value, the liveness judgment result loss function value, and the non-liveness attack type judgment result and / or the face attribute judgment result loss function, a comprehensive loss function value can also be obtained by weighted summation. Based on the comprehensive loss function value, the model parameters of the training model are tuned. The specific details are not repeated here.

[0072] In some embodiments of the present application, considering that non-liveness training images often contain obvious attack clues visible to the human eye, such as information surrounding the face, such as a handheld device, a handheld photo, or a noticeable screen border, expanding the face area before performing model training can improve the ability to detect this type of attack. Specifically, based on the liveness training images, multiple different data augmentation processes are performed, which may include: determining the face area in the liveness training images; expanding the range of the face area to obtain the expanded area; and performing multiple different data augmentation processes on the expanded area, thereby improving the accuracy of liveness detection.

[0073] In some embodiments of the present application, the facial region may be expanded by proportional expansion. For example, if the width of the facial region is w and the height is h, the facial region may be expanded by w / 6 on both sides and h / 8 on both top and bottom sides.

[0074] In some embodiments of the present application, since the expanded area can be used to detect whether obvious clues of attacks visible to the human eye appear, the function of the above-mentioned non-living attack type classification layer and the corresponding label information of the expanded area in the non-living attack type classification layer will also be adjusted accordingly. For example, the above-mentioned non-living attack type classification layer can be replaced by a classification layer for whether there are clues of non-living attacks visible to the human eye. The corresponding label information of the expanded area in the classification layer for whether there are clues of non-living attacks visible to the human eye may include: non-living attack with clues visible to the human eye, non-living attack without clues visible to the human eye, etc.

[0075] Accordingly, based on the image feature invariance loss function value and the liveness judgment result loss function value, the model parameter tuning of the to-be-trained model can include: inputting at least one image feature into the non-living attack visible clue classification layer in the to-be-trained model to obtain the non-living attack visible clue classification layer outputs the human eye visible clue judgment result, and / or, inputting at least one image feature into the facial attribute analysis layer in the to-be-trained model to obtain the facial attribute judgment result output by the facial attribute analysis layer; based on the loss function value of the human eye visible clue judgment result and / or the facial attribute judgment result, as well as the image feature invariance loss function value and the liveness judgment result loss function value, the model parameter tuning of the to-be-trained model is performed.

[0076] In addition, when combining the loss function value of image feature invariance, the loss function value of liveness judgment results, and the loss function of whether there are visible clues to the human eye and / or the loss function of facial attribute judgment results, a comprehensive loss function value can also be obtained by weighted summation. Based on the comprehensive loss function value, the model parameters of the training model are tuned. The specific details are not repeated here.

[0077] In some embodiments of the present application, each data enhancement process may include random brightness adjustment processing and / or random motion blur processing. Random brightness adjustment processing refers to randomly adjusting the overall brightness of the image according to a set random brightness range and a set random adjustment probability. Random motion blur processing refers to performing motion blur processing on the image according to a set motion blur degree and a set random processing probability. In order to make the multiple data enhancement processes performed on the in vivo training images different, in the multiple data enhancement processes, the brightness adjustment range of the random brightness adjustment process is different, and / or the motion blur degree of the random motion blur process is different. In this way, the enhanced images obtained after different data enhancement processes will also be different to simulate changes in external environments such as lighting and shooting conditions.

[0078] In one embodiment, referring to Figure 2 ,The application methods of the liveness detection model include:

[0079] 201. Input the image to be detected into a liveness detection model, and receive a liveness determination result of the image to be detected output by the liveness detection model.

[0080] 202. Determine whether there is a living human face in the image to be detected based on the liveness determination result of the image to be detected.

[0081] In an embodiment of the present application, the liveness detection model is the liveness detection model in any of the above-mentioned liveness detection model training methods. By inputting the image to be detected into the liveness detection model, the liveness determination result of the image to be detected output by the liveness detection model can be obtained. Specifically, the liveness determination result of the image to be detected can be output by the liveness detection classification layer in the liveness detection model, while the non-liveness attack type classification layer, facial attribute analysis layer, and whether there are visible clues to the non-liveness attack in the liveness detection model are used to assist in the model training of the liveness detection model.

[0082] In some embodiments of the present application, the liveness determination result of the image to be detected may include a liveness probability value p live and / or non-living probability value p spoof , where p live +p spoof =1, and p live 、p spoof The value range of is [0, 1].

[0083] In some embodiments of the present application, according to the liveness judgment result of the image to be detected, determining whether there is a live face in the image to be detected may include: if the liveness probability value p is not spoof If the probability value p is greater than the set threshold (for example, 0.5), it is determined that there is no living face in the image to be detected; if the non-living face probability value p is greater than the set threshold (for example, 0.5), it is determined that there is no living face in the image to be detected; spoof If the value is less than or equal to a set threshold (for example, 0.5), it is determined that there is a living face in the image to be detected.

[0084] In some embodiments of the present application, the liveness determination result of the image to be detected may include a first liveness determination result and a second liveness determination result for the image to be detected, wherein the first liveness determination result is output by a liveness detection model that includes a non-liveness attack type classification layer, and the second liveness determination result is output by a liveness detection model that includes a non-liveness attack type classification layer and a non-liveness attack visible clue classification layer. It can be seen that the liveness detection model that includes a non-liveness attack type classification layer and the liveness detection model that includes a non-liveness attack visible clue classification layer have different model training processes, and therefore the output first liveness determination result and second liveness determination result may also be different.

[0085] Accordingly, determining whether there is a live face in the image to be detected based on the liveness judgment result of the image to be detected can include: performing weighted fusion processing on the first liveness judgment result and the second liveness judgment result of the image to be detected to obtain a third liveness judgment result of the image to be detected; and determining whether there is a live face in the image to be detected based on the third liveness judgment result of the image to be detected, thereby making the detection result of whether there is a live face in the image to be detected more accurate.

[0086] In some embodiments of the present application, the first liveness determination result of the image to be detected may include a liveness probability value p live and / or non-living probability value p spoof The second liveness judgment result of the image to be detected may include a liveness probability value p live-e and / or non-living probability value p spoof-e Performing weighted fusion processing on the first liveness judgment result and the second liveness judgment result of the image to be detected to obtain a third liveness judgment result of the image to be detected may include: performing weighted fusion processing on the liveness probability value p in the first liveness judgment result live , the liveness probability value p in the second liveness determination result live-e Perform weighted fusion processing to obtain a fused liveness probability value, and / or, perform weighted fusion processing on the non-liveness probability value p in the first liveness judgment result. spoof , the non-live probability value p in the second live body judgment result spoof-e Perform weighted fusion processing to obtain a fused non-live probability value, thereby obtaining a third liveness determination result for the image to be detected. The third liveness determination result may include the fused liveness probability value and / or the fused non-liveness probability value.

[0087] In some embodiments of the present application, determining whether there is a live face in the image to be detected based on the third liveness judgment result of the image to be detected may include: if the fused non-liveness probability value is greater than a set threshold (for example, 0.5), determining that there is no live face in the image to be detected; if the fused non-liveness probability value is less than or equal to the set threshold (for example, 0.5), determining that there is a live face in the image to be detected.

[0088] In some embodiments of the present application, there are multiple images to be detected that are collected continuously at the same time. For example, when an RGB sensor collects a face video, it can obtain N consecutive images to be detected for liveness detection. Accordingly, after determining whether there is a live face in the image to be detected based on the liveness judgment result of the image to be detected, it can also include: determining a comprehensive result of live faces based on whether there are live faces in the multiple images to be detected that are collected continuously, so as to make the detection of live faces more accurate. For example, among the multiple images to be detected that are collected continuously, if the number of images to be detected that contain live faces is greater than a preset ratio, then the comprehensive result of live faces is determined to be that the current face is a live face; if the number of images to be detected that contain live faces is less than or equal to the preset ratio, then the comprehensive result of live faces is determined to be that the current face is a non-live face. For example, when there are 5 consecutive images to be detected, if there is no live face in more than half of the images to be detected (for example, 3 images to be detected), the comprehensive result of live faces is considered to be that the current face is not a live face; if there is no live face in more than half of the images to be detected (for example, 2 images to be detected), the comprehensive result of live faces is considered to be that the current face is a live face.

[0089] In some embodiments of the present application, examples are given for the training method of the above liveness detection model and the application method of the liveness detection model, which may specifically include implementation plan 1 and implementation plan 2.

[0090] Implementation plan 1: obtain a live training data set of RGB faces, perform data enhancement processing based on the live training pictures in the live training data set, input the enhanced data into the deep learning network (i.e. the model to be trained) for training, and obtain a live detection model; obtain multiple pictures to be detected, where the pictures to be detected are multiple consecutive face pictures in the video captured by the camera, input each picture to be detected into the live detection model, obtain the live judgment result of the corresponding picture to be detected, and make a comprehensive judgment on the live judgment results of the multiple pictures to be detected to obtain a comprehensive result of live faces. The specific steps are as follows: Figure 3 As shown, including:

[0091] Step 301: Obtain a live training image from a live training dataset of RGB faces.

[0092] Step 302: Select the face area in the living body training image as input.

[0093] Step 303: Perform data enhancement processing on the input face area, wherein the data enhancement processing includes random brightness adjustment processing and random motion blur processing. Step 303 may include:

[0094] Step 303-a, data enhancement (first, randomly adjust the overall brightness of the input face area, and the range of random adjustment is set to br. Then, the brightness after adjustment is a random brightness in the range of [(1-br)B, (1+br)B], where B is the original brightness of the input face area, and the value range of br is [0, 1]. For example, br can be set to 0.2; the value range of the random adjustment probability pb is [0, 1], for example, pb can be set to 0.3).

[0095] Step 303 - b , random motion blur processing (also set with motion blur degree and random processing probability).

[0096] Step 304: Select a deep learning network to extract image features. The network for image feature extraction can be consistent with the MobileNet series network. When performing model training, load the MobileNet pre-trained model and initialize the model parameters. After image feature extraction, the output image feature F1 is obtained.

[0097] Step 305: Input the image feature F1 into the liveness detection classification layer, the non-liveness attack type classification layer, and the facial attribute analysis layer respectively. The implementation of each layer is the fully connected layer FC of the neural network, and the corresponding liveness judgment results, non-liveness attack type judgment results, and facial attribute judgment results can be obtained.

[0098] Step 306: Repeat steps 303 and 304 to ensure that the data enhancement processing processes for the same input face area are not exactly the same, that is, to ensure that the brightness after random brightness adjustment processing is different or the degree of motion blur after random motion blur processing is different, and then select the same deep learning network to extract image features to obtain the output image feature F2.

[0099] Step 307: Based on the liveness determination results, the non-liveness attack type determination results, the face attribute determination results, and the label information of the corresponding liveness training images, the model parameters are tuned in combination with the loss function settings. Step 307 may include:

[0100] Step 307a: When optimizing the model parameters, the loss function L for the liveness judgment result is s Using the Softmax Loss loss function, the loss function L for the non-living attack type judgment result is st Using the cross-entropy loss function, the loss function L for the face attribute judgment result is a The loss function used is Binary Cross-Entropy.

[0101] Step 307b: Considering whether the input face area is alive or not, regardless of changes in external environments such as lighting and shooting conditions, add the image feature invariance loss function L sim , L sim The mean square error (MSE) loss function can be used. Specifically, the image features F1 and F2 in steps 304 and 306 can be used to calculate the image feature invariance loss function L sim .

[0102] Step 307c: The final comprehensive loss function value L is calculated as follows: L = c * L s +c1*L st +c2*L a +c3*L sim , where c, c1, c2, and c3 can be set to 1.0, 0.1, 1.0, and 1.0, respectively.

[0103] Step 308: According to the above steps, a liveness detection model is obtained by training. Figure 4 shown.

[0104] In step 309, when the RGB sensor captures a face video, it obtains N consecutive images to be detected for detection. Since access control products and other liveness detection devices generally use a single face input when performing face liveness detection, step 309 uses images of the same face at adjacent moments in the video to make the judgment, which helps improve the stability of the liveness detection results. Step 309 may include:

[0105] Step 309a: For a single image to be detected, select the face area in the image and input it into the liveness detection model obtained in the above steps to obtain the liveness judgment result, which includes the liveness probability value p live and / or non-living probability value p spoof , where p live +p spoof =1, and p live 、p spoof The value range is [0, 1]. If the probability value p is not alive spoof If the probability value p is greater than the set threshold (for example, 0.5), it is determined that there is no living face in the image to be detected; if the non-living face probability value p is greater than the set threshold (for example, 0.5), it is determined that there is no living face in the image to be detected; spoof If the value is less than or equal to a set threshold (for example, 0.5), it is determined that there is a living face in the image to be detected.

[0106] Step 309b: Perform the judgment process of step 309a on all N samples to be detected. The value of N is determined by the detection time. For example, N can be selected as 5 while ensuring that the overall judgment time meets the system requirements. For example, when there are 5 consecutively collected images to be detected, if more than half of the images to be detected (for example, 3 images to be detected) do not contain a live face, then the comprehensive result of the live face is considered to be that the current face is a non-live face; if more than half of the images to be detected (for example, 2 images to be detected) do not contain a live face, then the comprehensive result of the live face is considered to be that the current face is a live face.

[0107] Implementation Plan 2:

[0108] The difference between Implementation Plan 2 and Implementation Plan 1 is that a liveness detection model with an expanded range of facial regions is retrained. The liveness detection model obtained in step 308 of Implementation Plan 1 is integrated to obtain a third liveness determination result for the image to be detected, thereby improving the generalization capability of liveness detection. The specific steps of Implementation Plan 2 are as follows:

[0109] Step 401, including steps 301 to 308 of implementation scheme 1, obtains a living body detection model M1.

[0110] Step 402, similar to step 301, expands the facial region in step 302 to obtain an expanded region. Considering that non-live facial images may contain obvious attack cues visible to the human eye, such as handheld devices, handheld photos, or obvious screen borders, expanding the facial region before model training can improve the ability to detect these types of attacks and increase the accuracy of liveness detection. The expanded region is used as input for the next step.

[0111] Step 403 is the same as step 303 and step 304 of implementation plan 1 and will not be repeated here.

[0112] Step 404: Input the image feature F1 into the liveness detection classification layer, the non-liveness attack visible clue classification layer, and the facial attribute analysis layer respectively. The implementation of each layer is the fully connected layer FC of the neural network, and the corresponding liveness judgment results, visible clue judgment results, and facial attribute judgment results can be obtained.

[0113] Step 405 and the subsequent implementation steps are essentially identical to Steps 306 through 308 of Implementation Plan 1 and are not further described. It should be noted that the loss function for determining the non-liveness attack type in Implementation Plan 1 has been replaced with the loss function for determining whether there are visible clues, also using the cross-entropy loss function. Model training is performed using the label information of the corresponding live training images to obtain the liveness detection model M2 with an expanded facial region.

[0114] Step 406: Referring to step 309, for a single face image to be detected, select the face region I in the image to be detected. f , perform the same face area expansion as in step 402 to obtain the expanded face area I fe . Will I fe Input into the liveness detection model M2, and obtain the second liveness judgment result (including the liveness probability value p live-e and the probability value of non-living body p spoof-e ); will I f Input into the liveness detection model M1, and obtain the first liveness judgment result (including the liveness probability value p live and the probability value of non-living body p spoof ).

[0115] Step 406 may further include:

[0116] Step 406a: Combine the first liveness judgment result output by the liveness detection model M1 and the second liveness judgment result output by the liveness detection model M2 to obtain a third liveness judgment result for the single image to be detected. The third liveness judgment result may include a fused liveness probability value and / or a fused non-liveness probability value, wherein the fused non-liveness probability value is: the non-liveness probability value p spoof-e and the probability value of non-living body p spoof The weighted summation results are as follows: the weights are 0.3 and 0.7 respectively; the fusion liveness probability value is: the liveness probability value p live-e and the probability of being alive p live The weighted summation results are calculated with weights of 0.3 and 0.7, respectively. If the fused non-live probability value is greater than a set threshold (e.g., 0.5), it is determined that there is no live face in the image to be detected, and the face in the image to be detected is an attacking face; if the fused non-live probability value is less than or equal to the set threshold (e.g., 0.5), it is determined that there is a live face in the image to be detected.

[0117] Step 406b is the same as step 309b and will not be described again.

[0118] As can be seen, after extracting image features, this example uses multiple classification and analysis layers to assist in model training. In addition to the liveness detection classification layer, it also utilizes a non-liveness attack type classification layer, a facial attribute analysis layer, and a classification layer to determine whether there are visible clues to the non-liveness attack. This helps to optimize the overall model training and improve the accuracy of the liveness detection model. Furthermore, during model training, image features after different data augmentations are used to generate an image feature invariance loss function. This loss function is incorporated into the overall model loss calculation to mitigate the impact of external environmental changes such as lighting and shooting environment on the training samples, improving the model's adaptability to different environments and, in turn, enhancing its generalization and accuracy. Furthermore, when using the liveness detection model for face liveness detection, the results of consecutive images are combined for judgment, improving the stability of the detection results. Furthermore, when performing face liveness detection, the combined results of two liveness detection models trained using different methods are used for judgment, further improving the accuracy of the detection results.

[0119] In order to better implement the training method of the liveness detection model in the embodiment of the present application, based on the training method of the liveness detection model, the embodiment of the present application also provides a training device for the liveness detection model, and the training device for the liveness detection model includes:

[0120] The data enhancement module is used to perform multiple different data enhancement processes based on the living body training images to obtain multiple enhanced images;

[0121] A feature extraction module is used to extract image features from the multiple enhanced images to obtain multiple image features;

[0122] A loss function module, used to determine the image feature invariance loss function value between multiple image features;

[0123] The parameter tuning module is used to tune the model parameters of the training model based on the image feature invariance loss function value and the loss function value of the liveness judgment result to obtain a liveness detection model, wherein at least one image feature is input into the liveness detection classification layer in the training model to obtain the liveness judgment result output by the liveness detection classification layer.

[0124] In order to better implement the application method of the liveness detection model in the embodiment of the present application, based on the application method of the liveness detection model, the embodiment of the present application also provides an application device of the liveness detection model, and the application device of the liveness detection model includes:

[0125] An image input module is used to input the image to be detected into any one of the liveness detection models and receive the liveness judgment result of the image to be detected output by the liveness detection model;

[0126] The result determination module is used to determine whether there is a living face in the image to be detected based on the liveness judgment result of the image to be detected.

[0127] The present application also provides a liveness detection device that integrates a training device for any liveness detection model or an application device for any liveness detection model provided in the present application. The liveness detection device can be used for any of the following scenarios: identity authentication in the financial payment field, security authentication in the public domain (such as security authentication when riding a high-speed train or checking into a hotel), and facial recognition authentication in the smart terminal field (such as face unlocking for smartphones or facial recognition authentication for access control products).

[0128] like Figure 5 , which shows a schematic structural diagram of a living body detection device involved in an embodiment of the present application, specifically:

[0129] The liveness detection device may include one or more processors 501, one or more computer-readable storage media memories 502, a power supply 503, an input unit 504, and other components. Those skilled in the art will appreciate that Figure 5 The structure of the liveness detection device shown in the figure is not intended to limit the liveness detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0130] The processor 501 is the control center of the liveness detection device. It connects the various components of the liveness detection device using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 502 and accessing data stored in the memory 502, it performs various functions of the liveness detection device and processes data, thereby providing overall monitoring of the liveness detection device. Optionally, the processor 501 may include one or more processing cores. Preferably, the processor 501 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 501.

[0131] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created based on the use of the liveness detection device, etc. In addition, the memory 502 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0132] The liveness detection device also includes a power supply 503 for supplying power to various components. Preferably, the power supply 503 can be logically connected to the processor 501 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 503 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0133] The liveness detection device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0134] Although not shown, the liveness detection device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the liveness detection device loads the executable files corresponding to one or more application processes into the memory 502 according to the following instructions, and the processor 501 runs the application stored in the memory 502, thereby implementing any of the above-described liveness detection model training methods or any of the above-described liveness detection model application methods.

[0135] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the training methods for a liveness detection model provided in the embodiments of the present application. For example, the computer program loaded by the processor can execute the steps in any of the training methods for a liveness detection model or any of the application methods for a liveness detection model.

[0137] Embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to implement any of the methods for training a liveness detection model or any of the methods for applying a liveness detection model.

[0138] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0139] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0140] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0141] The above is a detailed introduction to the training method, application method, equipment and medium of a liveness detection model provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A training method for a liveness detection model, characterized in that: The training method of the living body detection model includes: Based on the in vivo training images, multiple different data enhancement processes are performed to obtain multiple enhanced images; Extracting image features from the multiple enhanced images to obtain multiple image features; Determining a picture feature invariance loss function value between the plurality of picture features; Based on the image feature invariance loss function value and the loss function value of the liveness judgment result, the model parameters of the to-be-trained model are tuned to obtain a liveness detection model, wherein at least one of the image features is input into the liveness detection classification layer in the to-be-trained model to obtain the liveness judgment result output by the liveness detection classification layer.

2. The method for training a liveness detection model according to claim 1, wherein: The step of tuning the model parameters of the training model based on the image feature invariance loss function value and the liveness judgment result loss function value includes: Inputting at least one of the image features into a non-live attack type classification layer in the model to be trained, and obtaining a non-live attack type determination result output by the non-live attack type classification layer; and / or inputting at least one of the image features into a facial attribute analysis layer in the model to be trained, and obtaining a facial attribute determination result output by the facial attribute analysis layer; Based on the loss function value of the non-live attack type judgment result and / or the face attribute judgment result, as well as the image feature invariance loss function value and the loss function value of the liveness judgment result, the model parameters of the training model are tuned.

3. The training method of the living body detection model according to claim 1, characterized in that: The method performs multiple different data enhancement processes based on the live training images, including: Determine the face area in the live training image; Expanding the face area to obtain an expanded area; A plurality of different data enhancement processes are performed on the expanded areas.

4. The method for training a liveness detection model according to claim 3, wherein: The step of tuning the model parameters of the training model based on the image feature invariance loss function value and the liveness judgment result loss function value includes: Inputting at least one of the image features into a non-living attack visible to human eye clue classification layer in the to-be-trained model, and obtaining a visible to human eye clue judgment result output by the non-living attack visible to human eye clue classification layer; and / or inputting at least one of the image features into a facial attribute analysis layer in the to-be-trained model, and obtaining a facial attribute judgment result output by the facial attribute analysis layer; Based on the loss function value of the judgment result of whether there are visible clues to the human eye and / or the loss function value of the facial attribute judgment result, as well as the loss function value of the image feature invariance and the loss function value of the liveness judgment result, the model parameters of the training model are tuned.

5. The method for training a liveness detection model according to any one of claims 1 to 4, wherein: Each data enhancement process includes random brightness adjustment process and / or random motion blur process; In a plurality of different data enhancement processes, the brightness adjustment range of the random brightness adjustment process is different, and / or the motion blur degree of the random motion blur process is different.

6. An application method of a liveness detection model, characterized in that: The application method of the living body detection model includes: Inputting the image to be detected into the liveness detection model according to any one of claims 1 to 5, and receiving the liveness determination result of the image to be detected output by the liveness detection model; According to the liveness judgment result of the image to be detected, it is determined whether there is a live human face in the image to be detected.

7. The application method of the living body detection model according to claim 6, characterized in that: The liveness determination result of the image to be detected includes the first liveness determination result of the image to be detected output by the liveness detection model according to claim 2 and the second liveness determination result of the image to be detected output by the liveness detection model according to claim 4; The determining whether there is a living face in the image to be detected according to the liveness determination result of the image to be detected includes: performing weighted fusion processing on the first liveness determination result and the second liveness determination result of the image to be detected to obtain a third liveness determination result of the image to be detected; Determine whether there is a living human face in the image to be detected according to the third living body judgment result of the image to be detected.

8. The method for applying the living body detection model according to claim 6 or 7, wherein: There are a plurality of images to be detected that are collected continuously at the same time. After determining whether there is a living face in the image to be detected based on the liveness determination result of the image to be detected, the method further includes: Determine a comprehensive result of living faces based on whether there are living faces in the multiple continuously collected pictures to be detected.

9. A living body detection device, characterized in that: The living body detection device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the training method of the liveness detection model described in any one of claims 1 to 5 or the application method of the liveness detection model described in any one of claims 6 to 8.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the training method of the liveness detection model according to any one of claims 1 to 5 or the application method of the liveness detection model according to any one of claims 6 to 8.

Images