Face living body detection method, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the field of human-computer interaction, and discloses a human face living body detection method, electronic equipment and a storage medium. The method comprises the following steps: acquiring an infrared image and a natural light image; respectively performing face detection to obtain corresponding face areas; face quality judgment is carried out by combining the face areas of the two modals, and if the face quality does not pass, the face is directly judged to be a non-living body; if yes, inputting a trained infrared face living body model and a trained natural light face living body model respectively to obtain two living body probabilities; and carrying out weighted fusion on the two, and judging whether the human body is a living body or not according to a fusion result. Therefore, by combining quality judgment and multi-modal probability fusion, the detection efficiency is effectively improved while the in-vivo detection accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and in particular to a face liveness detection method, electronic device, and storage medium. Background Technology

[0002] Facial recognition technology has been widely used in access control systems, mobile payments, and smart terminal unlocking, and its security highly depends on the accuracy of facial liveness detection. Common attack methods in existing technologies include non-liveness attacks such as printed photos, video playback, and 3D masks or head models. To improve detection reliability, a single natural light (Red, Green, Blue, RGB) camera is insufficient for high-security scenarios, leading to the development of multimodal liveness detection solutions that combine infrared (IR) cameras.

[0003] However, near-infrared images are affected by factors such as shooting distance and light intensity, resulting in significant fluctuations in image quality, which directly impacts the accuracy of liveness detection. Furthermore, access control systems have high requirements for recognition efficiency; complex or computationally redundant liveness detection processes will negatively affect user experience. Traditional methods still have shortcomings in image quality adaptation, model generalization ability, and multimodal information fusion, necessitating a technical solution that can improve the accuracy of multimodal liveness detection while maintaining detection efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a face liveness detection method, electronic device, and storage medium, which improves the accuracy of liveness detection while effectively increasing detection efficiency.

[0005] To address the aforementioned technical problems, embodiments of this application provide a face liveness detection method, comprising: acquiring an infrared image to be detected and a corresponding natural light image; performing face detection on the infrared image and the natural light image respectively to obtain an infrared face region and a natural light face region; performing a joint face quality judgment based on the infrared face region and the natural light face region, and directly determining that the liveness detection fails when the judgment result is unsuccessful; when the face quality judgment is successful, inputting the infrared face region into a trained infrared face liveness model to obtain a first liveness probability, and inputting the natural light face region into a trained natural light face liveness model to obtain a second liveness probability; performing a weighted fusion of the first liveness probability and the second liveness probability, and determining whether it is a live face based on the fusion result.

[0006] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described face liveness detection method.

[0007] The embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described face liveness detection method.

[0008] In this embodiment, infrared and natural light images are acquired; face detection is performed separately to obtain corresponding face regions; the face quality is judged by combining the two modal face regions; if the quality fails, the face is directly determined to be non-live; if the quality passes, the trained infrared face liveness model and natural light face liveness model are input separately to obtain two liveness probabilities; the two probabilities are then weighted and fused, and the liveness is determined based on the fusion result. This application improves the accuracy of liveness detection while effectively increasing detection efficiency by combining quality judgment with multimodal probability fusion. Attached Figure Description

[0009] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0010] Figure 1 This is a flowchart of a face liveness detection method according to an embodiment of this application; Figure 2 This is a flowchart of a method for training an infrared face liveness model according to an embodiment of this application; Figure 3 This is a schematic diagram of the transmission relationship of image feature extraction according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to another embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable readers to better understand this application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0012] One embodiment of this application relates to a face liveness detection method, which can be applied to electronic devices equipped with dual cameras (RGB+IR) and a computing module. This can be an access control system terminal or any smart device with face recognition functionality, such as a smartphone, tablet, smart lock, or attendance machine. For ease of explanation, the following embodiments will refer to the aforementioned execution entity as a face liveness detection device. In this embodiment, infrared and natural light images are acquired; face detection is performed separately to obtain corresponding face regions; the face quality is judged by combining the two modal face regions; if the quality fails, the face is directly determined to be non-live; if the quality passes, the trained infrared face liveness model and natural light face liveness model are input respectively to obtain two liveness probabilities; the two probabilities are weighted and fused, and the liveness is determined based on the fusion result. This invention improves the accuracy of liveness detection while effectively increasing detection efficiency by combining quality judgment with multimodal probability fusion. The implementation details of the face liveness detection method in this embodiment are described below. The following content is only for ease of understanding and is not essential for implementing this solution.

[0013] like Figure 1 As shown, in step 101, the face liveness detection device acquires the infrared image to be detected and the corresponding natural light image; In step 102, the face liveness detection device performs face detection on the infrared image and the natural light image respectively to obtain the infrared face region and the natural light face region; In step 103, the face liveness detection device performs a joint judgment on face quality based on the infrared face region and the natural light face region. When the judgment result is "not passed", the liveness detection is directly determined to be "not passed".

[0014] In one example, there are many ways to jointly judge face quality based on the infrared face region and the natural light face region. One or more constraints can be given; if the pass condition is met, the face is considered passed; if the fail condition is met, the face is considered failed. The fail condition can be one of the following or any combination thereof: 1. Calculate the proportion of the infrared face region to the entire infrared image; if the proportion is lower than a preset threshold, the joint judgment result for face quality is "fail"; by judging the proportion of the infrared face region in the image, faces that are too small or of poor quality due to excessive distance are effectively filtered out, avoiding subsequent invalid calculations and improving system efficiency. Specifically: The width or height of the IR face region is detected. Since the camera's resolution and other settings are fixed, the proportion of the face region in the entire image within a certain recognition distance will remain within a certain range. This invention uses the width or height of the face region as a percentage of the overall image resolution. or The height ratio measures the imaging distance, such as or ,(in and All values ​​are preset values, and if 0.013 is taken, the face quality is judged to be unqualified.

[0015] 2. Calculate the overall brightness of the infrared face area; if the brightness is outside the preset range, the joint face quality judgment result is "failed"; by detecting whether the overall brightness of the infrared face area is within a reasonable range, image quality problems caused by abnormal lighting are eliminated, further improving the reliability of liveness detection. Specifically: Calculate the overall brightness value of the detected IR face region. If the brightness is within the set range (e.g., […]) , ], This is the lowest brightness value. If the brightness value is the highest, then the face is judged to have normal IR brightness; otherwise, the brightness detection fails.

[0016] 3. Perform facial landmark detection and pose correction on the face region under natural light; calculate the Euclidean distance between the corrected facial landmarks and the standard facial landmarks; when the Euclidean distance is greater than a preset threshold, the joint judgment result for face quality is "fail"; using natural light images for facial landmark pose correction and quality assessment can effectively identify low-quality faces caused by abnormal poses or image blurring, enhancing the robustness of the overall system. Specifically: Face quality is assessed using the corresponding RGB face region. Facial landmark detection methods are used to obtain RGB face landmarks. Face correction is then performed to obtain corrected facial landmarks. The Euclidean distance between the corrected facial landmarks and the standard facial landmarks is calculated. ,like Greater than the set threshold ( If the set experience value is used, the face quality is judged as failing.

[0017] Conversely, if the above-mentioned conditions for failure are not met, it is considered as passing.

[0018] In step 104, when the face quality judgment is passed, the face liveness detection device inputs the infrared face region into the trained infrared face liveness model to obtain the first liveness probability, and inputs the natural light face region into the trained natural light face liveness model to obtain the second liveness probability. In one example, the aforementioned infrared face liveness detection model was trained in the following manner: Figure 2 As shown, in step 201, a dataset for infrared face liveness training is obtained; the images in the dataset are subjected to data augmentation processing, which may be an enhancement operation to simulate the image quality degradation caused by different distances between the face and the camera.

[0019] In step 202, a deep learning network is trained using the data augmentation-processed dataset. The deep learning network includes a main feature extraction network and a multi-scale fusion auxiliary network for outputting auxiliary judgments on face distance categories. By introducing data augmentation methods that simulate different imaging distances, the infrared face liveness model is trained, improving the model's generalization ability to distant, low-quality IR images.

[0020] In one example, the dataset used for infrared face liveness training in step 201 is images containing only the face region.

[0021] In one example, the enhancement operation in step 201 above, which simulates the image quality degradation caused by different distances between the face and the camera, can include one of the following and any combination thereof: horizontal flipping, brightness and contrast changes, adding motion blur, adding camera sensor noise, etc.

[0022] In one example, the quality of facial images captured by an infrared camera is significantly affected by the distance to the face; the farther the face is from the IR camera and infrared light source, the worse the image quality. To better adapt the model to the image quality degradation of IR faces at greater distances, the enhancement operation in step 201 above, which simulates the image quality degradation caused by different distances between the face and the camera, may also include one or a combination of the following operations: random probability downsampling and then upsampling the image to its original size, and random probability image compression. By employing enhancement operations such as downsampling and upsampling, and image compression, the quality degradation in actual imaging is simulated, further enhancing the model's adaptability to complex scenes. Specifically: Downsampling enhancement involves downsampling the training image with random probability (probability range [0, 1], e.g., set to 0.5) according to a set ratio, then upsampling it to restore the original image size. The downsampling ratio is set to a certain range of reduction ratios, such as reducing it to [0.4, 0.6] times the original size. Then, the enhanced image undergoes random probability compression enhancement (probability range [0, 1], e.g., set to 0.5), with the compression quality set to a certain range of quality control values, ranging from (0, 100), e.g., a minimum compression quality of 0.6 and a maximum of 0.95. Subsequently, in step 202, the enhanced data is used as the model training input. A deep learning network is selected as the main image feature extraction network, and a multi-scale fusion network is selected as the auxiliary network for training.

[0023] In one example, the loss function used in training the infrared face liveness detection model is a joint loss function. This joint loss function is also used to optimize the liveness detection and face distance determination tasks. By simultaneously optimizing both liveness detection and face distance determination tasks using the joint loss function, the model's performance and stability in multi-task learning are improved.

[0024] In one example, such as Figure 3 As shown, after data augmentation, the image enters the main feature extraction network. The main feature extraction network in step 202 can be a commonly used deep learning image feature extraction network such as MobileNet or ResidualNet. The main feature output is passed through a fully connected layer (FC) to obtain the face liveness classification output. Taking MobileNetv3 as an example, the auxiliary network extracts the intermediate layer feature output after the third stage (stage 3) of MobileNetv3. This, combined with the main feature output after the completion of MobileNetv3, is used as the input to the multi-scale fusion network to obtain the face distance category output. Based on the above network outputs, dataset label information, and loss function calculation, the network is trained. The loss function is used for liveness detection. The softmax loss function is used for face distance determination. Using cross-entropy loss (ce loss), the final network training loss is: .in, for loss weights, for The loss weight.

[0025] This loss function is an example of the joint loss function mentioned above. It can also be used to optimize liveness detection and face distance determination tasks.

[0026] In step 104 above, the IR face image was input into the IR face liveness model to obtain the IR face liveness probability, i.e., the first liveness probability. RGB face images are input into a trained RGB face liveness model to obtain the RGB face liveness probability, i.e., the second liveness probability. Then proceed to step 105.

[0027] In step 105, the face liveness detection device performs a weighted fusion of the first liveness probability and the second liveness probability, and determines whether it is a live face based on the fusion result.

[0028] In one example, the formula for calculating the weighted fusion of the first and second liveness probabilities is as follows:

[0029] in, The first probability of survival. This represents the second probability of survival. The weighting coefficient corresponding to the first liveness probability. This is the weighting coefficient corresponding to the second liveness probability. For the fusion result; The above-mentioned determination of whether a face is a live face based on the fusion result can be as follows: if the fusion result is greater than a preset probability threshold, then it is determined to be a live face. Specifically, if... The probability is greater than the set threshold. (like If the value is set to 0.5, the currently detected face is considered a live face; otherwise, it is considered a non-live face. The above approach uses a weighted fusion method combined with dual-modal liveness probabilities to flexibly adjust the contribution of each modality, thereby improving the accuracy of the final liveness detection and the system's configurability.

[0030] In this embodiment, a method for judging face quality using both IR and RGB face regions is proposed. Face quality filtering is performed before face liveness detection, and faces with low quality are directly excluded from liveness detection, avoiding unnecessary calculations and improving the efficiency of liveness detection. The proposed face quality includes the region proportion of the IR face image (i.e., the distance between the face and the camera), the brightness range of the IR face image, and the quality of the RGB face image. Faces that meet the quality requirements are input into the face liveness model for joint judgment, avoiding inaccurate judgments by the face liveness model in special cases and improving the accuracy of face liveness detection. A face distance-assisted network is proposed to optimize the training method of the IR face liveness detection model, and data augmentation methods such as dynamic resolution adjustment are added to improve the problem of inaccurate IR face liveness detection caused by unclear IR face images when the face is far from the camera.

[0031] The steps described above are for clarity only. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0032] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0033] Another embodiment of this application relates to an electronic device, such as... Figure 4 As shown, it includes at least one processor 501; and a memory 502 communicatively connected to the at least one processor; wherein the memory 502 stores instructions executable by the at least one processor 501, the instructions being executed by the at least one processor 501 to enable the at least one processor 501 to perform the face liveness detection method as described above.

[0034] The memory 502 and processor 501 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 501 and memory 502 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 501 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 501.

[0035] Processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 502 can be used to store data used by processor 501 during operation.

[0036] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0037] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0038] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for detecting human face liveness, characterized in that, include: Acquire the infrared image to be detected and the corresponding natural light image; Face detection is performed on the infrared image and the natural light image respectively to obtain the infrared face region and the natural light face region; A joint judgment on face quality is performed based on the infrared face region and the natural light face region. If the judgment result is "failed", then the liveness detection is directly determined to be "failed". When the face quality judgment is passed, the infrared face region is input into the trained infrared face liveness model to obtain the first liveness probability, and the natural light face region is input into the trained natural light face liveness model to obtain the second liveness probability. The first liveness probability and the second liveness probability are weighted and fused, and the result is used to determine whether it is a live face.

2. The face liveness detection method according to claim 1, characterized in that, The joint judgment of face quality based on the infrared face region and the natural light face region includes: Calculate the proportion of the size of the infrared face region to the entire infrared image; When the ratio is lower than a preset threshold, the joint judgment result of the face quality is "fail".

3. The face liveness detection method according to claim 1, characterized in that, The joint face quality judgment based on the infrared face region and the natural light face region also includes: Calculate the overall brightness of the infrared face region; If the brightness is not within the preset brightness range, the joint judgment result of the face quality is "fail".

4. The face liveness detection method according to claim 1, characterized in that, The joint face quality judgment based on the infrared face region and the natural light face region also includes: Facial landmark detection and pose correction are performed on the natural light face area; Calculate the Euclidean distance between the corrected facial landmarks and the standard facial landmarks; When the Euclidean distance is greater than a preset threshold, the joint judgment result of the face quality is "fail".

5. The face liveness detection method according to claim 1, characterized in that, The infrared face liveness model was trained using the following method: Obtain the dataset for infrared face liveness detection training; The images in the dataset are subjected to data augmentation processing, which includes: enhancement operations to simulate the image quality degradation caused by different distances between the face and the camera; A deep learning network is trained using the data augmentation-processed dataset. The deep learning network includes a main feature extraction network and a multi-scale fusion auxiliary network for outputting auxiliary judgments on face distance categories.

6. The face liveness detection method according to claim 5, characterized in that, The enhancement operation used to simulate the image quality degradation caused by different distances between the face and the camera includes one or a combination of the following operations: The operation involves downsampling and upsampling the image to its original size using random probability, and compressing the image using random probability.

7. The face liveness detection method according to claim 5, characterized in that, The loss function used in training the infrared face liveness model is a joint loss function, which is also used to optimize the liveness detection task and the face distance judgment task.

8. The face liveness detection method according to claim 1, characterized in that, The formula for weighted fusion of the first liveness probability and the second liveness probability is as follows: in, This represents the first liveness probability. This represents the second liveness probability. The weighting coefficient corresponding to the first liveness probability. This is the weighting coefficient corresponding to the second liveness probability. The fusion result; The step of determining whether a face is a live face based on the fusion result includes: when the fusion result is greater than a preset probability threshold, it is determined to be a live face.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the face liveness detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the face liveness detection method according to any one of claims 1 to 8.

Images