Liveness detection method, electronic device, and storage medium

Abstract

The present disclosure provides a living body detection method, an electronic device and a storage medium. The method obtains a full-view image frame sequence and a face image frame sequence of a target content read by a user to be detected, inputs the full-view image frame sequence into a first living body detection sub-model to obtain a first living body detection result, inputs the face image frame sequence into a second living body detection sub-model to obtain a second living body detection result, and obtains a target detection result based on the first living body detection result and the second living body detection result. In the method, the living body detection is realized based on the frame sequences of the full-view image and the face image, and the global image, the local face region and the like are comprehensively and fully detected, so that the target detection result obtained can fully represent the image information, effectively improves the classification accuracy of the living body detection, and also ensures the generalization and stability of the living body detection by detecting the global and local information in different counterfeit modes.

Description

Technical Field

[0001] This disclosure belongs to the field of computer vision technology, specifically relating to a liveness detection method, electronic device, and storage medium. Background Technology

[0002] Liveness detection is a biometric method that verifies whether a subject is alive. It effectively defends against attacks using photos, videos, or occlusion techniques in facial recognition verification scenarios. Lip-reading liveness detection is a reading-based method that acquires a video of a user reading randomly selected text, extracts lip movement features from the video, and performs liveness detection based on these lip movements.

[0003] Currently, in order to improve the accuracy of liveness detection and enhance the performance of liveness detection algorithms, frames can be extracted from the video, and partial images of faces can be cropped out and then classified as real or fake using face detection algorithms.

[0004] It can be seen that existing improvements simplify image processing by employing techniques such as frame extraction and local facial image extraction to enhance the model's focus on feature information. However, this method also hinders the model's ability to fully learn image information, resulting in insufficient detection and recognition capabilities, weak generalization, and unstable detection rates for different forgery methods. Summary of the Invention

[0005] The purpose of this disclosure is to provide a liveness detection method, electronic device, and storage medium that can ensure the model fully learns from image information, enhance the generalization of liveness detection, and guarantee the detection rate for different forgery methods.

[0006] To solve the above-mentioned technical problems, this disclosure is implemented as follows:

[0007] In a first aspect, this disclosure provides a liveness detection method, which includes: acquiring a sequence of full-image frames and a sequence of face images when a user is reading target content, wherein each full-image frame in the full-image frame sequence includes a global region when the user is reading the target content, and each face image frame in the face image frame sequence includes a local facial region during the user's reading of the target content; inputting the full-image frame sequence into a first liveness detection sub-model, and performing liveness detection on the user based on the global region in each full-image frame to obtain a first liveness detection result for the user; and inputting the face image frame sequence into a second liveness detection sub-model, and performing liveness detection on the user based on the local facial region in each face image frame to obtain a second liveness detection result for the user; and determining a target detection result for the user based on the first liveness detection result and the second liveness detection result.

[0008] Optionally, before inputting the full-image frame sequence into the first liveness detection sub-model, the method further includes: performing inter-frame difference processing on the full-image frame sequence to obtain full-image frame difference images of any two adjacent full-image frames in the full-image frame sequence, and determining the full-image frame difference image sequence; correspondingly, inputting the full-image frame sequence into the first liveness detection sub-model includes: inputting the full-image frame difference image sequence into the first liveness detection sub-model.

[0009] Optionally, before inputting the face image frame sequence into the second liveness detection sub-model, the method further includes: performing inter-frame difference processing on the face image frame sequence to obtain face frame difference images of any two adjacent face image frames in the face image frame sequence, and determining the face frame difference image sequence; correspondingly, inputting the face image frame sequence into the second liveness detection sub-model includes: inputting the face frame difference image sequence into the second liveness detection sub-model.

[0010] Optionally, the first liveness detection sub-model includes a first temporal sub-network and a first fully connected sub-network configured in parallel. The number of first fully connected sub-networks is equal to the number of full-image frame difference images in the full-image frame difference image sequence. The first liveness detection sub-model performs liveness detection on the user under test based on the global region in each full-image frame to obtain the first liveness detection result for the user under test. This includes: extracting the first temporal features of the full-image frame difference image sequence through the first temporal sub-network, and performing liveness detection on the user under test based on the first temporal features to obtain the first temporal detection result. The first temporal features represent the temporal changes of each full-image frame difference image; and extracting the global image features of a full-image frame difference image through each first fully connected sub-network, and performing liveness detection on the user under test based on the global image features to obtain the first fully connected detection result corresponding to each full-image frame difference image; and fusing the first temporal detection result and each first fully connected detection result to obtain the first liveness detection result corresponding to the full-image frame sequence.

[0011] Optionally, the second liveness detection sub-model includes a second temporal sub-network and a second fully connected sub-network. The number of second fully connected sub-networks is equal to the number of face frame difference images in the face frame difference image sequence. The second liveness detection sub-model performs liveness detection on the user under test based on the local facial regions in each face image frame to obtain a second liveness detection result for the user under test. This includes: extracting second temporal features from the face frame difference image sequence through the second temporal sub-network, and performing liveness detection on the user under test based on the second temporal features to obtain a second temporal detection result. The second temporal features represent the temporal changes of each face frame difference image; and extracting facial image features from each face frame difference image through each second fully connected sub-network, and performing liveness detection on the user under test based on the facial image features to obtain a second fully connected detection result corresponding to each full-frame difference image; and fusing the second temporal detection result and the second fully connected detection result to obtain a second liveness detection result corresponding to the face image frame sequence.

[0012] Optionally, acquiring the full-image frame sequence and face image frame sequence when the user under test reads the target content includes: acquiring the original video when the user under test reads the target content; performing frame extraction processing on the original video to obtain the full-image frame sequence; determining the face key points in each full-image frame; performing mean processing on the face key points in each full-image frame to obtain the registration key points of each full-image frame; and performing centering processing on the local face regions in each full-image frame based on the registration key points to obtain the corresponding face image frames, thereby determining the face image frame sequence.

[0013] Optionally, based on the first liveness detection result and the second liveness detection result, the target detection result of the user to be detected is determined, including: fusing the first liveness detection result and the second liveness detection result to obtain the target detection result of the user to be detected; wherein, the fusing process includes taking the mean, taking the maximum value, or performing classification detection based on the first liveness detection result and the second liveness detection result.

[0014] Optionally, the training steps for the first and second liveness detection sub-models are as follows: Obtain a full-image frame sample sequence and a face image frame sample sequence; input the full-image frame sample sequence into the initial first liveness detection sub-model to obtain the third liveness detection result output by the initial first liveness detection sub-model, and iterate the initial first liveness detection sub-model based on the third liveness detection result to obtain the first liveness detection sub-model; and input the face image frame sample sequence into the initial second liveness detection sub-model to obtain the fourth liveness detection result output by the initial second liveness detection sub-model, and iterate the initial second liveness detection sub-model based on the fourth liveness detection result to obtain the second liveness detection sub-model.

[0015] In a second aspect, this disclosure provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the liveness detection method of the first aspect.

[0016] Thirdly, this disclosure provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the liveness detection method of the first aspect.

[0017] Fourthly, this disclosure provides a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the steps of the liveness detection method as described in the first aspect.

[0018] Fifthly, this disclosure provides a computer program product on which a computer program is stored, which, when executed by a processor, implements the liveness detection method as described in the first aspect.

[0019] In the liveness detection method provided in this disclosure, a full-image frame sequence and a face image frame sequence are obtained when the user under test reads the target content. Each full-image frame in the full-image frame sequence includes the global region when the user reads the target content, and each face image frame in the face image frame sequence includes the local facial region during the user's reading of the target content. The full-image frame sequence is then input into a first liveness detection sub-model, and the face image frame sequence is input into a second liveness detection sub-model to obtain first and second liveness detection results, respectively. Based on the first and second liveness detection results, the target detection result for the user under test is obtained. In this embodiment, liveness detection is implemented based on the frame sequences of the full-image and face images, comprehensively and fully detecting the global image and local facial regions. This ensures that the obtained target detection results can fully represent image information, effectively improving the classification accuracy of liveness detection. Combining global and local information for detection when different forgery methods are used also guarantees the generalization and stability of liveness detection. Attached Figure Description

[0020] Figure 1 This is one of the flowcharts of the liveness detection method provided in the embodiments of this disclosure;

[0021] Figure 2 This is the second flowchart of the liveness detection method provided in the embodiments of this disclosure;

[0022] Figure 3 This is a schematic diagram of the structure of the liveness detection model provided in the embodiments of this disclosure;

[0023] Figure 4A flowchart illustrating the steps of a training method for a liveness detection model provided in this embodiment of the disclosure;

[0024] Figure 5 This is a structural block diagram of the liveness detection device provided in the embodiments of this disclosure;

[0025] Figure 6 A structural block diagram of a training device for a liveness detection model provided in an embodiment of this disclosure;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0027] Figure 8 This is a hardware schematic diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0028] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0029] The terms "first," "second," etc., used in this disclosure and in the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0030] It should be noted that the data obtained in this public disclosure, including full-image frames, face image frames, user identity information, and other related data, were accessed, collected, stored, and used for subsequent analysis and processing after the user or relevant data owner was clearly informed of the content of the data collection, the purpose of the data, and the processing method, and with the consent and authorization of the user or relevant data owner. Furthermore, the public can provide the user or relevant data owner with ways to access, correct, or delete the data, as well as methods to revoke consent or authorization.

[0031] The liveness detection provided by the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings and through specific examples and application scenarios.

[0032] Figure 1This is one of the flowcharts of a liveness detection method provided in an embodiment of the present disclosure. The method may include the following steps 101 to 103.

[0033] Step 101: Obtain the full-image frame sequence and face image frame sequence when the user under test reads the target content. Each full-image frame in the full-image frame sequence includes the global region when the user under test reads the target content, and each face image frame in the face image frame sequence includes the local face region of the user under test during the process of reading the target content.

[0034] In this context, the "user to be tested" refers to a user waiting to undergo liveness detection in a liveness detection scenario. Liveness detection scenarios can include user identity verification, such as facial recognition payment, access control verification, online login, and language testing. Lip-reading liveness detection can be used, which involves acquiring image frames of the user reading target content and performing liveness detection on the user. The target content can be numbers, text, or question prompts, allowing the user to provide a verbal response. Furthermore, liveness detection can be performed by analyzing the temporal changes in the user's facial movements during the reading of target content, thus determining whether the user is alive.

[0035] In this embodiment of the disclosure, a full-image frame sequence and a face image frame sequence can be obtained when the user under test reads the target content. The full-image frame sequence includes full-image frames arranged in time sequence, and each full-image frame can be an image of the global region of the target content read by the user under test in the corresponding time sequence. The face image frame sequence includes face image frames arranged in time sequence, and each face image frame can be an image of a local face region of the target content read by the user under test in the corresponding time sequence.

[0036] In an optional embodiment of this disclosure, the full-image frame sequence and the face image frame sequence can be obtained by configuring a shooting module to capture data during the process of the user reading the target content, or by establishing a communication connection with other electronic devices configured with shooting modules, thereby acquiring the full-image frame sequence and face image frame sequence of the user reading the target content through those other electronic devices. These other electronic devices can be mobile phones, tablets, computers, cameras, display panels, monitoring equipment, etc.

[0037] Step 102: Input the full-image frame sequence into the first liveness detection sub-model. The first liveness detection sub-model performs liveness detection on the user to be tested based on the global region in each full-image frame to obtain the first liveness detection result for the user to be tested. Then, input the face image frame sequence into the second liveness detection sub-model. The second liveness detection sub-model performs liveness detection on the user to be tested based on the local face region in each face image frame to obtain the second liveness detection result for the user to be tested.

[0038] In this embodiment, there is a first liveness detection sub-model and a second liveness detection sub-model. The first liveness detection sub-model is trained on full-image samples and can perform liveness detection based on the global region of the entire image frame. The second liveness detection sub-model is trained on face image samples and can perform liveness detection on the local face region of the face image frame. Based on this, the first liveness detection sub-model focuses more on the overall information of the image, while the second liveness detection sub-model focuses more on the local information of the face region.

[0039] Furthermore, after obtaining the full-image frame sequence and the face image frame sequence, the first liveness detection sub-model and the second liveness detection sub-model can be used in combination. The full-image frame sequence is input into the first liveness detection sub-model so that the first liveness detection sub-model performs liveness detection on the overall image information based on the global region in each full-image frame to obtain the first liveness detection result corresponding to the user under test. And, the face image frame sequence is input into the second liveness detection sub-model so that the second liveness detection sub-model performs liveness detection on the local face information based on the local face region in each face image frame to obtain the second liveness detection result corresponding to the user under test.

[0040] Step 103: Based on the first liveness detection result and the second liveness detection result, determine the target detection result for the user to be detected.

[0041] In this embodiment, after obtaining the first liveness detection result output by the first liveness detection sub-model and the second liveness detection result output by the second liveness detection sub-model, a target detection result can be further obtained based on the first liveness detection result and the second liveness detection result. The target detection result can comprehensively integrate the full-image structural information and local facial information during the process of the user reading the target content, thereby achieving more accurate liveness detection of the user and avoiding the problem that the detection rate may decrease due to changes in the forgery method, thus improving the generalization and robustness of liveness detection classification.

[0042] Figure 2 This is a second flowchart illustrating the steps of a liveness detection method provided in this embodiment of the disclosure. The method may include the following steps 201 to 207.

[0043] Step 201: Obtain the original video of the user reading the target content.

[0044] In this embodiment of the disclosure, the original video of the user under test reading the target content can be acquired, and the original video includes the user's facial movements. Acquiring the original video can be achieved by configuring a local shooting module to capture the user's facial movements while reading the target content, or by establishing a communication connection with other electronic devices configured with shooting modules, and acquiring the original video of the user reading the target content through these other devices.

[0045] Step 202: Perform frame extraction on the original video to obtain a full-image frame sequence.

[0046] Furthermore, the acquired raw video can be processed by frame extraction to obtain a full-image frame sequence. By extracting partial image information from the raw video to represent the overall information, it is possible to avoid performing overall analysis and processing on all image information in the raw video. This reduces computational overhead and model computation and deployment costs while ensuring detection accuracy. The method of frame extraction from the raw video can be selected by those skilled in the art based on computational conditions and analysis requirements. For example, frame extraction can involve extracting one full-image frame at equal time intervals from the raw video, or it can involve setting entry points at equal time intervals in the raw video and continuously extracting a certain number of full-image frames at each entry point. Moreover, frame extraction can be performed after the raw video is captured or during the capture process. This disclosure does not impose specific limitations on the acquisition of the raw video, the method of frame extraction, or the timing of frame extraction.

[0047] For example, in the original video of the user reading the numbers "6, 1, 9, 4, 2, 9", one full-image frame is extracted every time interval t1, and the full-image frames are arranged in time order to obtain the full-image frame sequence.

[0048] Alternatively, in the original video of the user reading the numbers "6, 1, 9, 4, 2, 9", n consecutive full-image frames are extracted at time intervals t2, and the full-image frames are arranged in time sequence to obtain the full-image frame sequence.

[0049] Step 203: Determine the facial key points in each full-image frame.

[0050] In this embodiment, facial landmarks are used to identify and locate different features of the face, such as the eyes, nose, mouth, eyebrows, and facial contours. Depending on the required recognition accuracy and actual data processing conditions, the location and number of facial landmarks can be set; the number can be 68, 49, 21, or 5, etc. In a full-image frame sequence, the number of facial landmarks and the identified features should be the same in different full-image frames. Facial landmarks can be manually annotated or annotated using a pre-trained facial landmark annotation model; this embodiment does not impose specific limitations on this.

[0051] Step 204: Perform mean processing on the facial key points of each full-image frame to obtain the registration key points of each full-image frame.

[0052] In this embodiment, the averaging process can be performed by averaging the coordinates of facial key points in the entire image frame. By averaging the facial key points, the coordinates of the registration key points in the entire image frame can be obtained. The registration key points are obtained based on the average of each facial key point. While preserving the representation of facial features by the facial key points, it can also preserve the facial motion information of the user when reading the target content, thereby improving the accuracy and efficiency of liveness detection.

[0053] Step 205: Based on the registration key points, center the local face region in each full-image frame to obtain the corresponding face image frame, so as to determine the face image frame sequence.

[0054] In this embodiment, the face image frame sequence can be obtained by face registration of the entire image frame sequence. Face registration, also known as face alignment, is a technique for detecting and locating key points of a face in an image. Through face registration processing, local facial regions can be determined in the entire image frame, thereby weakening and reducing the interference of background information on facial information in the obtained face image frame sequence, and highlighting the features of local facial regions. Specifically, face registration can be performed on the entire image frame based on registration key points to center the local facial regions. For example, affine transformations can be performed on different entire image frames based on registration key points to generate face image frames of the same size with the local facial regions centered. In the entire image frame sequence, each entire image frame can be aligned, matched, and scaled based on registration key points to obtain face image frames of the same size with the faces centered, thus forming a face image frame sequence.

[0055] Step 206: Perform inter-frame difference processing on the full-image frame sequence to obtain the full-image frame difference image of any two adjacent full-image frames in the full-image frame sequence, and determine the full-image frame difference image sequence.

[0056] In this embodiment, inter-frame difference refers to pixel-by-pixel difference processing of two adjacent image frames to obtain a frame difference image corresponding to the two adjacent image frames. The frame difference image can characterize the dynamic changes between adjacent image frames over time, thereby representing the behavioral characteristics of the user being tested during the reading of target content. In the liveness detection process, focus can be placed on abnormal behavior caused by forgery for liveness detection, reducing attention to image texture, structure, and other information. This reduces data volume while maintaining liveness detection accuracy and improving efficiency. In this case, within the full-image frame sequence, pixel-by-pixel difference processing can be performed on any two adjacent full-image frames according to the temporal sequence to obtain a full-image frame difference image corresponding to any two adjacent full-image frames, thus obtaining a full-image frame difference image sequence corresponding to the full-image frame sequence.

[0057] Step 207: Input the full-frame differential image sequence into the first liveness detection sub-model. The first liveness detection sub-model performs liveness detection on the user to be tested based on the global region in each full-frame image, and obtains the first liveness detection result for the user to be tested.

[0058] Furthermore, the full-frame differential image sequence can be input into the first liveness detection sub-model. Since the full-frame differential image sequence re-extracts the dynamic change information of the video based on the full-frame image sequence, the amount of information that the first liveness detection sub-model needs to focus on is further reduced, thereby simplifying the network structure of the first liveness detection sub-model and reducing the number of parameters. With low deployment cost, the first liveness detection sub-model can be applied to perform liveness detection on the full-frame differential image sequence to obtain the liveness detection results corresponding to the full-frame differential image sequence.

[0059] In this embodiment, after obtaining the full-frame differential image sequence, feature extraction can be performed on the full-frame differential image sequence to obtain a feature representation of the full-frame differential image sequence adapted to the structure of the first liveness detection sub-model. The feature extraction method can be selected based on the task requirements and deployment conditions of the first liveness detection sub-model. For example, lightweight neural networks with small parameter counts, such as ResNet (Residual Network), MobileNet, ShuffleNet, EffNet, Vit, and SwinTransformer, can be used to extract features from the full-frame differential image sequence. In practical applications, when using a CPU for feature extraction model inference, less computationally demanding networks such as MobileNet, ShuffleNet, and EffNet can be used; when using a GPU for feature extraction model inference, other network structures with higher computational demands can be used to improve the accuracy of feature extraction while meeting deployment conditions.

[0060] In an optional embodiment of the method disclosed herein, the first liveness detection sub-model may include a first temporal sub-network and a first fully connected sub-network configured in parallel, wherein the number of the first fully connected sub-networks is equal to the number of full-frame differential images in the full-frame differential image sequence.

[0061] The first temporal sub-network extracts dynamic behavioral features from the time series, enabling liveness detection on the entire image frame difference sequence based on the dynamic changes between the difference images. This first temporal sub-network can be any neural network suitable for processing temporal information, such as RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), or CNN (Convolutional Neural Network). The first fully connected sub-network performs liveness detection on each image frame difference based on the dynamic changes represented by the pixel differences. The number of fully connected sub-networks can be equal to the number of image frame difference images in the sequence. By configuring the first temporal sub-network and the first fully connected sub-network in parallel, the first liveness detection sub-model can perform combined liveness detection on both the overall and local levels of the image frame difference sequence, further improving the accuracy of full-image liveness detection.

[0062] Step 207 may include steps A1 to A2.

[0063] Step A1: Extract the first temporal features of the full-frame difference image sequence through the first temporal sub-network, and perform liveness detection on the user under test based on the first temporal features to obtain the first temporal detection result. The first temporal features represent the temporal changes of each full-frame difference image. Also, extract the global image features of a full-frame difference image through each first fully connected sub-network, and perform liveness detection on the user under test based on the global image features to obtain the first fully connected detection result corresponding to each full-frame difference image.

[0064] In this embodiment of the disclosure, after inputting the full-frame differential image sequence into the first liveness detection sub-model, the first temporal sub-network can perform liveness detection on the entire full-frame differential image sequence. By extracting the first temporal features of the full-frame differential images between sequences, it detects the temporal changes of each full-frame differential image to identify possible abnormal behaviors of the user under test during the reading of target content, and outputs the first temporal detection result corresponding to the user under test. At the same time, the first fully connected sub-network can perform liveness detection on the entirety of each full-frame differential image. By extracting the global image features of each full-frame differential image, it performs liveness detection on the user under test based on the global image features, and outputs the first fully connected detection result corresponding to each full-frame differential image.

[0065] Step A2: Fuse the first temporal detection results and each first fully connected detection result to obtain the first liveness detection result corresponding to the full image frame sequence.

[0066] In this embodiment, the first temporal detection result and the first fully connected detection result can be fused to obtain the first liveness detection result. By fusing the detection results of the overall sequence and each image frame in the fused sequence, the liveness detection accuracy can be effectively improved, thus maintaining a stable detection rate even against different forgery methods, and improving the versatility and robustness of liveness detection. The method of fusing the first temporal detection result and the first fully connected detection result can be selected according to the requirements of subsequent task processing and computational conditions. For example, the maximum value or average of the first temporal detection result and all first fully connected detection results can be taken, or a fully connected network can be used to further classify the first temporal detection result and all first fully connected detection results to output a single first liveness detection result.

[0067] Step 208: Perform inter-frame difference processing on the face image frame sequence to obtain the face frame difference image of any two adjacent face image frames in the face image frame sequence, and determine the face frame difference image sequence.

[0068] Step 209: Input the face frame differential image sequence into the second liveness detection sub-model. The second liveness detection sub-model performs liveness detection on the user to be tested based on the local facial regions in each face image frame, and obtains the second liveness detection result for the user to be tested.

[0069] In this embodiment of the disclosure, the face image frame sequence in steps 208 to 209 can be referred to the relevant description of the full image frame sequence in steps 206 to 207 above. To avoid repetition, it will not be repeated here.

[0070] In an optional embodiment of the method disclosed herein, the second liveness detection sub-model may include a second temporal sub-network and a second fully connected sub-network configured in parallel, wherein the number of second fully connected sub-networks is equal to the number of face frame difference images in the face frame difference image sequence. Then step 209 may include steps B1 to B2 as follows.

[0071] Step B1: Extract the second temporal features of the face frame difference image sequence through the second temporal sub-network, and perform liveness detection on the user under test based on the second temporal features to obtain the second temporal detection result. The second temporal features represent the temporal changes of each face frame difference image. Also, extract the face image features of a face frame difference image through each second fully connected sub-network, and perform liveness detection on the user under test based on the face image features to obtain the second fully connected detection result corresponding to each full-frame difference image.

[0072] In this embodiment of the disclosure, after inputting the face frame difference image sequence into the second liveness detection sub-model, the second temporal sub-network can perform liveness detection on the entire face frame difference image sequence. By extracting the second temporal features of the face frame difference images between sequences, it detects the temporal changes of each face frame difference image to identify possible abnormal behaviors of the user under test during the reading of target content, and outputs the second temporal detection result corresponding to the user under test. At the same time, the second fully connected sub-network can perform liveness detection on the entirety of each face frame difference image separately. By extracting the face image features of each face frame difference image, and performing liveness detection on the user under test based on the face image features, it outputs the second fully connected detection result corresponding to each face frame difference image.

[0073] Step B2: Fuse the second temporal detection result and the second fully connected detection result to obtain the second liveness detection result corresponding to the face image frame sequence.

[0074] In this embodiment of the disclosure, step B2 can be referred to the relevant description of step A2 above. To avoid repetition, it will not be repeated here.

[0075] Step 210: Perform fusion processing on the first liveness detection result and the second liveness detection result to obtain the target detection result of the user to be tested; wherein, the fusion processing includes taking the mean, taking the maximum value, or performing classification detection based on the first liveness detection result and the second liveness detection result.

[0076] In this embodiment, the first liveness detection result and the second liveness detection result can be fused to obtain the target detection result corresponding to the video. By fusing the detection results of the global region and the local face region in the video, the liveness detection accuracy of the video can be further improved, thereby enhancing the versatility and robustness of liveness detection. The method of fusing the first and second liveness detection results can be selected according to the requirements of subsequent task processing and computational conditions. For example, the maximum value or the average value of the first and second liveness detection results can be taken, or a fully connected network can be used to further classify and detect based on the first and second liveness detection results, outputting a single target detection result.

[0077] Figure 3 This is a schematic diagram of the structure of a liveness detection model provided in an embodiment of this disclosure. This liveness detection model can be a first liveness detection sub-model when trained using a full-image frame sample sequence, and a second liveness detection sub-model when trained using a face image frame sample sequence, depending on the training data. Figure 3 As shown, the liveness detection model may include an inter-frame difference processing module 301, a feature extraction module 302, a liveness detection module 303, and a result fusion module 304. The feature extraction module 302 may be composed of a residual neural network. The liveness detection module 303 may include a temporal sub-network 3031 and m fully connected sub-networks 3032, which may be composed of recurrent neural networks. Taking a liveness detection model including a first liveness detection sub-model as an example, the image frame sequence X0 includes a full-image frame sequence. The full-image frame sequence X0 extracted from the original video is input into the first liveness detection sub-model. The liveness detection steps are shown in steps C1 to C4 below:

[0078] Step C1: The inter-frame difference processing module 301 performs inter-frame difference processing on the full-image frame sequence X0 to obtain the full-image frame difference image sequence X1. The full-image frame difference image sequence X1 contains n full-image frame difference images, and m≥n.

[0079] Step C2: The feature extraction module 302 converts the full-frame differential image sequence X1 into a high-dimensional feature sequence X2;

[0080] In step C3, the liveness detection module 303 performs liveness detection based on the transformed high-dimensional feature sequence X2. The temporal sub-network 3031 performs liveness detection based on the temporal information of the high-dimensional feature sequence X2 to obtain the first temporal detection result. Each fully connected sub-network 3032 performs liveness detection on each frame of the full-image frame difference image to obtain the first fully connected detection result.

[0081] Step C4: The result fusion module 304 fuses the first temporal detection result and the first fully connected detection result to obtain the first liveness detection result corresponding to the full-image frame sequence X0.

[0082] Alternatively, the liveness detection model may include a second liveness detection sub-model. In this case, the image frame sequence X0 includes a face image frame sequence. The full-image face frame sequence X0 is input into the second liveness detection sub-model, and the liveness detection steps are shown in steps D1 to D4 below:

[0083] Step D1: The inter-frame difference processing module 301 performs inter-frame difference processing on the face image frame sequence X0 to obtain the face frame difference image sequence X1. The face frame difference image sequence X1 contains n face frame difference images, and m≥n.

[0084] Step D2: The feature extraction module 302 converts the face frame difference image sequence X1 into a high-dimensional feature sequence X2;

[0085] In step D3, the liveness detection module 303 performs liveness detection based on the transformed high-dimensional feature sequence X2. The temporal sub-network 3031 performs liveness detection based on the temporal information of the high-dimensional feature sequence X2 to obtain the second temporal detection result. Each fully connected sub-network 3032 performs liveness detection on each frame of face frame difference image to obtain the second fully connected detection result.

[0086] Step D4: The result fusion module 304 fuses the second temporal detection result and the second fully connected detection result to obtain the second liveness detection result corresponding to the face image frame sequence X0.

[0087] Furthermore, the first and second liveness detection results can be fused to obtain the target detection results corresponding to the user to be tested.

[0088] In the liveness detection method provided in this disclosure, a full-image frame sequence and a face image frame sequence are obtained when the user under test reads the target content. Each full-image frame in the full-image frame sequence includes the global region when the user reads the target content, and each face image frame in the face image frame sequence includes the local facial region during the user's reading of the target content. The full-image frame sequence is then input into a first liveness detection sub-model, and the face image frame sequence is input into a second liveness detection sub-model to obtain first and second liveness detection results, respectively. Based on the first and second liveness detection results, the target detection result for the user under test is obtained. In this embodiment, liveness detection is implemented based on the frame sequences of the full-image and face images, comprehensively and fully detecting the global image and local facial regions. This ensures that the obtained target detection results can fully represent image information, effectively improving the classification accuracy of liveness detection. Combining global and local information for detection when different forgery methods are used also guarantees the generalization and stability of liveness detection.

[0089] Figure 4 This is a flowchart illustrating the steps of a training method for a liveness detection model provided in this embodiment. The liveness detection model may include a first liveness detection sub-model and a second liveness detection sub-model, and can be applied to the aforementioned... Figure 1 , 2 Any liveness detection method. For example... Figure 4 As shown, the training method may include the following steps 401 to 403.

[0090] Step 401: Obtain the full image frame sample sequence and the face image frame sample sequence.

[0091] In this embodiment, the full-image frame sample sequence and the face image frame sample sequence can correspond to the process of any user reading the target content. This user can be a real user or a fake user. For example, a positive sample can be obtained by acquiring the global image frame sample sequence and the face image frame sample sequence when a real user reads the target content; a negative sample can be obtained by spoofing a real user and acquiring the global image frame sample sequence and the face image frame sample sequence when the user reads the target content, thus obtaining image frame sample sequences corresponding to different spoofing types and real users. The method of acquiring the full-image frame sample sequence and the face image frame sample sequence can refer to the relevant description of step 101 above, and will not be repeated here to avoid repetition.

[0092] The forgery of real users can include deepfakes such as facial fusion and tampering, or motion-driven static images based on facial action videos of real users reading target content, or image stitching; it can also collect global image frame sample sequences and face image frame sample sequences under conditions such as T-masks, stereoscopic head models, screen re-photographs, paper masks, and occlusion to obtain negative samples of entity data. During the acquisition of the full-image frame sample sequence and the face image frame sample sequence, different lighting conditions and backgrounds can be adjusted to collect image frame sequences from users with different identity information. To balance model performance and training efficiency, the number of image frame sequences acquired can be determined based on computational conditions. In this embodiment, the forgery method of negative samples can be selected according to computational cost and task requirements. During the acquisition process, the full-image frame sample sequence and the face image frame sample sequence can be labeled according to the acquisition method or data type to distinguish between positive and negative samples.

[0093] In one optional embodiment of the method disclosed herein, augmentation processing can also be performed on the full-image frame sample sequence, face image frame sample sequence, etc., such as image quality compression processing, or adding noise, blurring, etc., or adjusting image brightness, contrast, color, etc., thereby expanding the number and types of samples and improving the model detection accuracy and versatility.

[0094] In this embodiment, an initial first liveness detection sub-model and an initial second liveness detection sub-model can be constructed according to task requirements and computational conditions. Since the full-image frame sample sequence undergoes frame extraction processing to obtain partial information to represent the whole, and the face image frame sample sequence further performs face centering processing on the full-image frame sequence to obtain partial information of the local face region, the corresponding initial liveness detection model and initial second liveness detection sub-model can adopt a simple structure to reduce the number of model parameters without affecting model performance, thereby effectively balancing the detection accuracy and training deployment cost. The initial first liveness detection sub-model and initial second liveness detection sub-model have the same structure but different input training data, thus allowing for the training of different first and second liveness detection sub-models.

[0095] Step 402: Input the full image frame sample sequence into the initial first liveness detection sub-model, obtain the third liveness detection result output by the initial first liveness detection sub-model, and iterate the initial first liveness detection sub-model based on the third liveness detection result to obtain the first liveness detection sub-model.

[0096] In this embodiment, a full-image frame sample sequence can be input into an initial first liveness detection sub-model to perform liveness detection on the full-image frame sample sequence, obtaining a third liveness detection result corresponding to the training video. This third liveness detection result includes whether the user corresponding to the full-image frame sequence is a real user or a fake user. Based on this, and using the third liveness detection result and the classification labeled during the acquisition process of the full-image frame sequence, the initial first liveness detection sub-model can be iterated until a convergence condition is met, thereby obtaining the first liveness detection sub-model. The convergence condition can be that the loss value of the third full-image detection result relative to the labeled classification is within a preset numerical range, or that the number of iterations reaches a preset number, etc., ensuring that the model's accuracy and generalization meet the task requirements. This embodiment does not impose specific limitations on this.

[0097] In an optional embodiment of the method disclosed herein, the initial first liveness detection sub-model includes a third temporal sub-network and a third fully connected sub-network configured in parallel, and the number of the third fully connected sub-networks is equal to the number of full-image frame samples in the full-image frame sample sequence.

[0098] Step 402 may include steps E1 to E3.

[0099] Step E1: Perform inter-frame difference processing on the full-image frame sample sequence to obtain the corresponding full-image frame difference image sample sequence.

[0100] In this embodiment of the disclosure, the inter-frame difference processing of the whole image frame sample sequence can be referred to the relevant description of the inter-frame difference processing of the whole image frame sequence in step 206 above. The inter-frame difference processing can reduce the interference of background information to improve detection accuracy while retaining the dynamic change information between the whole image frame samples, and simplify the amount of data, thereby further simplifying the model structure, reducing the number of parameters, improving the model training efficiency, and reducing the model deployment cost.

[0101] Step E2: Input the full-frame differential image sample sequence into the initial first liveness detection sub-model so that the third temporal sub-network performs liveness detection on the full-frame differential image sample sequence to obtain the third temporal detection result, and so that the third fully connected sub-network performs liveness detection on each second full-frame differential image to obtain the third fully connected detection result corresponding to each second full-frame differential image.

[0102] In this embodiment, the initial first liveness detection sub-model may include a third temporal sub-network and a third fully connected sub-network whose number matches the number of second full-frame difference images in the full-frame difference image sample sequence. The third temporal sub-network can perform liveness detection based on overall temporal information within the sequence, while the third fully connected sub-network can perform liveness detection on each of the second full-frame difference images based on image structure and other information. This allows for obtaining the overall third temporal detection result for the full-frame difference image sample sequence, as well as the third fully connected detection result corresponding to each second full-frame difference image in the full-frame difference image sample sequence.

[0103] Step E3: Fuse the third temporal detection result and the third fully connected detection result to obtain the third liveness detection result corresponding to the full image frame sample sequence.

[0104] Furthermore, the third temporal detection result of the whole image frame difference image sample sequence and the third fully connected detection result corresponding to each second image frame difference image can be fused. For details, please refer to the relevant description of fusing the first temporal detection result and the first fully connected detection result in step A2 above. To avoid repetition, it will not be repeated here.

[0105] Step 403: Input the face image frame sample sequence into the initial second liveness detection sub-model, obtain the fourth liveness detection result output by the initial second liveness detection sub-model, and iterate the initial second liveness detection sub-model based on the fourth liveness detection result to obtain the second liveness detection sub-model.

[0106] In this embodiment of the present disclosure, the step of training the initial second liveness detection sub-model using a sequence of face image frames in step 403 can be referred to in the description of the step of training the initial first liveness detection sub-model using a sequence of full-image frames in step 402 above. To avoid repetition, it will not be repeated here.

[0107] In an optional embodiment of the method disclosed herein, the initial second liveness detection sub-model includes a fourth temporal sub-network and a fourth fully connected sub-network configured in parallel, and the number of fourth fully connected sub-networks is equal to the number of face image frame samples in the face image frame sample sequence. Then step 403 may include steps F1 to F3 as follows.

[0108] Step F1: Perform inter-frame difference processing on the face image frame sample sequence to obtain the corresponding face frame difference image sample sequence.

[0109] In this embodiment of the disclosure, the inter-frame difference processing of the face image frame sample sequence can be referred to the relevant description of the inter-frame difference processing of the whole image frame sequence in step 205 above. To avoid repetition, it will not be repeated here.

[0110] Step F2: Input the face frame difference image sample sequence into the initial second liveness detection sub-model so that the fourth temporal sub-network performs liveness detection on the face frame difference image sample sequence to obtain the fourth temporal detection result, and so that the fourth fully connected sub-network performs liveness detection on each second face frame difference image to obtain the fourth fully connected detection result corresponding to each second face frame difference image.

[0111] In this embodiment of the disclosure, the initial second liveness detection sub-model may include a fourth temporal sub-network and a fourth fully connected sub-network whose number matches the number of second face frame difference images in the face frame difference image sample sequence. The fourth temporal sub-network can perform liveness detection based on overall temporal information in the sequence, and the fourth fully connected sub-network can perform liveness detection on the second face frame difference images based on image structure and other information, thereby obtaining the overall fourth temporal detection result of the face frame difference image sample sequence and the fourth fully connected detection result corresponding to each second face frame difference image in the face frame difference image sample sequence.

[0112] Step F3: Fuse the fourth temporal detection result and the fourth fully connected detection result to obtain the fourth liveness detection result corresponding to the face image frame sample sequence.

[0113] Furthermore, the fourth temporal detection result of the entire face frame difference image sample sequence and the fourth fully connected detection result corresponding to each second face frame difference image can be fused. For details, please refer to the relevant description of fusing the first temporal detection result and the first fully connected detection result in step A2 above. To avoid repetition, it will not be repeated here.

[0114] In this embodiment, the first and second liveness detection sub-models are trained separately, enabling each model to achieve the desired classification performance for two different sets of inputs, thus avoiding inter-model dependencies that may result from combined training. In subsequent liveness detection tasks, the detection results output by the first and second liveness detection sub-models are fused, fully utilizing local facial information and overall image structure information to effectively improve classification accuracy and enhance the model's generalization and robustness.

[0115] Figure 5 This is a structural block diagram of the liveness detection device 500 provided in an embodiment of this disclosure. Figure 5As shown, the device may include: a sequence acquisition module 501, which acquires a sequence of full-image frames and a sequence of face images when the user under test reads target content, wherein each full-image frame in the full-image frame sequence includes the global region when the user under test reads the target content, and each face image frame in the face image frame sequence includes the local facial region of the user under test during the reading of the target content; a full-image detection module 502, which inputs the full-image frame sequence into a first liveness detection sub-model, and performs liveness detection on the user under test based on the global region in each full-image frame, to obtain a first liveness detection result for the user under test; and a face detection module 503, which inputs the face image frame sequence into a second liveness detection sub-model, and performs liveness detection on the user under test based on the local facial region in each face image frame, to obtain a second liveness detection result for the user under test; and a result determination module 504, which determines the target detection result for the user under test based on the first liveness detection result and the second liveness detection result.

[0116] In an optional device embodiment of this disclosure, the sequence acquisition module 501 is specifically used to perform inter-frame difference processing on the full-image frame sequence to obtain the full-image frame difference image of any two adjacent full-image frames in the full-image frame sequence, and to determine the full-image frame difference image sequence; correspondingly, the full-image detection module 502 is specifically used to input the full-image frame difference image sequence into the first liveness detection sub-model.

[0117] In an optional device embodiment of this disclosure, the sequence acquisition module 501 is specifically used to perform inter-frame difference processing on the face image frame sequence to obtain the face frame difference image of any two adjacent face image frames in the face image frame sequence, and determine the face frame difference image sequence; correspondingly, the face detection module 503 is specifically used to input the face frame difference image sequence into the second liveness detection sub-model.

[0118] In an optional device embodiment of this disclosure, the first liveness detection sub-model includes a first temporal sub-network and a first fully connected sub-network configured in parallel. The number of first fully connected sub-networks is equal to the number of full-image frame difference images in the full-image frame difference image sequence. The full-image detection module 502 is specifically used to extract first temporal features of the full-image frame difference image sequence through the first temporal sub-network, and perform liveness detection on the user to be tested based on the first temporal features to obtain a first temporal detection result. The first temporal features characterize the temporal changes of each full-image frame difference image. It also extracts global image features of a full-image frame difference image through each first fully connected sub-network, and performs liveness detection on the user to be tested based on the global image features to obtain a first fully connected detection result corresponding to each full-image frame difference image. The first temporal detection result and each first fully connected detection result are fused to obtain a first liveness detection result corresponding to the full-image frame sequence.

[0119] In an optional device embodiment of this disclosure, the second liveness detection sub-model includes a second temporal sub-network and a second fully connected sub-network. The number of second fully connected sub-networks is equal to the number of face frame difference images in the face frame difference image sequence. The face detection module 503 is specifically used to extract second temporal features from the face frame difference image sequence through the second temporal sub-network, and perform liveness detection on the user under test based on the second temporal features to obtain a second temporal detection result. The second temporal features characterize the temporal changes of each face frame difference image. Furthermore, it extracts face image features from each face frame difference image through each second fully connected sub-network, and performs liveness detection on the user under test based on the face image features to obtain a second fully connected detection result corresponding to each full-frame difference image. Finally, it fuses the second temporal detection result and the second fully connected detection result to obtain a second liveness detection result corresponding to the face image frame sequence.

[0120] In an optional device embodiment of this disclosure, the sequence acquisition module 501 is specifically used to acquire the original video when the user under test reads the target content; perform frame extraction processing on the original video to obtain a full-image frame sequence; determine the facial key points in each full-image frame; perform mean processing on the facial key points in each full-image frame to obtain the registration key points of each full-image frame; and perform centering processing on the local facial regions in each full-image frame based on the registration key points to obtain the corresponding facial image frames, thereby determining the facial image frame sequence.

[0121] In an optional device embodiment of this disclosure, the result determination module 504 is specifically used to perform fusion processing on the first liveness detection result and the second liveness detection result to obtain the target detection result of the user to be tested; wherein, the fusion processing includes taking the average value, taking the maximum value, or performing classification detection based on the first liveness detection result and the second liveness detection result.

[0122] In the liveness detection device provided in this disclosure, a full-image frame sequence and a face image frame sequence are acquired when the user under test reads the target content. Each full-image frame in the full-image frame sequence includes the global region when the user under test reads the target content, and each face image frame in the face image frame sequence includes the local face region of the user under test during the reading of the target content. The full-image frame sequence is then input into the first liveness detection model and the first liveness detection sub-model, and the face image frame sequence is input into the second liveness detection model and the second liveness detection sub-model, respectively, to obtain the first liveness detection result and the second liveness detection result. Based on the first liveness detection result and the second liveness detection result, the target detection result of the user under test is obtained. In this embodiment, liveness detection is implemented based on frame sequences of the full-image and face images, which simplifies the computational load of model training and reduces the tedious processing of image frequency domain information conversion. Moreover, it comprehensively and fully detects the global image and local face regions, so that the obtained target detection results can fully represent the image information, effectively improving the classification accuracy of liveness detection. Furthermore, by combining global and local information to detect different forgery methods, it also ensures the generalization and robust stability of forgery detection.

[0123] Figure 6 This is a structural block diagram of a training device for a liveness detection model provided in an embodiment of the present disclosure. The liveness detection model trained by this device may include a first liveness detection sub-model and a second liveness detection sub-model, and can be applied to the aforementioned... Figure 5 The aforementioned liveness detection device is in operation. For example... Figure 6 As shown, the device may include: a sample sequence acquisition module 601, used to acquire a full-image frame sample sequence and a face image frame sample sequence; a first model training module 602, which inputs the full-image frame sample sequence into an initial first liveness detection sub-model, acquires a third liveness detection result output by the initial first liveness detection sub-model, and iterates the initial first liveness detection sub-model based on the third liveness detection result to obtain a first liveness detection sub-model; and a second model training module 602, which inputs a face image frame sample sequence into an initial second liveness detection sub-model, acquires a fourth liveness detection result output by the initial second liveness detection sub-model, and iterates the initial second liveness detection sub-model based on the fourth liveness detection result to obtain a second liveness detection sub-model.

[0124] In this embodiment, the first and second liveness detection sub-models are trained separately, enabling each model to achieve the desired classification performance for two different sets of inputs, thus avoiding the dependency issues that may arise between models during integrated training. In subsequent liveness detection tasks, the detection results output by the first and second liveness detection sub-models are fused, fully utilizing local facial information and overall image structure information to effectively improve classification accuracy and enhance the model's generalization and robustness.

[0125] Figure 7 This is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present disclosure, as shown below. Figure 7 As shown, the electronic device 700 may include a processor 701, a memory 702, and a program or instructions stored in the memory 702 and executable on the processor 701. When the program or instructions are executed by the processor 701, they implement the various processes of the above-described liveness detection method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0126] It should be noted that, Figure 7 The illustrated electronic device 700 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0127] Figure 8 This is a hardware schematic diagram of an electronic device 800 provided in an embodiment of the present disclosure, such as... Figure 8 As shown, the electronic device 800 includes a Central Processing Unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in ROM (Read-Only Memory) 802 or programs loaded from storage section 808 into RAM (Random Access Memory) 803. RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. An I / O (Input / Output) interface 805 is also connected to bus 804.

[0128] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0129] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by the central processing unit (CPU 801), it performs various functions defined in the system of this application.

[0130] This disclosure also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described liveness detection method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here.

[0131] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as ROM, RAM, magnetic disk, or optical disk.

[0132] This disclosure also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described liveness detection method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0133] It should be understood that the chip mentioned in the embodiments of this disclosure may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0134] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this disclosure is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, electronic device, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0136] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, this disclosure is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this disclosure without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this disclosure.

Claims

1. A method for detecting liveness, characterized in that, The method includes: Obtain a full-image frame sequence and a face image frame sequence when the user under test reads the target content. Each full-image frame in the full-image frame sequence includes the global region when the user under test reads the target content, and each face image frame in the face image frame sequence includes the local face region of the user under test during the process of reading the target content. The full-image frame sequence is input into a first liveness detection sub-model, which performs liveness detection on the user under test based on the global regions in each of the full-image frames, to obtain a first liveness detection result for the user under test; and the face image frame sequence is input into a second liveness detection sub-model, which performs liveness detection on the user under test based on the local face regions in each of the face image frames, to obtain a second liveness detection result for the user under test. Based on the first liveness detection result and the second liveness detection result, the target detection result of the user to be tested is determined; Before inputting the full-image frame sequence into the first liveness detection sub-model, the method further includes: Perform inter-frame difference processing on the full-image frame sequence to obtain the full-image frame difference image of any two adjacent full-image frames in the full-image frame sequence, and determine the full-image frame difference image sequence; Accordingly, the full-image frame sequence is input into the first liveness detection sub-model, including: The full-frame differential image sequence is input into the first liveness detection sub-model.

2. The method according to claim 1, characterized in that, Before inputting the face image frame sequence into the second liveness detection sub-model, the method further includes: The face image frame sequence is subjected to inter-frame difference processing to obtain the face frame difference image of any two adjacent face image frames in the face image frame sequence, and the face frame difference image sequence is determined. Accordingly, the face image frame sequence is input into the second liveness detection sub-model, including: The face frame difference image sequence is input into the second liveness detection sub-model.

3. The method according to claim 1, characterized in that, The first liveness detection sub-model includes a first temporal sub-network and a first fully connected sub-network configured in parallel. The number of the first fully connected sub-networks is equal to the number of the full-image frame difference images in the full-image frame difference image sequence. The step of performing liveness detection on the user under test based on the global region in each of the full-image frame using the first liveness detection sub-model to obtain a first liveness detection result for the user under test includes: The first temporal feature of the full-frame difference image sequence is extracted through the first temporal sub-network, and the liveness detection of the user under test is performed based on the first temporal feature to obtain a first temporal detection result. The first temporal feature represents the temporal change of each full-frame difference image. In addition, the global image feature of one full-frame difference image is extracted through each of the first fully connected sub-networks, and the liveness detection of the user under test is performed based on the global image feature to obtain a first fully connected detection result corresponding to each full-frame difference image. By fusing the first temporal detection result and each of the first fully connected detection results, the first liveness detection result corresponding to the full-image frame sequence is obtained.

4. The method according to claim 2, characterized in that, The second liveness detection sub-model includes a second temporal sub-network and a second fully connected sub-network. The number of the second fully connected sub-networks is equal to the number of face frame difference images in the face frame difference image sequence. The step of performing liveness detection on the user under test based on the local facial regions in each face image frame using the second liveness detection sub-model to obtain a second liveness detection result for the user under test includes: The second temporal features of the face frame difference image sequence are extracted through the second temporal sub-network, and liveness detection is performed on the user under test based on the second temporal features to obtain a second temporal detection result. The second temporal features represent the temporal changes of each face frame difference image. In addition, the face image features of one face frame difference image are extracted through each of the second fully connected sub-networks, and liveness detection is performed on the user under test based on the face image features to obtain a second fully connected detection result corresponding to each face frame difference image. By fusing the second temporal detection result and the second fully connected detection result, the second liveness detection result corresponding to the face image frame sequence is obtained.

5. The method according to claim 1, characterized in that, The acquisition of the full-image frame sequence and face image frame sequence when the user under test reads the target content includes: Obtain the original video of the user reading the target content; The original video is subjected to frame extraction to obtain a full-image frame sequence; Determine the facial key points in each of the full-image frames; The facial key points of each full-image frame are averaged to obtain the registration key points of each full-image frame. Based on the registration key points, the local facial regions in each of the full-image frames are centered to obtain the corresponding facial image frames, thereby determining the facial image frame sequence.

6. The method according to claim 1, characterized in that, The step of determining the target detection result of the user to be tested based on the first liveness detection result and the second liveness detection result includes: The first liveness detection result and the second liveness detection result are fused to obtain the target detection result of the user to be tested; wherein, the fusion processing includes taking the mean, taking the maximum value, or performing classification detection based on the first liveness detection result and the second liveness detection result.

7. The method according to claim 1, characterized in that, The training steps for the first and second liveness detection sub-models are as follows: Obtain the full-image frame sample sequence and the face image frame sample sequence; The entire image frame sample sequence is input into the initial first liveness detection sub-model to obtain the third liveness detection result output by the initial first liveness detection sub-model. Based on the third liveness detection result, the initial first liveness detection sub-model is iterated to obtain the first liveness detection sub-model; and... The face image frame sample sequence is input into the initial second liveness detection sub-model to obtain the fourth liveness detection result output by the initial second liveness detection sub-model. The initial second liveness detection sub-model is then iterated based on the fourth liveness detection result to obtain the second liveness detection sub-model.

8. An electronic device, characterized in that, The electronic device includes: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to execute the liveness detection method according to any one of claims 1-7 by executing the executable instructions.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the liveness detection method according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product stores a computer program, which, when executed by a processor, implements the liveness detection method according to any one of claims 1-7.

Images