Liveness detection method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a living body detection method, an electronic device and a storage medium. The method comprises: obtaining a to-be-detected video; inputting the to-be-detected video into a first living body detection model, performing two-dimensional convolution processing on the to-be-detected video based on different dimensions of target parameters by the first living body detection model, and obtaining facial action features of the to-be-detected video; wherein the target parameters are parameters related to video frames in the to-be-detected video; and identifying the facial action features by the first living body detection model to obtain a first living body detection identification result of whether the to-be-detected video is a fake action video. Embodiments of the present application improve the processing speed and improve the detection efficiency.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a liveness detection method, electronic device, and storage medium. Background Technology

[0002] With the rapid development of facial recognition and related technologies, many conveniences have been provided to people's lives and work, but at the same time, new challenges have been posed to information security. Some users often use synthetic facial data to forge identity information in order to pass the identity verification of facial recognition systems. Therefore, liveness detection technology has emerged, and there are currently many liveness detection methods. One effective method is to use video data as input and require the user to perform specific actions to perform liveness detection.

[0003] However, some attackers use software tools to synthesize videos containing prescribed actions; a single photograph can produce a video with a specific facial pose. Current technologies typically use network models for recognition of these action videos. However, the network models used in video liveness detection algorithms all face a trade-off between speed and accuracy. The biggest advantage of video algorithms lies in using 3D convolution to extract and utilize motion information from multiple frames, but this inevitably increases computational load and affects processing speed. Summary of the Invention

[0004] In view of the above problems, embodiments of this application are proposed to provide a liveness detection method, electronic device, and storage medium that overcomes or at least partially solves the above problems.

[0005] According to a first aspect of the embodiments of this application, a liveness detection method is provided, comprising:

[0006] Obtain the video of the target object to be inspected;

[0007] The video to be detected is input into a first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain the facial motion features of the video to be detected. The target parameters are parameters related to the video frames in the video to be detected.

[0008] The facial motion features are identified using the first liveness detection model to obtain a first liveness detection result indicating whether the video to be detected is a fake motion video.

[0009] According to a second aspect of the embodiments of this application, a liveness detection device is provided, comprising:

[0010] The video acquisition module is used to acquire the video of the target object to be detected;

[0011] The feature extraction module is used to input the video to be detected into a first liveness detection model, and to perform two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions by the first liveness detection model to obtain the facial action features of the video to be detected; wherein, the target parameters are parameters related to the video frames in the video to be detected.

[0012] The liveness detection module is used to identify the facial action features through the first liveness detection model to obtain a first liveness detection result as to whether the video to be detected is a fake action video.

[0013] According to a third aspect of the embodiments of this application, an electronic device is provided, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the liveness detection method as described in the first aspect.

[0014] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the liveness detection method as described in the first aspect.

[0015] According to a fifth aspect of the embodiments of this application, a computer program product is provided, characterized in that it includes a computer program or computer instructions, which, when executed by a processor, implement the liveness detection method described in the first aspect.

[0016] The liveness detection method, electronic device, and storage medium provided in this application embodiment input the video to be detected into a first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain facial action features of the video to be detected. The extracted facial action features are then identified to obtain a first liveness detection result as to whether the video to be detected is a fake action video. Since two-dimensional convolution is used to extract features from the video to be detected, it replaces the process of using three-dimensional convolution processing. Moreover, the use of two-dimensional convolution of different dimensions achieves the effect of three-dimensional convolution, which facilitates hardware acceleration, improves processing speed, and improves detection efficiency.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application.

[0019] Figure 1 This is a flowchart of the steps of a liveness detection method provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram illustrating the dimensional changes of using cubic two-dimensional convolution to achieve three-dimensional convolution in an embodiment of this application;

[0021] Figure 3 This is a flowchart of the training process of the first liveness detection model in the embodiments of this application;

[0022] Figure 4 This is a structural block diagram of a liveness detection device provided in an embodiment of this application;

[0023] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0025] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security and prevention, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement. This application also relates to computer vision technology, specifically a liveness detection method, the specific solution of which is as follows:

[0026] Figure 1 This is a flowchart of a liveness detection method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method may include:

[0027] Step 101: Obtain the video of the target object to be detected.

[0028] In this embodiment of the application, when performing liveness detection, the main focus is on detecting whether the video to be detected is a fake action video. Liveness detection requires detecting a video containing facial movements, which can be acquired via a camera. The target object can be a face, and the video to be detected includes the face and facial movement information.

[0029] Step 102: Input the video to be detected into the first liveness detection model, and perform two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain the facial action features of the video to be detected; wherein, the target parameters are parameters related to the video frames in the video to be detected.

[0030] The first liveness detection model can be obtained by modifying a neural network-based liveness detection model, changing the three-dimensional (3D) convolutional structure in the liveness detection model to a two-dimensional (2D) convolutional structure of different dimensions, that is, using a two-dimensional convolutional structure of different dimensions to replace the three-dimensional convolutional structure, and obtaining the same result as the three-dimensional convolutional processing.

[0031] The video to be detected is input into the first liveness detection model. In this model, three-dimensional convolution is achieved through two-dimensional convolution based on different dimensions. Then, two-dimensional convolution processing can be performed on the video to be detected based on target parameters of different dimensions to extract facial motion features. During the two-dimensional convolution processing of the video to be detected based on target parameters of different dimensions, the video first needs to be reconstructed in dimensions. Then, two-dimensional convolution is performed on the reconstructed data, followed by another dimensional reconstruction to convert the convolution result into a video data representation. During the convolution process, for two-dimensional convolution processing, a large number of evolutionary processing techniques can be used, such as residuals, attention, and feature pyramid structures, which raises the upper limit of the overall performance of the multi-frame spatiotemporal joint network model and improves the accuracy of feature extraction.

[0032] In one embodiment of this application, the step of performing two-dimensional convolution processing on the video to be detected using the first liveness detection model based on target parameters of different dimensions includes at least one of the following convolution processing methods:

[0033] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on width and height information;

[0034] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the height information and video frame information.

[0035] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the width information and the video frame information.

[0036] Wherein, the width information is the width of the video frame in the video to be detected, the height information is the height of the video frame in the video to be detected, and the video frame information is the number of video frames in the video to be detected.

[0037] When performing two-dimensional convolution processing on the video to be detected using the first liveness detection model, each two-dimensional convolution processing can be based on two different dimensions in the video to be detected. For example, two-dimensional convolution processing can be performed on the width and height directions of the video to be detected based on width information and height information. Two-dimensional convolution processing can also be performed on the height and video frame direction of the video to be detected based on height information and video frame information. Two-dimensional convolution processing can also be performed on the width and video frame direction of the video to be detected based on width information and video frame information.

[0038] By using two-dimensional convolutions of different dimensions instead of three-dimensional convolutions to extract facial motion features from the video to be detected, the computational load can be reduced and the processing speed can be improved.

[0039] Step 103: The facial motion features are identified using the first liveness detection model to obtain the first liveness detection result of whether the video to be detected is a fake motion video.

[0040] The first liveness detection model uses a fully connected layer to classify facial motion features into real videos and synthetic attack videos, thus obtaining a first liveness detection result to determine whether the video to be detected is a fake action video. A fake action video, also known as a synthetic attack video, is typically created by combining a facial image with a video containing motion.

[0041] The liveness detection method provided in this embodiment inputs the video to be detected into a first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain facial action features of the video to be detected. The extracted facial action features are then identified to obtain the first liveness detection result of whether the video to be detected is a fake action video. Since two-dimensional convolution is used to extract features from the video to be detected, it replaces the process of using three-dimensional convolution. Moreover, the use of two-dimensional convolution of different dimensions achieves the effect of three-dimensional convolution, which facilitates hardware acceleration, improves processing speed, and improves detection efficiency.

[0042] Based on the above technical solution, the step of performing two-dimensional convolution processing on the video to be detected using the first liveness detection model based on target parameters of different dimensions includes:

[0043] For each dimension of the target parameters, the first five-dimensional data matrix corresponding to the video to be detected is reconstructed into a four-dimensional data matrix using the first liveness detection model. Then, the four-dimensional data matrix is ​​subjected to two-dimensional convolution based on the target parameters of the dimension to obtain a first processing result. The first processing result is then reconstructed into a second five-dimensional data matrix. The dimension parameters of the first five-dimensional data matrix and the second five-dimensional data matrix are respectively the number of videos in the video to be detected, the number of video frames in the video to be detected, the number of image channels in the video frames in the video to be detected, and the width and height information of the video frames.

[0044] In the case of non-first two-dimensional convolution processing, the number of image channels in the first five-dimensional data matrix is ​​the same as the number of image channels in the second five-dimensional data matrix output from the previous two-dimensional convolution processing.

[0045] The video to be detected can be represented by a five-dimensional data matrix. When performing a two-dimensional convolution process on the target parameters of each dimension, the data dimensions are first reconstructed, that is, the first five-dimensional data matrix corresponding to the video to be detected is reconstructed into a four-dimensional data matrix. Then, a two-dimensional convolution process can be performed on this four-dimensional data matrix based on the target parameters of the stated dimensions to obtain a first processing result in the form of a four-dimensional data matrix. This first processing result is then reconstructed into a second five-dimensional data matrix. Specifically, the method for reconstructing the first five-dimensional data matrix into a four-dimensional data matrix is ​​to merge the parameters of two dimensions in the first five-dimensional data matrix. The method for reconstructing the first processing result into a second five-dimensional matrix is ​​to split the parameters of the merged dimensions in the first processing result back into the parameters of the original two dimensions.

[0046] The first five-dimensional data matrix can be represented as (N, T, C, W, H), where N represents the number of videos to be detected in the first liveness detection model input at one time, T represents the number of video frames in the video to be detected, C represents the number of image channels in each video frame (3 for ordinary color images), and W and H are the width and height information of each video frame, respectively. A five-dimensional data matrix (N, T, C, W, H) can be directly calculated using three-dimensional convolution. However, for two-dimensional convolution, the input object is usually a four-dimensional data matrix (N, C, W, H). Therefore, when using two-dimensional convolution to process the video to be detected, the dimensions of the data need to be reconstructed.

[0047] For example, a single 3D convolution can be achieved using three 2D convolutions. When performing the first 2D convolution on the video to be detected, the dimensions of the first 5D data matrix are reconstructed to a 4D data matrix. In this 4D data matrix, width and height information are guaranteed to be separate dimension parameters. This allows for 2D convolution in the width and height directions based on the width and height parameters, resulting in a first processing result represented by a 4D data matrix. The dimensions of this first processing result are the same as those of the 4D data matrix. The first processing structure is then reconstructed into a 5D data matrix to obtain a second 5D data matrix, thus achieving a single 2D convolution. During this process, the number of image channels can be adjusted; that is, the number of input channels can be adjusted to the number of output channels. For example, if the first 5D data matrix is ​​represented as (N, T, C1, W, H), then the second 5D data matrix is ​​represented as (N, T, C2, W, H), where C1 is the number of input channels and C2 is the number of output channels. If downsampling is required, it is also performed during this convolution process, that is, downsampling is performed on the first processing result, adjusting the width and height information of the image to 1 / 2 of the original.

[0048] When performing a second 2D convolution on the video to be detected, the output of the previous convolution is also subjected to 2D convolution. This output corresponds to a first 5D data matrix. The dimensions of this first 5D data matrix are then reconstructed to form a 4D data matrix. In this 4D data matrix, height information and video frame count are ensured to be separate dimension parameters. This allows for 2D convolution in the height and video frame count directions based on these parameters, yielding a first processing result represented as a 4D data matrix. The dimensions of this first processing result are the same as those of the first 5D data matrix. This first processing result is then reconstructed into a 5D data matrix to obtain a second 5D data matrix. The dimensions of the second 5D data matrix are the same as those of the first 5D data matrix.

[0049] When performing the third 2D convolution on the video to be detected, the output of the previous convolution is also subjected to 2D convolution. This output corresponds to the first 5D data matrix. The dimensions of this first 5D data matrix are then reconstructed to form a 4D data matrix. In this 4D data matrix, width and video frame count are ensured to be separate dimension parameters. This allows for 2D convolution in the width and video frame count directions based on these parameters, yielding a first processing result represented by the 4D data matrix. The dimension of this first processing result is the same as that of the 4D data matrix. This first processing result is then reconstructed into a 5D data matrix, resulting in the second 5D data matrix. This second 5D data matrix is ​​the output data of the three 2D convolution processes. This output data can be the final facial motion features or intermediate data obtained after processing the video through these three 2D convolutions. The dimensions of the second 5D data matrix are the same as those of the first 5D data matrix.

[0050] In the above-mentioned two-dimensional convolution process, when performing a non-first two-dimensional convolution process, the output result of the previous two-dimensional convolution process is processed. That is, the number of image channels in the first five-dimensional data matrix of the current process is the same as the number of image channels in the second five-dimensional data matrix output by the previous two-dimensional convolution process, and the target parameters of other dimensions are also the same.

[0051] For each dimension of the target parameters, two-dimensional convolution is performed after dimensional reconstruction, realizing the process of using two-dimensional convolution to replace three-dimensional convolution. This reduces the amount of computation required for convolution, improves processing speed, increases the efficiency of liveness detection, and meets the real-time requirements of the recognition system.

[0052] In one embodiment of this application, the step of reconstructing the first five-dimensional data matrix corresponding to the video to be detected into a four-dimensional data matrix through the first liveness detection model includes: merging the target parameters of the video quantity dimension and the target parameters of the target dimension in the first five-dimensional data matrix into parameters of the first joint dimension through the first liveness detection model to obtain the four-dimensional data matrix, wherein the target parameters of the target dimension include the number of video frames, the width information, or the height information;

[0053] The step of reconstructing the first processing result into a second five-dimensional data matrix includes: splitting the parameters of the first joint dimension in the first processing result into target parameters of the video quantity dimension and target parameters of the target dimension to obtain the second five-dimensional data matrix.

[0054] When reshaping the first five-dimensional data matrix into a four-dimensional data matrix, the target parameters of the video quantity dimension and the target dimension in the first five-dimensional data matrix are merged into a single target parameter. This merged dimension is used as the first joint dimension, resulting in a four-dimensional data matrix. The target parameter of the target dimension may include the video frame count, the width information, or the height information. The first processing result obtained after performing two-dimensional convolution on the four-dimensional data matrix is ​​also a four-dimensional data matrix. The dimensions of the first processing result are the same as those of the four-dimensional data matrix, including the first joint dimension. The first processing result is then reshaped to split the parameters of the first joint dimension into target parameters of the video quantity dimension and the video frame count dimension, resulting in a second five-dimensional data matrix.

[0055] When the target dimension is the number of video frames, the first five-dimensional data matrix is ​​represented as (N,T,C1,W,H). After dimension reconstruction, the four-dimensional data matrix is ​​represented as (N×T,C1,W,H), meaning the dimensional parameters of the four-dimensional data matrix include the parameters of the first joint dimension, the number of image channels, width information, and height information. The first processing result obtained after performing two-dimensional convolution on the four-dimensional data matrix is ​​also a four-dimensional data matrix, with dimensional parameters including the parameters of the first joint dimension, the number of image channels, width information, and height information. If the number of image channels needs to be changed, it can be changed during this two-dimensional convolution process. The dimensionality of the first processing result is then represented as (N×T,C2,W,H). Dimension reconstruction is then performed on the first processing result to split the parameters of the first joint dimension into target parameters for the number of video frames and the number of video frames, resulting in the second five-dimensional data matrix, represented as (N,T,C2,W,H). C1 represents the number of input channels, and C2 represents the number of output channels.

[0056] When the target dimension is the width information dimension, during non-first 2D convolution processing, the dimensions in the second 5D data matrix obtained from the previous convolution processing can be dim-shuffled. The width information in the fourth dimension and the video frame count in the second dimension can be dim-shuffled. After dim-shuffling, the first 5D data matrix to be processed in the current iteration is obtained. This first 5D data matrix is ​​represented as (N, W, C2, T, H). When reconstructing the first 5D data matrix into a 4D data matrix, the target parameters of the video count dimension and the width information dimension in the first 5D data matrix after dim-shuffling are merged into a single dimension. The merged dimension is used as the first joint dimension. After merging dimensions, a 4D data matrix is ​​obtained. The dimensions of this 4D data matrix are represented as (N×W, C2, T, H). That is, the dimensional parameters of the 4D data matrix include the parameters of the first joint dimension, the number of image channels, the number of video frames, and the height information. The first processing result obtained after performing two-dimensional convolution on the four-dimensional data matrix is ​​also a four-dimensional data matrix. The parameters of the dimension include the parameters of the first joint dimension, the number of image channels, the number of video frames, and the height information. The dimension of the first processing result is represented as (N×W,C2,T,H). The first processing result is then reconstructed to split the parameters of the first joint dimension in the first processing result into target parameters of the number of video frames and target parameters of the width information dimension, resulting in a second five-dimensional data matrix, which is also represented as (N,W,C2,T,H).

[0057] When the target dimension is the height information dimension, during the non-first 2D convolution processing, the dimensions in the second 5D data matrix obtained from the previous convolution processing can be transformed. The width information of the second dimension and the height information of the fifth dimension are transformed. After the transformation, the first 5D data matrix to be processed is obtained. This first 5D data matrix is ​​represented as (N,H,C2,T,W). When reconstructing the first 5D data matrix into a 4D data matrix, the target parameters of the video quantity dimension and the target parameters of the height information dimension in the first 5D data matrix are merged into a single dimension. The merged dimension is used as the first joint dimension. After merging the dimensions, a 4D data matrix is ​​obtained. The dimensions of this 4D data matrix are represented as (N×H,C2,T,W). That is, the parameters of the 4D data matrix include the parameters of the first joint dimension, the number of image channels, the number of video frames, and the width information. The first processing result obtained after performing two-dimensional convolution on the four-dimensional data matrix is ​​also a four-dimensional data matrix. The parameters of the dimension include the parameters of the first joint dimension, the number of image channels, the number of video frames, and the width information. The dimension of the first processing result is represented as (N×H,C2,T,W). The first processing result is then reconstructed to split the parameters of the first joint dimension into target parameters of the number of video frames and the target parameters of the height information dimension, resulting in a second five-dimensional data matrix, represented as (N,H,C2,T,W). The data dimensions of the second five-dimensional data matrix are then transformed: the number of video frames is transformed from the fourth dimension to the second dimension, the width information is transformed from the fifth dimension to the fourth dimension, and the height information is transformed from the second dimension to the fifth dimension. The second data matrix after the data dimensions are transformed is the output result of performing one three-dimensional convolution using three two-dimensional convolutions. The dimension of the output result is represented as (N,T,C2,W,H).

[0058] By merging the target parameters for the number of videos and the target dimensions in the first five-dimensional data matrix into the parameters of the first joint dimension, and after performing two-dimensional convolution processing, the first joint dimension in the first processing result is split into the target parameters for the number of videos and the target dimensions, and the first processing result is converted into a five-dimensional data matrix to obtain the second five-dimensional data matrix. In this way, based on the reconstruction of data dimensions, a two-dimensional convolution processing is achieved.

[0059] Figure 2 This is a schematic diagram illustrating the dimensionality change of using cubic two-dimensional convolution to achieve three-dimensional convolution in an embodiment of this application, as shown below. Figure 2As shown, the input data for implementing a single three-dimensional convolution using cubic two-dimensional convolution is a first five-dimensional data matrix, whose dimensions can be represented as (N,T,C1,W,H). After reconstructing the dimensions of the first five-dimensional data matrix, the resulting four-dimensional data matrix can be represented as (N×T,C1,W,H). Performing two-dimensional (2D) convolution on this four-dimensional data matrix in both width and height directions yields a first processing result whose dimensions are represented as (N×T,C2,W,H). After reconstructing the dimensions of the first processing result, the resulting second five-dimensional data matrix is ​​represented as (N,W,C2,T,H). This process implements a single three-dimensional convolution. If downsampling is required during 3D convolution, the first processing result is downsampled before dimension reconstruction. The dimensions of the second five-dimensional data matrix are transformed to obtain the input data for the second 2D convolution, which is the first five-dimensional data matrix for the second 2D convolution. The dimensions of this first five-dimensional data matrix are represented as (N, W, C2, T, H). After dimension reconstruction of this first five-dimensional data matrix, the resulting four-dimensional data matrix can be represented as (N×W, C2, T, H). This four-dimensional data matrix is ​​then subjected to 2D convolution along the height and video frame number directions to obtain the dimension of the first processing result. The dimensions of the second five-dimensional data matrix obtained after dimensional reconstruction of the first processing result are represented as (N×W,C2,T,H). The dimensions of the second five-dimensional data matrix are then transformed to obtain the input data for the third two-dimensional convolution process, which is the first five-dimensional data matrix of the third two-dimensional convolution process. The dimensions of this first five-dimensional data matrix are represented as (N,H,C2,T,W). After dimensional reconstruction of this first five-dimensional data matrix, the dimensions of the resulting four-dimensional data matrix can be represented as (N×H,C2,T,W). The dimensions of this four-dimensional data matrix are then determined by the width and video frame number directions. The first processing result obtained by the two-dimensional convolution is represented by the dimension (N×H,C2,T,W). The dimension of the second five-dimensional data matrix obtained by reconstructing the dimension of the first processing result is represented by the dimension (N,H,C2,T,W). The dimension position of the second five-dimensional data matrix is ​​transformed so that the dimension position of the second five-dimensional data matrix is ​​consistent with the dimension position order of the first five-dimensional data matrix during the first two-dimensional convolution. Then the dimension position transformed second five-dimensional data matrix is ​​represented by (N,T,C2,W,H). This dimension position transformed second five-dimensional data matrix is ​​the output result of using three two-dimensional convolutions to achieve one three-dimensional convolution.

[0060] The two-dimensional convolution in the first liveness detection model can refer to any two-dimensional convolutional neural network model. The structure that uses three two-dimensional convolutions to achieve one three-dimensional convolution is replaced with the two-dimensional convolution in the two-dimensional convolutional neural network model. In this way, the feature extraction task of a five-dimensional data matrix (N,T,C,W,H) can be achieved. Furthermore, the evolutionary processing of two-dimensional convolution, such as residuals, attention, and feature pyramid structures, can be used during the two-dimensional convolution process to improve the overall performance of the model.

[0061] When implementing 3D convolution using triple 2D convolution, the effect of 3D convolution can be simulated by repeatedly combining different dimensions using only 2D convolution operations. Practice has proven that this design can achieve comparable results to 3D convolution in multiple tasks and is easy to accelerate with hardware, showing a speed advantage far exceeding that of 3D convolution. If downsampling is required, downsampling is only performed in the spatial dimensions of width (W) and height (H), without downsampling in the temporal dimension of video frames (T). This approach can preserve the integrity of motion information to the greatest extent.

[0062] Based on the above technical solution, the method further includes: obtaining a second liveness detection result output by another liveness detection model deployed in parallel with the first liveness detection model; wherein the second liveness detection result is the detection result of the video to be detected or the detection result of the target object; and determining the liveness detection result of the target object based on the first liveness detection result and the second liveness detection result.

[0063] When deploying the first liveness detection model, other liveness detection models with different functions can be deployed in parallel, such as a second liveness detection model to detect whether a video is a copy. In this case, the second liveness detection results output by the other parallel liveness detection models can be obtained. Combining the first and second liveness detection results, the liveness detection result of the target object can be determined. If there are non-liveness detection results in both the first and second liveness detection results, the target object's liveness detection result is determined to be non-live. If both the first and second liveness detection results are live, the target object's liveness detection result is determined to be live. For example, if the first detection model outputs a first detection result indicating that the video to be detected is a fake action video targeting the target object (meaning the video does not contain a live target), then regardless of whether the second liveness detection result indicates a live target, the target object's liveness detection result can be determined to be non-live.

[0064] The first liveness detection model can be integrated in parallel with other existing liveness detection models during deployment to achieve liveness detection against different types of attacks and improve detection accuracy.

[0065] Figure 3 This is a flowchart of the training process of the first liveness detection model in the embodiments of this application, such as... Figure 3 As shown, the training process of the first liveness detection model includes:

[0066] Step 301: Obtain a sample set, which includes multiple real videos and multiple synthetic attack videos. Each real video and each synthetic attack video is a sample video. The synthetic attack video is synthesized from a source domain image and a target domain action video.

[0067] The source domain image and target domain motion video can include the faces of different target objects. The synthesized video is obtained by transferring the facial movements from the target domain motion video to the facial regions in the source domain image. In other words, the facial regions in the source domain are made to move according to the facial movements in the target domain video (including but not limited to head shaking, blinking, and mouth opening), thus obtaining a synthesized attack video of the source domain face. This synthesized attack video is used as a fake motion video for attack. The synthesized attack video can be synthesized using various methods, such as 3DFaceWarp, FirstOrder, and MugLife. In practical application, the proportion of each principle can be determined based on the synthesis methods of real-world fake motion videos, and the synthesis method can be selected accordingly. This sample set includes the real video and the real video. Alternatively, only one synthesis method can be used. The real video is a video of the actual facial movements of the target object.

[0068] Step 302: Train the first liveness detection model to be trained based on the sample set to obtain the trained first liveness detection model.

[0069] Select a target batch number of sample videos from the sample set, and input the target batch number of sample videos into the multi-frame spatiotemporal joint network model. The multi-frame spatiotemporal joint network model extracts features from the target batch number of sample videos to obtain the video features of each sample video.

[0070] A target batch size of sample videos is selected from the sample set and input into the first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the sample videos based on target parameters of different dimensions to obtain the facial motion features of each sample video. Here, the target batch size is the number of sample videos that need to be input into the multi-frame spatiotemporal joint network model for training a batch.

[0071] The facial motion features of each sample video are classified by the fully connected layer in the first liveness detection model to obtain the detection result of each sample video. The detection result can include the confidence score corresponding to the real video and the confidence score corresponding to the synthetic attack video.

[0072] Based on the recognition results of each sample video and the annotation results indicating whether each sample video is a real video or a synthetic video, the network parameters of the first liveness detection model are adjusted. After one training batch, it is determined whether the training termination condition is met. If the training termination condition is not met, the above steps are iteratively executed, namely, iteratively selecting sample videos from the sample set, performing two-dimensional convolution processing on the sample videos through the first liveness detection model, and adjusting the network parameters of the first liveness detection model, until the training termination condition is met, the training ends, and the trained first liveness detection model is obtained.

[0073] The first liveness detection model trained using synthetic attack videos and real videos in the above manner can effectively defend against real fake action attacks, and can be integrated in parallel with other existing liveness detection models during deployment to achieve liveness detection against different types of attacks.

[0074] Based on the above technical solution, the synthetic attack video is synthesized through the following steps:

[0075] Obtain facial key points of the face region in the source domain image as source domain facial key points; and obtain facial key points of the face region in each frame of the target domain image in the target domain motion video as target domain facial key points.

[0076] Based on the source domain facial key points and the target domain facial key points, the facial pose information in the facial region of the target domain image in each frame is transferred to the facial region of the source domain image using the corresponding facial key points, thereby obtaining the source domain facial action sequence.

[0077] The source domain facial motion sequences are synthesized into the source domain images to generate a synthesized video.

[0078] A face detection model can be used to detect faces in the source domain image, obtaining facial key points (including but not limited to 5 points, 68 points, and 81 points) in the source domain image. The face detection model is then used to detect faces in each frame of the target domain motion video, obtaining facial key points for each frame. Face alignment is performed between the source domain image and each frame of the target domain image using these facial key points. Based on the facial key points in the source domain image, a face region in the source domain image is obtained as the source domain face region S. Similarly, based on the facial key points in each frame of the target domain image, a face region in each frame of the target domain image is obtained. This face region in each frame of the target domain image is then used as the target domain face region (t1, t2, ..., t...). n The source domain face region S and the target domain face region (t1, t2, ..., t) of each frame are compared. n () is used as input for synthesizing attack videos.

[0079] Based on the target domain facial key points in each frame of the target domain image, the source domain facial key points of the source domain face region S are transformed to transfer the facial pose information of each frame's target domain face region to the face region in the source domain image, thus obtaining the source domain facial action sequence (s1, s2, ..., s...). n Based on the facial key points in the source domain, the facial motion sequence (s1, s2, ..., s...) is generated. n The images are pasted into the source domain image respectively to obtain multiple images as video frames. According to the FPS (Frames Per Second) parameter of the target domain action video, these video frames including multiple images are combined into a video to obtain the synthetic attack video, which is the fake action video.

[0080] By using facial key points in the source domain image and facial key points in each frame of the target domain motion video, the facial pose information of each frame in the target domain motion video can be accurately transferred to the source domain image, improving the effect of synthesized attack videos.

[0081] In one embodiment of this application, based on the source domain facial key points and the target domain facial key points of each frame, the facial pose information in the facial region of each frame of the target domain image is transferred to the facial region of the source domain image using corresponding key points, to obtain a source domain facial action sequence, including:

[0082] For each frame of the target domain image, the position of the target domain facial key points in the face region of the source domain image is determined based on the source domain facial key points.

[0083] Based on the location and the key facial points in the source domain, the facial pose information in the facial region of the source domain image is adjusted to the facial pose information in the target domain image through triangulation to obtain the source domain composite image;

[0084] The source domain composite images obtained from each frame of the target domain image are sorted according to the order of the target domain images to obtain the source domain facial action sequence.

[0085] When using the 3DFaceWarp principle to generate composite videos, the input is the source domain face region S and the corresponding source domain face keypoints lmk. s The first frame of the target domain motion video shows the target domain face region t1 and the corresponding target domain facial key points lmk. t Based on the source domain facial key points lmk s Calculate the facial key points lmk in the target domain image of the first frame. t The position lmk on the face region S in the source domain. ts The source domain face region S is adjusted to the pose of the target domain face region t1 in the first frame using 3D triangulation technology. Specifically, triangulation is used based on the target domain face key points lmk in the first frame. t The position lmk on the face region S in the source domain. ts lmk for facial key points in the target domain t Position adjustments and color rendering are performed to obtain the source domain composite image s1 corresponding to the first frame of the source domain face region S. For (t1, t2, ..., t...) n The above synthesis operation is repeated for each frame to obtain the source domain facial motion sequence (s1, s2, ..., s...). n ).

[0086] When transferring facial pose information through triangulation, in order to transfer the facial region (t1, t2, ..., t) of the target domain in each frame... n To transfer facial pose information from the source domain to the target domain face region S, the source domain facial key points in the source domain face region S need to be aligned with the target domain face region (t1, t2, ..., t3) in each frame. n The target domain facial keypoints in the source domain face region are used. To align the source domain facial keypoints in the source domain face region with the target domain facial keypoints in each frame, four keypoints targeting the boundary corners of the original image are added to the existing facial keypoints. A triangulation algorithm is used to construct triangles between the different keypoints. The triangles generated from the source domain facial keypoints in the source domain face region S are then mapped to the target domain facial keypoints lmk in each frame of the target domain face region. t The position lmk on the face region S in the source domain. tsWithin the generated triangles, keypoint correction is performed, resulting in the source domain facial motion sequence (s1, s2, ..., s...). n This allows for changes in facial posture.

[0087] The source domain facial motion sequence obtained by the above method cannot accurately generate large-pose movements after generating the synthetic attack video, but it can preserve the facial details in the synthetic domain video.

[0088] In another embodiment of this application, the step of transferring the facial pose information in the facial region of each frame of the target domain image to the facial region of the source domain image based on the source domain facial key points and the target domain facial key points to obtain the source domain facial action sequence includes: inputting the facial region of the source domain image and the facial region of each frame of the target domain image in the target domain action video into a video synthesis model, and transferring the facial pose information in the facial region of each frame of the target domain image to the facial region of the source domain image based on the corresponding facial key points through the video synthesis model to obtain the source domain facial action sequence.

[0089] The video synthesis model can be a FirstOrder pre-trained model that uses affine transformations to describe motion.

[0090] The face region S in the source domain image and the face region (t1, t2, ..., t3) in each frame of the target domain motion video are compared. n The input is fed into a video synthesis model, which transfers the facial pose information from the face region in each frame of the target domain image to the face region in the source domain image, using the corresponding facial key points. This involves transferring facial pose information for the same identified facial key points, directly generating the source domain facial action sequence (s1, s2, ..., s...) end-to-end. n ).

[0091] The FirstOrder-based approach is a deep learning-based method. The FirstOrder-based video synthesis model is trained on a large amount of facial data, so even if the movements in the generated synthesized video are not completely accurate, the overall facial features can still maintain reasonable coordination.

[0092] In another embodiment of this application, the step of migrating the facial pose information in the facial region of each frame of the target domain image to the facial region of the source domain image based on the source domain facial key points and the target domain facial key points to obtain a source domain facial motion sequence includes: determining the key point coordinate transformation matrix of the facial region between every two adjacent frames in the target domain motion video; performing transformation processing on the facial region in the source domain image according to the key point coordinate transformation matrix of the facial region between every two adjacent frames to obtain an initial source domain facial motion sequence composed of multiple synthetic source domain facial regions; and performing detail restoration processing on each synthetic source domain facial region in the initial source domain facial motion sequence to obtain the source domain facial motion sequence.

[0093] When using the MugLife principle to generate synthetic attack videos, optical flow tracing is used to calculate the target domain action video (t1, t2, ..., t...). n The transformation matrix of key point coordinates of the facial region between each two adjacent frames is used to construct a triangular network deformation. Based on the transformation matrix of key point coordinates of the facial region between each two adjacent frames, the source domain facial region is transformed. The background change is recorded using optical flow tracing, resulting in an initial source domain facial motion sequence composed of multiple synthesized source domain facial regions, i.e., initially obtaining n motion states of the facial region S in the source domain image. Detail restoration processing is performed on each synthesized source domain facial region in the initial source domain facial motion sequence, i.e., Poisson fusion is used to fill the mouth, and frequency domain transformation is used to migrate facial wrinkle shadows. These two fine-tuning processes on the n motion states of the facial region S in the source domain image yield the source domain facial motion sequence (s1, s2, ..., s...). n ).

[0094] The method based on MugLife is a kinematic optical flow analysis approach that is not trained on facial data. Therefore, the resulting synthetic attack videos may have localized lighting inconsistencies, but its advantage lies in its ability to simulate any motion posture.

[0095] In this embodiment of the application, when generating synthetic attack videos to form a sample set, the three methods mentioned above can be randomly selected according to a certain proportion. This allows the advantages of the three methods to complement each other in the data distribution of the sample set, resulting in a diversity of synthetic attack videos. The first liveness detection model trained with such data will be more robust to real attacks. The defense performance of the first liveness detection model is easy to maintain stability over a long period of time. Furthermore, due to the high diversity of the training data (synthetic attack videos), this defense capability can remain at a high level over a long period of time and will not decline rapidly due to minor changes in the attack method. Thus, the active defense task against facial attacks in fake action videos is achieved without relying on massive amounts of real attack data.

[0096] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0097] Figure 4 This is a structural block diagram of a liveness detection device provided in an embodiment of this application, as shown below. Figure 4 As shown, the liveness detection device may include:

[0098] Video acquisition module 401 is used to acquire the video to be detected of the target object;

[0099] The feature extraction module 402 is used to input the video to be detected into a first liveness detection model, and to perform two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions by the first liveness detection model to obtain the facial action features of the video to be detected; wherein, the target parameters are parameters related to the video frames in the video to be detected.

[0100] The liveness detection module 403 is used to identify the facial action features through the first liveness detection model to obtain a first liveness detection result as to whether the video to be detected is a fake action video.

[0101] Optionally, the feature extraction module is used to perform at least one of the following convolutional processes:

[0102] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on width and height information;

[0103] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the height information and video frame information.

[0104] The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the width information and the video frame information.

[0105] Wherein, the width information is the width of the video frame in the video to be detected, the height information is the height of the video frame in the video to be detected, and the video frame information is the number of video frames in the video to be detected.

[0106] Optionally, the feature extraction module is specifically used for:

[0107] For each dimension of the target parameters, the first five-dimensional data matrix corresponding to the video to be detected is reconstructed into a four-dimensional data matrix using the first liveness detection model. Then, the four-dimensional data matrix is ​​subjected to two-dimensional convolution based on the target parameters of the dimension to obtain a first processing result. The first processing result is then reconstructed into a second five-dimensional data matrix. The dimension parameters of the first five-dimensional data matrix and the second five-dimensional data matrix are respectively the number of videos in the video to be detected, the number of video frames in the video to be detected, the number of image channels in the video frames in the video to be detected, and the width and height information of the video frames.

[0108] In the case of non-first two-dimensional convolution processing, the number of image channels in the first five-dimensional data matrix is ​​the same as the number of image channels in the second five-dimensional data matrix output from the previous two-dimensional convolution processing.

[0109] Optionally, the feature extraction module includes:

[0110] The first dimension reconstruction unit is used to merge the target parameters of the video quantity dimension and the target parameters of the target dimension in the first five-dimensional data matrix into the parameters of the first joint dimension through the first liveness detection model to obtain the four-dimensional data matrix. The target parameters of the target dimension include the video frame count, the width information, or the height information.

[0111] The second dimension reconstruction unit is used to split the parameters of the first joint dimension in the first processing result into target parameters of the video quantity dimension and target parameters of the target dimension, so as to obtain the second five-dimensional data matrix.

[0112] Optionally, the device further includes:

[0113] The second result acquisition module is used to acquire the second liveness detection result output by other liveness detection models deployed in parallel with the first liveness detection model; wherein, the second liveness detection result is the detection result of the video to be detected or the detection result of the target object;

[0114] The detection result determination module is used to determine the liveness detection result of the target object based on the first liveness detection result and the second liveness detection result.

[0115] Optionally, the device further includes a model training module, the model training module comprising:

[0116] A sample set acquisition unit is used to acquire a sample set, which includes multiple real videos and multiple synthetic attack videos. Each real video and each synthetic attack video is a sample video. The synthetic attack video is synthesized from a source domain image and a target domain action video.

[0117] The model training unit is used to train the first liveness detection model to be trained based on the sample set, so as to obtain the first liveness detection model after training.

[0118] Optionally, the model training module further includes a video synthesis unit, which includes:

[0119] The key point acquisition subunit is used to acquire facial key points in the face region of the source domain image as source domain facial key points; and to acquire facial key points in the face region of each frame of the target domain motion video as target domain facial key points.

[0120] The pose information transfer subunit is used to transfer the facial pose information in the facial region of the target domain image in each frame to the facial region of the source domain image according to the source domain facial key points and the target domain facial key points, so as to obtain the source domain facial action sequence.

[0121] The video synthesis subunit is used to synthesize the source domain facial action sequences into the source domain images respectively, and generate a synthesized attack video.

[0122] Optionally, the attitude information transfer subunit is specifically used for:

[0123] For each frame of the target domain image, the position of the target domain facial key points in the face region of the source domain image is determined based on the source domain facial key points.

[0124] Based on the location and the key facial points in the source domain, the facial pose information in the source domain image is adjusted to the facial pose information in the target domain image through triangulation to obtain a synthesized image in the source domain.

[0125] The source domain composite images obtained from each frame of the target domain image are sorted according to the order of the target domain images to obtain the source domain facial action sequence.

[0126] Optionally, the attitude information transfer subunit is specifically used for:

[0127] The facial regions in the source domain image and the facial regions in each frame of the target domain motion video are input into the video synthesis model. The facial pose information in the facial regions of each frame of the target domain image is transferred to the facial regions in the source domain image using the corresponding facial key points through the video synthesis model to obtain the source domain facial motion sequence.

[0128] Optionally, the attitude information transfer subunit is specifically used for:

[0129] Determine the transformation matrix of key point coordinates of the facial region between every two adjacent frames in the target domain motion video;

[0130] Based on the transformation matrix of the key point coordinates of the facial region between each two adjacent frames, the facial region in the source domain image is transformed to obtain an initial source domain facial action sequence composed of multiple synthesized source domain facial regions.

[0131] The source domain facial action sequence is obtained by performing detail restoration processing on each synthesized source domain facial region in the initial source domain facial action sequence.

[0132] For the specific implementation process of the functions corresponding to each module and unit in the device provided in this application embodiment, please refer to... Figures 1-3 The specific implementation process of the functions corresponding to each module and unit of the device is not described in detail here as shown in the method embodiment.

[0133] The liveness detection device provided in this embodiment inputs the video to be detected into a first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain facial action features of the video to be detected. The extracted facial action features are then identified to obtain the first liveness detection result of whether the video to be detected is a fake action video. Since feature extraction is performed on the video to be detected, the process of using three-dimensional convolution processing is replaced. Moreover, the effect of three-dimensional convolution is achieved by using two-dimensional convolution of different dimensions, which facilitates hardware acceleration, improves processing speed, and improves recognition efficiency.

[0134] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0135] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the electronic device 500 may include one or more processors 510 and one or more memories 520 connected to the processors 510. The electronic device 500 may also include an input interface 530 and an output interface 540 for communicating with another device or system. Program code executed by the processor 510 may be stored in the memory 520.

[0136] The processor 510 in the electronic device 500 calls the program code stored in the memory 520 to execute the liveness detection method in the above embodiment.

[0137] According to one embodiment of this application, a computer-readable storage medium is also provided, including but not limited to disk storage, CD-ROM, optical storage, etc., wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the liveness detection method of the foregoing embodiment.

[0138] According to one embodiment of this application, a computer program product is also provided, including a computer program or computer instructions, which, when executed by a processor, implement the liveness detection method described in the above embodiments.

[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0140] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0145] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0146] The present application provides a detailed description of a liveness detection method, electronic device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present application. Therefore, the content of this specification should not be construed as a limitation of the present application.

Claims

1. A method for detecting liveness, characterized in that, include: Obtain the video of the target object to be inspected; The video to be detected is input into a first liveness detection model. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on target parameters of different dimensions to obtain the facial motion features of the video to be detected. The target parameters are parameters related to the video frames in the video to be detected. The facial movement features are identified by the first liveness detection model to obtain the first liveness detection result of whether the video to be detected is a fake action video; The step of performing two-dimensional convolution processing on the video to be detected using the first liveness detection model based on target parameters of different dimensions includes: For each dimension of the target parameters, the first five-dimensional data matrix corresponding to the video to be detected is reconstructed into a four-dimensional data matrix using the first liveness detection model. Then, the four-dimensional data matrix is ​​subjected to two-dimensional convolution based on the target parameters of the dimension to obtain a first processing result. The first processing result is then reconstructed into a second five-dimensional data matrix. The dimension parameters of the first five-dimensional data matrix and the second five-dimensional data matrix are respectively the number of videos in the video to be detected, the number of video frames in the video to be detected, the number of image channels in the video frames in the video to be detected, and the width and height information of the video frames. In the case of non-first two-dimensional convolution processing, the number of image channels in the first five-dimensional data matrix is ​​the same as the number of image channels in the second five-dimensional data matrix output from the previous two-dimensional convolution processing.

2. The method according to claim 1, characterized in that, The step of performing two-dimensional convolution processing on the video to be detected using the first liveness detection model based on target parameters of different dimensions includes at least one of the following convolution processing methods: The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on width and height information; The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the height information and video frame information. The first liveness detection model performs two-dimensional convolution processing on the video to be detected based on the width information and the video frame information. Wherein, the width information is the width of the video frame in the video to be detected, the height information is the height of the video frame in the video to be detected, and the video frame information is the number of video frames in the video to be detected.

3. The method according to claim 1, characterized in that, The step of reconstructing the first five-dimensional data matrix corresponding to the video to be detected into a four-dimensional data matrix using the first liveness detection model includes: The first liveness detection model merges the target parameters of the video quantity dimension and the target parameters of the target dimension in the first five-dimensional data matrix into the parameters of the first joint dimension to obtain the four-dimensional data matrix. The target parameters of the target dimension include the video frame count, the width information, or the height information. The step of reconstructing the first processing result into a second five-dimensional data matrix includes: The parameters of the first joint dimension in the first processing result are split into target parameters of the video quantity dimension and target parameters of the target dimension to obtain the second five-dimensional data matrix.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain the second liveness detection result output by other liveness detection models deployed in parallel with the first liveness detection model; wherein, the second liveness detection result is the detection result of the video to be detected or the detection result of the target object; Based on the first liveness detection result and the second liveness detection result, the liveness detection result of the target object is determined.

5. The method according to any one of claims 1-4, characterized in that, The training process of the first liveness detection model includes: A sample set is obtained, which includes multiple real videos and multiple synthetic attack videos. Each real video and each synthetic attack video is a sample video. The synthetic attack video is synthesized from a source domain image and a target domain action video. The first liveness detection model to be trained is trained based on the sample set to obtain the first liveness detection model after training.

6. The method according to claim 5, characterized in that, The synthetic attack video was obtained through the following steps: Obtain facial key points in the face region of the source domain image as source domain facial key points; and obtain facial key points in the face region of each frame of the target domain image in the target domain motion video as target domain facial key points. Based on the source domain facial key points and the target domain facial key points, the facial pose information in the facial region of the target domain image in each frame is transferred to the facial region of the source domain image using the corresponding facial key points, thereby obtaining the source domain facial action sequence. The source domain facial motion sequences are synthesized into the source domain images to generate a synthetic attack video.

7. The method according to claim 6, characterized in that, Based on the source domain facial key points and the target domain facial key points, the facial pose information in the facial region of each frame of the target domain image is transferred to the facial region of the source domain image using the corresponding facial key points, to obtain a source domain facial action sequence, including: For each frame of the target domain image, the position of the target domain facial key points in the face region of the source domain image is determined based on the source domain facial key points. Based on the location and the key facial points in the source domain, the facial pose information in the facial region of the source domain image is adjusted to the facial pose information in the target domain image through triangulation to obtain the source domain composite image; The source domain composite images obtained from each frame of the target domain image are sorted according to the order of the target domain images to obtain the source domain facial action sequence.

8. The method according to claim 6, characterized in that, The step of transferring facial pose information from the face region in the target domain image to the face region in the source domain image based on the source domain facial key points and the target domain facial key points to obtain a source domain facial action sequence includes: The facial regions in the source domain image and the facial regions in each frame of the target domain motion video are input into the video synthesis model. The facial pose information in the facial regions of each frame of the target domain image is transferred to the facial regions in the source domain image using the corresponding facial key points through the video synthesis model to obtain the source domain facial motion sequence.

9. The method according to claim 6, characterized in that, The step of transferring facial pose information from the face region in the target domain image to the face region in the source domain image based on the source domain facial key points and the target domain facial key points to obtain a source domain facial action sequence includes: Determine the transformation matrix of key point coordinates of the facial region between every two adjacent frames in the target domain motion video; Based on the transformation matrix of the key point coordinates of the facial region between each two adjacent frames, the facial region in the source domain image is transformed to obtain an initial source domain facial action sequence composed of multiple synthesized source domain facial regions. The source domain facial action sequence is obtained by performing detail restoration processing on each synthesized source domain facial region in the initial source domain facial action sequence.

10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the liveness detection method as described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the liveness detection method as described in any one of claims 1-9.

12. A computer program product, characterized in that, It includes a computer program or computer instructions that, when executed by a processor, implement the liveness detection method according to any one of claims 1 to 9.

Images