A living body recognition method and device

Abstract

The present disclosure provides a kind of living body identification method, device.The method can be for each target video frame of target video, respectively according to each attribute feature information of the target video frame each respectively corresponding context perception feature information, determine the target attribute state corresponding to the target video frame, so that when determining the target attribute state corresponding to each target video frame, the attention image area of the target attribute state of the attribute of target video frame can be flexibly learned respectively, and rich context global information is also fused, so that the discrimination of each branch is very accurate, i.e. the identification result of the target attribute state of each attribute of target video frame is more accurate, so as to ensure that the living body identification result corresponding to target video determined according to the target attribute state corresponding to each target video frame is more accurate, and then improve user experience.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method and apparatus for liveness detection. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. It offers advantages such as being non-intrusive and contactless. With improvements in the accuracy of facial recognition algorithms and the development of large-scale parallel computing technology, facial recognition applications have gradually become commercialized. Facial recognition systems are increasingly being used in security, finance, e-commerce, and other scenarios requiring identity verification, such as remote bank account opening, access control systems, and remote transaction verification.

[0003] To improve the security and reliability of facial recognition systems, liveness detection technology has gradually become a core technology. Liveness detection technology can determine whether the detected object is a living individual, rather than an inanimate object such as a photograph or video, thus preventing malicious attackers from using recorded videos, photographs, 3D facial models, or forged masks to launch malicious attacks. However, current liveness detection technology has relatively poor recognition accuracy, which affects response speed and user experience. Summary of the Invention

[0004] In view of this, the present disclosure provides a liveness detection method, apparatus, computer device, and computer-readable storage medium to solve the problem that the poor detection accuracy of current liveness detection technologies affects response speed and user experience.

[0005] A first aspect of this disclosure provides a liveness detection method, the method comprising:

[0006] Acquire the target video and extract several target video frames from the target video;

[0007] For each target video frame, feature extraction is performed on the target video frame to obtain a first feature image; based on the first feature image, several attribute feature information is obtained, wherein the attribute corresponding to each attribute feature information is different; based on each attribute feature information, context-aware feature information corresponding to the attribute feature information is obtained; based on the context-aware feature information corresponding to each of the several attribute feature information, the target attribute state corresponding to the target video frame is determined.

[0008] Based on the target attribute states corresponding to each of the target video frames, the liveness detection result corresponding to the target video is determined.

[0009] A second aspect of this disclosure provides a liveness detection device, the device comprising:

[0010] A video frame acquisition unit is used to acquire a target video and extract several target video frames from the target video.

[0011] A state determination unit is configured to: extract features from each target video frame to obtain a first feature image; obtain several attribute feature information based on the first feature image, wherein each attribute feature information corresponds to a different attribute; obtain context-aware feature information corresponding to each attribute feature information; and determine the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information.

[0012] The result determination unit is used to determine the liveness detection result corresponding to the target video based on the target attribute state corresponding to each of the plurality of target video frames.

[0013] A third aspect of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0014] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of this disclosure embodiment compared with the prior art are as follows: After acquiring the target video, this disclosure embodiment can first extract several target video frames from the target video. Then, for each target video frame, feature extraction is performed on the target video frame to obtain a first feature image; based on the first feature image, several attribute feature information is obtained, wherein the attribute corresponding to each attribute feature information is different; based on each attribute feature information, context-aware feature information corresponding to the attribute feature information is obtained; based on the context-aware feature information corresponding to each of the several attribute feature information, the target attribute state corresponding to the target video frame is determined. Then, based on the target attribute state corresponding to each of the several target video frames, the liveness detection result corresponding to the target video can be determined. In this embodiment, for each target video frame, the target attribute state can be determined based on the context-aware feature information corresponding to each attribute feature information of that target video frame. This allows for flexible learning of the attention image region of the target attribute state of each attribute of the target video frame when determining the target attribute state of each target video frame. At the same time, it integrates rich global context information, making the discrimination of each branch very accurate. This results in more accurate recognition results of the target attribute states of each attribute of the target video frame, thereby ensuring that the liveness detection results of the target video determined based on the target attribute states of several target video frames are more accurate, and thus improving the user experience. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0018] Figure 2 This is a flowchart of the liveness detection method provided in the embodiments of this disclosure;

[0019] Figure 3 This is a block diagram of the liveness detection device provided in the embodiments of this disclosure;

[0020] Figure 4 This is a schematic diagram of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0022] A liveness detection method and apparatus according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0023] In existing technologies, multiple networks are typically used to determine various facial states and attributes in liveness detection tasks. However, deploying multiple network models at the edge is not conducive to success, leading to problems such as numerous models, slow speed, and low recognition accuracy, thus affecting response speed and user experience. Therefore, a new liveness detection method for facial recognition is needed.

[0024] To address the aforementioned issues, this invention provides a liveness detection method. In this method, for each target video frame, the target attribute state is determined based on the context-aware feature information corresponding to each attribute feature of that target video frame. This allows for flexible learning of the attention image region for each attribute's target attribute state when determining the target attribute state of the target video frame, while also incorporating rich global contextual information. This results in highly accurate discrimination across different branches, leading to more precise recognition results for each attribute's target attribute state of the target video frame. Consequently, the liveness detection results determined based on the target attribute states of several target video frames are more accurate, thereby improving the user experience.

[0025] For example, embodiments of the present invention can be applied to, for example... Figure 1 The application scenario shown can include terminal device 1 and server 2.

[0026] Terminal device 1 can be hardware or software. When terminal device 1 is hardware, it can be various electronic devices with image acquisition capabilities and supporting communication with server 2, including but not limited to smartphones, tablets, laptops, and desktop computers; when terminal device 1 is software, it can be installed in the aforementioned electronic devices. Terminal device 1 can be implemented as multiple software programs or software modules, or as a single software program or software module; this embodiment of the disclosure does not impose any limitations on this. Server 2 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 2 can be a single server, a server cluster consisting of several servers, or a cloud computing service center; this embodiment of the disclosure does not impose any limitations on this.

[0027] It should be noted that server 2 can be either hardware or software. When server 2 is hardware, it can be various electronic devices that provide various services to terminal device 1. When server 2 is software, it can be multiple software programs or software modules that provide various services to terminal device 1, or it can be a single software program or software module that provides various services to terminal device 1. This disclosure does not impose any limitations on this aspect.

[0028] Terminal device 1 and server 2 can communicate via a network. The network can be a wired network using coaxial cable, twisted pair, or fiber optic connection, or a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), or Infrared. This disclosure does not limit the scope of the embodiments.

[0029] Specifically, a user can input a target video through terminal device 1, which then sends the target video to server 2. Server 2 first extracts several target video frames from the target video. Then, for each target video frame, server 2 performs feature extraction to obtain a first feature image; based on the first feature image, it obtains several attribute feature information, where each attribute feature information corresponds to a different attribute; based on each attribute feature information, it obtains context-aware feature information corresponding to that attribute feature information; based on the context-aware feature information corresponding to each of the several attribute feature information, it determines the target attribute state corresponding to the target video frame. Next, server 2 can determine the liveness detection result corresponding to the target video based on the target attribute states corresponding to each of the several target video frames. In this embodiment, for each target video frame, the target attribute state can be determined based on the context-aware feature information corresponding to each attribute feature information of that target video frame. This allows for flexible learning of the attention image region of the target attribute state of each attribute of the target video frame when determining the target attribute state of each target video frame. At the same time, it integrates rich global context information, making the discrimination of each branch very accurate. This results in more accurate recognition results of the target attribute states of each attribute of the target video frame, thereby ensuring that the liveness detection results of the target video determined based on the target attribute states of several target video frames are more accurate, and thus improving the user experience.

[0030] It should be noted that the specific types, quantities, and combinations of terminal device 1, server 2, and network can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any restrictions on this.

[0031] It should be noted that the above application scenarios are shown only for the purpose of understanding this disclosure, and the implementation of this disclosure is not limited in any way. On the contrary, the implementation of this disclosure can be applied to any applicable scenario.

[0032] Figure 2 This is a flowchart of a liveness detection method provided in an embodiment of this disclosure. Figure 2 A liveness detection method can be derived from Figure 1 The terminal device or server executes the command. For example... Figure 2 As shown, the liveness detection method includes:

[0033] S101: Acquire the target video and extract several target video frames from the target video.

[0034] In this embodiment, the target video can be understood as the image for which liveness detection is required. In one implementation, the target video can be the video for which liveness detection is required. As an example, the target video can be captured by a surveillance camera installed in a fixed location, captured by a mobile terminal device, or read from a storage device that pre-stores video. It should be noted that, in one implementation, the target video can also be a video extracted from another video.

[0035] Because the target video contains a large number of video frames, and some video frames are of poor quality (e.g., blurry or missing users), determining accurate liveness detection results is difficult and computationally intensive. Therefore, in one implementation, after acquiring the target video, several target video frames can be extracted. These target video frames can be understood as video frames in the target video whose image quality scores meet preset quality conditions. This allows for the calculation of human feature information in the target video based on these target video frames, thereby reducing the computational burden.

[0036] S102: For each target video frame, perform feature extraction on the target video frame to obtain a first feature image; obtain several attribute feature information based on the first feature image; obtain context-aware feature information corresponding to each attribute feature information; determine the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information.

[0037] After acquiring several target video frames, the target attribute state corresponding to each target video frame can be determined. It should be noted that the target attribute state corresponding to each target video frame can include multiple target attribute states. These target attribute states can be understood as the attribute state of a face within the target video frame. For example, the target attribute state corresponding to a target video frame can include the attribute state of the eyes (e.g., closed eyes, open eyes, half-open eyes), the attribute state of the mouth (e.g., closed mouth, open mouth, half-open mouth), and the attribute state of the face orientation (e.g., frontal view, side view).

[0038] Specifically, features can be extracted from the target video frame to obtain a first feature image. This first feature image includes batch dimension, channel number dimension, height dimension, and width dimension.

[0039] Then, based on the first feature map, several attribute feature information can be obtained. Each attribute feature information corresponds to a different attribute. These attribute feature information are local facial features, such as local features for the eyes, local features for the mouth, etc. Furthermore, the several attribute feature information includes at least two attribute feature information. For example, based on the first feature map, two attribute feature information can be obtained, and the attributes corresponding to these two attribute feature information can be eye attributes and mouth attributes, respectively.

[0040] Next, context-aware feature information corresponding to each attribute feature can be obtained. It is understood that in this embodiment, the context-aware feature information corresponding to the attribute feature is fused with long-range context information from the target video frame; that is, the context-aware feature information corresponding to the attribute feature learns spatial and channel context information. It should be noted that by extracting local features strongly correlated with the subtasks of each attribute feature, better context-aware feature information corresponding to the attribute feature can be obtained. Furthermore, the multi-branch extraction structure of the context-aware feature information corresponding to each attribute feature makes each branch more flexible and allows for more refined design based on different subtasks.

[0041] Next, the target attribute state corresponding to the target video frame can be determined based on the context-aware feature information corresponding to each of the several attribute feature information. In this embodiment, the target attribute state of each attribute feature information can be determined based on the context-aware feature information corresponding to each attribute feature information; and the target attribute states of all attribute feature information can be used as the target attribute state corresponding to the target video frame. In this way, the structure of determining the target attribute state of each attribute feature information through multiple branches makes each branch more flexible and allows for more refined design according to different subtasks.

[0042] S103: Determine the liveness detection result corresponding to the target video based on the target attribute state corresponding to each of the plurality of target video frames.

[0043] In this embodiment, after determining the target attribute states corresponding to each of several target video frames, if the target attribute states corresponding to each of the several target video frames satisfy the liveness attribute condition, then the liveness recognition result corresponding to the target video is determined to be that the target video includes a live person; if the target attribute states corresponding to each of the several target video frames do not satisfy the liveness attribute condition, then the liveness recognition result corresponding to the target video is determined to be that the target video does not include a live person. The liveness attribute condition is that the target attribute states of at least two target video frames with adjacent playback order are different. It should be noted that if the target attribute states of at least two target video frames with adjacent playback order are different, it indicates that the face in the target video has changed (e.g., changes in the eyes or mouth), which can further indicate that the target video includes a live person.

[0044] The beneficial effects of this disclosure embodiment compared with the prior art are as follows: After acquiring the target video, this disclosure embodiment can first extract several target video frames from the target video. Then, for each target video frame, feature extraction is performed on the target video frame to obtain a first feature image; based on the first feature image, several attribute feature information is obtained, wherein the attribute corresponding to each attribute feature information is different; based on each attribute feature information, context-aware feature information corresponding to the attribute feature information is obtained; based on the context-aware feature information corresponding to each of the several attribute feature information, the target attribute state corresponding to the target video frame is determined. Then, based on the target attribute state corresponding to each of the several target video frames, the liveness detection result corresponding to the target video can be determined. In this embodiment, for each target video frame, the target attribute state can be determined based on the context-aware feature information corresponding to each attribute feature information of that target video frame. This allows for flexible learning of the attention image region of the target attribute state of each attribute of the target video frame when determining the target attribute state of each target video frame. At the same time, it integrates rich global context information, making the discrimination of each branch very accurate. This results in more accurate recognition results of the target attribute states of each attribute of the target video frame, thereby ensuring that the liveness detection results of the target video determined based on the target attribute states of several target video frames are more accurate, and thus improving the user experience.

[0045] Next, we will introduce one implementation method of "extracting features from the target video frame to obtain a first feature image" in S202. That is, in this embodiment, the step of extracting features from the target video frame to obtain a first feature image may include the following steps:

[0046] The target video frame is input into the trained base network to obtain the first feature image of the target video frame.

[0047] The underlying network includes any of the following network models: residual network (e.g., ResNet), convolutional network (e.g., EfficientNet, RegNet), and separable convolutional network (e.g., MobileNet).

[0048] The first feature image includes batch dimension, channel number dimension, height dimension and width dimension, that is, the dimensions of the first feature image are (B,C,H,W), where B represents the batch dimension, C represents the channel number dimension, H represents the height dimension and W represents the width dimension.

[0049] Next, we will introduce one implementation method of "obtaining several attribute feature information based on the first feature map" in S202. That is, in this embodiment, the step of obtaining several attribute feature information based on the first feature map may include the following steps:

[0050] The first feature map is input into several attribute feature extraction models to obtain several attribute feature information.

[0051] In this model, the attribute features extracted by each attribute feature extraction model correspond to different attributes. Each attribute feature extraction model includes a depthwise separable convolutional layer (e.g., Separable Conv), a batch normalization layer (e.g., BatchN, BatchNorm), and a functional activation layer (e.g., ReLU activation function). It can be understood that the model architecture of each attribute feature extraction model is the same.

[0052] For example, taking the attribute feature extraction model corresponding to the mouth attribute as an example, the first feature map f1 is input into a depthwise separable convolutional layer (Separable Conv), where the kernel of the depthwise separable convolution is 3x3, the expansion ratio is 2, the padding is 1, and the stride is 1. Then, the output of the depthwise separable convolutional layer is input into a batch normalization layer for batch normalization (BN, BatchNorm), and then ReLU activation is performed using a function activation layer to obtain the attribute feature information f'1 corresponding to the eye attribute, with dimensions (B, 2C, H, W). It should be noted that since depthwise separable convolution is used, the number of channels is appropriately expanded to ensure the number of parameters, thereby ensuring the expressive power of the network.

[0053] Next, we will introduce one implementation method of "obtaining the context-aware feature information corresponding to each attribute feature information according to each attribute feature information" in S202. That is, in this embodiment, the step of obtaining the context-aware feature information corresponding to each attribute feature information according to each attribute feature information may include the following steps:

[0054] Step a: For each attribute feature, input the attribute feature into the trained context-aware model to obtain the first grid context-aware feature map corresponding to the attribute feature.

[0055] The attribute feature information can be input into the segmentation grid layer of the context-aware model to obtain several grid features of the attribute feature information. Let the input feature be attribute feature information f1, with dimensions (B, C, H, W). The following processing is performed in the segmentation grid layer: First, the feature map f1 is segmented into a grid, that is, the (B, C, H, W) feature map is cut into NxN grids along the H and W dimensions. The size of each grid sub-feature map is (B, C, H / N, W / N). Then, a global average pooling operation (GAP) is performed on the H and W dimensions of each grid, so that the dimensions of each grid sub-feature map become (B, C, 1, 1). The specific formula is as follows: P i =GAP(cut(f1)) i = 1, 2... N. It can be understood that the segmented mesh layer in this embodiment is used to compress the spatial resolution of each mesh, that is, to compress all pixels within a mesh into one point.

[0056] Then, each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the inter-channel aware feature vector corresponding to each grid feature. For example, the following operations are performed on the N*N grids output by the grid segmentation layer: In the inter-channel context-aware learning layer, the inter-channel context-aware learning layer first inputs the grid feature P... i The system performs convolutional computation using a 1x1 kernel, C input channels, C / 2 output channels, and a stride of 1. After ReLU activation, it passes through another 1x1 kernel with C / 2 input channels, C output channels, and a stride of 1. Finally, a softmax operation is performed to obtain the channel-aware feature vector P' corresponding to the grid features. iIt is understandable that the inter-channel context-aware learning layer actually captures and learns inter-channel attention for each compressed pixel in the grid, that is, it performs inter-channel context-aware learning. Learning the channel context by grid is mainly to compress the channels to 1 according to the weights later. Compressing the number of channels is to ensure that the subsequent learning of grid spatial context awareness is not affected by the channels, but only focuses on spatial context information, and can reduce the amount of subsequent computation.

[0057] Next, for each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer of the context-aware model to obtain the grid channel attention summation map corresponding to that grid feature. Specifically, in the channel importance weighting layer of the context-aware model, for each grid feature, the grid feature P can be... i Channel-aware feature vector P' corresponding to each grid feature i The dimensions are reshaped from (B, C, 1, 1) to (B, C, 1), and then multiplied to obtain the mesh channel attention summation map P corresponding to this mesh feature. i c Its dimension is (B, 1, 1), and the specific formula is as follows: P i c =reshape(P i ) T *reshape(P' i ), where reshape() is the dimension adjustment function. It can be understood that P i c It's essentially about applying the grid feature P i The spatial region is first compressed to (1, 1) to facilitate context-aware learning of channel attention. After context-aware learning, the channels are weighted and compressed to 1 to facilitate subsequent context-aware learning between grids. Each grid can learn the importance of each channel independently. After learning the importance of each channel, the grid can multiply it with the original feature, meaning that each grid has performed a weighted calculation of channel importance.

[0058] Next, using the context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature; and the grid channel attention feature map is then input into the three convolutional layers of the context-aware model to obtain the first feature, the second feature, and the third feature. Specifically, the context-aware model can concatenate the grid channel attention summation maps corresponding to all grid features to obtain the feature map. Its dimensions are (B, 1, N, N); next, the feature map... Attention is calculated between grid cells to obtain feature maps. Next, the feature map Inputting the data into three convolutional layers with 1x1 kernels yields the first, second, and third features, α, β, and γ, each with dimensions (B, 1, N, N). Then, the N and N dimensions (height and width) of the first, second, and third features are flattened, resulting in dimensions (B, 1, N). 2 The specific formula is as follows:

[0059]

[0060]

[0061]

[0062] It should be noted that this step is based on the original feature map (i.e. attribute feature information) and through channel context-aware learning, the features of each grid are aggregated into a (B, 1, 1, 1) feature map. In other words, the space and channels of each grid are compressed into a single pixel for subsequent grid context-aware learning.

[0063] Next, using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature. Specifically, using the context-aware model, the transpose of the first feature α is multiplied by the second feature β to obtain a dimension (B, N). 2 N 2 This matrix calculates the correlation and relevance between the grid cells. Then, a softmax operation is performed on this matrix. Finally, this matrix is ​​multiplied by the flattened and transposed third feature γ to obtain a matrix with dimensions (B, N). 2 Weighted feature map of 1) The specific formula is as follows:

[0064]

[0065] It should be noted that, by taking into account the similarity between grids, that is, the correlation between each grid, which is to say, context-aware learning was performed, and then the context information was fused, long-distance spatial dependence was obtained, and richer context information was extracted.

[0066] Finally, using the context-aware model, based on the weight feature map and the attribute feature information, a first grid context-aware feature map corresponding to the attribute feature information is obtained. Specifically, using the context-aware model, the weight feature map... Transpose (B, 1, N)2 Then, the dimensions of the weight feature map are reshaped to (B, 1, N, N), and then the weight feature map is padded to dimensions (B, 1, H, W). Each pixel in the weight feature map is padded with a (H / N, W / N) matrix at its original position, meaning the relative positions of the pixels in the weight feature map remain unchanged. Then, the inner product is calculated by multiplying the weight feature map position-by-position with the attribute feature information (i.e., the original input feature map) f1. It is important to emphasize that each channel of f1 is multiplied by this matrix, ultimately yielding the first grid context-aware feature map f. c The specific formula is as follows:

[0067] In summary, since the information of pixels surrounding a given pixel is similar, the feature map is divided into a grid. This grid is used to perceive and fuse global contextual information, focusing attention more on the features represented by each grid. Communication between grids allows each grid to adaptively acquire long-distance spatial and channel dependencies, thus obtaining rich contextual information. This approach, by first compressing spatial pixels within a grid, then compressing channels, and finally acquiring and fusing contextual information between grids, offers advantages such as low computational cost and efficient fusion of global context. This method is suitable for downstream tasks involving local attribute discrimination, such as determining whether a pedestrian has a backpack, whether a face's eyes are open or closed, and whether a face's mouth is open or closed.

[0068] Step b: Input the first grid context-aware feature map into the trained channel augmentation model to obtain the channel augmentation context-aware feature map.

[0069] As an example, the first grid context-aware feature map can be first input into a depthwise separable convolutional layer in the channel augmentation model to obtain a context-aware feature map. Then, the context-aware feature map is input into a batch normalization layer in the channel augmentation model to obtain a normalized context-aware feature map. Next, the normalized context-aware feature map is input into a function activation layer in the channel augmentation model to obtain a channel augmentation context-aware feature map.

[0070] For example, the first grid context-aware feature map f'1 is input into a depthwise separable convolutional layer (Separable Conv) with a kernel of 1x1, 3C channels, 1 padding, and 1 stride for depthwise separable convolution processing to obtain a context-aware feature map with dimensions (B, 3C, H, W). Then, the context-aware feature map is batch normalized (BN, BatchNorm) and then ReLU activated to obtain a channel-amplified context-aware feature map f”1. The specific formula is as follows: f”1 = ReLU(BN(Separable Conv(f1, 1x1))).

[0071] Step c: Input the channel amplified context-aware feature map into the context-aware model to obtain the second grid context-aware feature map corresponding to the attribute feature information.

[0072] The channel-amplified context-aware feature map can be input into the segmentation grid layer of the context-aware model to obtain several grid features of the attribute feature information. Let the input feature be the channel-amplified context-aware feature map f1, with dimensions (B, C, H, W). The following processing is performed in the segmentation grid layer: First, the feature map f1 is segmented into a grid, that is, the (B, C, H, W) feature map is cut into NxN grids along the H and W dimensions. The size of each grid sub-feature map is (B, C, H / N, W / N). Then, a global average pooling operation (GAP) is performed on the H and W dimensions of each grid, so that the dimensions of each grid sub-feature map become (B, C, 1, 1). The specific formula is as follows: P i =GAP(cut(f1)) i = 1, 2... N. It can be understood that the segmented mesh layer in this embodiment is used to compress the spatial resolution of each mesh, that is, to compress all pixels within a mesh into one point.

[0073] Then, each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the inter-channel aware feature vector corresponding to each grid feature. For example, the following operations are performed on the N*N grids output by the grid segmentation layer: In the inter-channel context-aware learning layer, the inter-channel context-aware learning layer first inputs the grid feature P... iThe system performs convolutional computation using a 1x1 kernel, C input channels, C / 2 output channels, and a stride of 1. After ReLU activation, it passes through another 1x1 kernel with C / 2 input channels, C output channels, and a stride of 1. Finally, a softmax operation is performed to obtain the channel-aware feature vector P' corresponding to the grid features. i It is understandable that the inter-channel context-aware learning layer actually captures and learns inter-channel attention for each compressed pixel in the grid, that is, it performs inter-channel context-aware learning. Learning the channel context by grid is mainly to compress the channels to 1 according to the weights later. Compressing the number of channels is to ensure that the subsequent learning of grid spatial context awareness is not affected by the channels, but only focuses on spatial context information, and can reduce the amount of subsequent computation.

[0074] Next, for each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer of the context-aware model to obtain the grid channel attention summation map corresponding to that grid feature. Specifically, in the channel importance weighting layer of the context-aware model, for each grid feature, the grid feature P can be... i Channel-aware feature vector P' corresponding to each grid feature i The dimensions are reshaped from (B, C, 1, 1) to (B, C, 1), and then multiplied to obtain the mesh channel attention summation map P corresponding to this mesh feature. i c Its dimension is (B, 1, 1), and the specific formula is as follows: P i c =reshape(P i ) T *reshape(P' i ), where reshape() is the dimension adjustment function. It can be understood that P i c It's essentially about applying the grid feature P i The spatial region is first compressed to (1, 1) to facilitate context-aware learning of channel attention. After context-aware learning, the channels are weighted and compressed to 1 to facilitate subsequent context-aware learning between grids. Each grid can learn the importance of each channel independently. After learning the importance of each channel, the grid can multiply it with the original feature, meaning that each grid has performed a weighted calculation of channel importance.

[0075] Using the context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature; and the grid channel attention feature map is then input into the three convolutional layers of the context-aware model to obtain the first feature, the second feature, and the third feature. Specifically, the context-aware model can concatenate the grid channel attention summation maps corresponding to all grid features to obtain the feature map. Its dimensions are (B, 1, N, N); next, the feature map... Attention is calculated between grid cells to obtain feature maps. Next, the feature map Inputting the data into three convolutional layers with 1x1 kernels yields the first, second, and third features, α, β, and γ, each with dimensions (B, 1, N, N). Then, the N and N dimensions (height and width) of the first, second, and third features are flattened, resulting in dimensions (B, 1, N). 2 The specific formula is as follows:

[0076]

[0077]

[0078]

[0079] It should be noted that this step is based on the original feature map (i.e. attribute feature information) and through channel context-aware learning, the features of each grid are aggregated into a (B, 1, 1, 1) feature map. In other words, the space and channels of each grid are compressed into a single pixel for subsequent grid context-aware learning.

[0080] Using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature. Specifically, using the context-aware model, the transpose of the first feature α is multiplied by the second feature β to obtain a dimension (B, N). 2 N 2 This matrix calculates the correlation and relevance between the grid cells. Then, a softmax operation is performed on this matrix. Finally, this matrix is ​​multiplied by the flattened and transposed third feature γ to obtain a matrix with dimensions (B, N). 2 Weighted feature map of 1) The specific formula is as follows:

[0081]

[0082] It should be noted that, by taking into account the similarity between grids, that is, the correlation between each grid, which is to say, context-aware learning was performed, and then the context information was fused, long-distance spatial dependence was obtained, and richer context information was extracted.

[0083] Finally, using the context-aware model, based on the weighted feature map and the channel-amplified context-aware feature map, a second grid context-aware feature map corresponding to the attribute feature information is obtained. Specifically, using the context-aware model, the weighted feature map... Transpose (B, 1, N) 2 Then, the dimensions of the weighted feature map are reshaped to (B, 1, N, N), and then the padding of the weighted feature map is made to dimensions (B, 1, H, W). Each pixel in the weighted feature map is padded with a (H / N, W / N) matrix at its original position, meaning the relative positions of the pixels in the weighted feature map remain unchanged. Then, the inner product is calculated by multiplying the inner product position-by-position with the channel-amplified context-aware feature map (i.e., the original input feature map) f1. It is important to emphasize that each channel of f1 is multiplied by this matrix, ultimately resulting in a second grid-based context-aware feature map f1 that incorporates long-range contextual information. c The specific formula is as follows:

[0084] In summary, since the information of pixels surrounding a given pixel is similar, the feature map is divided into a grid. This grid is used to perceive and fuse global contextual information, focusing attention more on the features represented by each grid. Communication between grids allows each grid to adaptively acquire long-distance spatial and channel dependencies, thus obtaining rich contextual information. This approach, by first compressing spatial pixels within a grid, then compressing channels, and finally acquiring and fusing contextual information between grids, offers advantages such as low computational cost and efficient fusion of global context. This method is suitable for downstream tasks involving local attribute discrimination, such as determining whether a pedestrian has a backpack, whether a face's eyes are open or closed, and whether a face's mouth is open or closed.

[0085] Step d: Obtain the context-aware feature information corresponding to the attribute feature information based on the second grid context-aware feature map corresponding to the attribute feature information.

[0086] In this embodiment, the second grid context-aware feature map corresponding to the attribute feature information can be used as the context-aware feature information corresponding to the attribute feature information.

[0087] Next, we will introduce one implementation method of "determining the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information" in S202. That is, in this embodiment, the step of determining the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information may include the following steps:

[0088] Step A: For each attribute feature information, the context-aware feature information corresponding to the attribute feature information is input into the global average pooling layer to obtain the average pooling context-aware feature information corresponding to the attribute feature information; the dimension of the average pooling context-aware feature information corresponding to the attribute feature information is adjusted to obtain the adjusted average pooling context-aware feature information; the adjusted average pooling context-aware feature information is input into the fully connected layer to obtain the target attribute state corresponding to the attribute feature information.

[0089] As an example, we can first perform global average pooling (GAP) on the context-aware feature information f”1 corresponding to each attribute feature information to obtain the average pooled context-aware feature information corresponding to that attribute feature information, whose dimension becomes (B, 3C, 1, 1); then, we reshape the dimension of the average pooled context-aware feature information corresponding to that attribute feature information to (B, 3C*1*1) to obtain the adjusted average pooled context-aware feature information; next, the adjusted average pooled context-aware feature information is passed through a fully connected layer (FC) with a dimension of 3 for classification (here, the dimension is 3 because there are three states of the eyes: open, closed, and occluded, and the mouth also has three states: open, closed, and occluded); then, it is passed through a softmax operation (e.g., input logistic regression layer, softmax layer) to obtain the target attribute state corresponding to that attribute feature information. The specific formula is as follows: It should be noted that the features output by the fully connected layer can be converted into probabilities for each category after softmax calculation. For example, for the eyes: each image outputs three probabilities [0.1, 0.8, 0.1], representing the probabilities of open eyes, closed eyes, and eyes being covered, respectively. The mouth attribute is similar.

[0090] Step B: Take the target attribute state corresponding to each of the several attribute feature information as the target attribute state corresponding to the target video frame.

[0091] Next, the target attribute state corresponding to each of the attribute feature information can be used as the target attribute state corresponding to the target video frame.

[0092] As can be seen, this application extracts specific local features of the face by breaking it down into several attributes (such as eyes and mouth) in the target video frame. Then, for each attribute, different branches are used to determine the target attribute state of each attribute (such as eyes and mouth). Each branch that determines the target attribute state embeds a context-aware model. This context-aware model uses inter-grid communication to obtain rich global contextual information, resulting in more targeted and discriminative features. This helps each branch determine the target attribute state (i.e., the state of the eyes or mouth) of its corresponding attribute. Therefore, the accuracy of liveness detection results can be improved.

[0093] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0094] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0095] Figure 3 This is a schematic diagram of a liveness detection device provided in an embodiment of this disclosure. Figure 3 As shown, the liveness detection device includes:

[0096] The video frame acquisition unit 301 is used to acquire a target video and extract a number of target video frames from the target video.

[0097] The state determination unit 302 is configured to: extract features from each target video frame to obtain a first feature image; obtain several attribute feature information based on the first feature image, wherein each attribute feature information corresponds to a different attribute; obtain context-aware feature information corresponding to each attribute feature information; and determine the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information.

[0098] The result determination unit 303 is used to determine the liveness detection result corresponding to the target video based on the target attribute state corresponding to each of the plurality of target video frames.

[0099] In some embodiments, the state determination unit 302 is configured to:

[0100] The target video frame is input into the trained base network to obtain the first feature image of the target video frame; wherein, the first feature image includes batch dimension, channel number dimension, height dimension and width dimension;

[0101] The underlying network includes any of the following network models: residual network, convolutional network, or separable convolutional network.

[0102] In some embodiments, the state determination unit 302 is configured to:

[0103] The first feature map is input into several attribute feature extraction models to obtain several attribute feature information; wherein, the attribute feature information extracted by each attribute feature extraction model corresponds to different attributes, and each attribute feature extraction model includes a depthwise separable convolutional layer, a batch quantization normalization layer and a function activation layer.

[0104] In some embodiments, the state determination unit 302 is configured to:

[0105] For each attribute feature, the attribute feature is input into a trained context-aware model to obtain a first grid context-aware feature map corresponding to the attribute feature; the first grid context-aware feature map is input into a trained channel augmentation model to obtain a channel augmentation context-aware feature map; the channel augmentation context-aware feature map is input into the context-aware model to obtain a second grid context-aware feature map corresponding to the attribute feature; and the context-aware feature information corresponding to the attribute feature is obtained based on the second grid context-aware feature map.

[0106] In some embodiments, the state determination unit 302 is configured to:

[0107] The attribute feature information is input into the segmentation grid layer in the context-aware model to obtain several grid features of the attribute feature information;

[0108] Each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the channel-aware feature vector corresponding to each grid feature.

[0109] For each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer in the context-aware model to obtain the grid channel attention summation map corresponding to the grid feature.

[0110] Using the aforementioned context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature;

[0111] The grid channel attention feature map is input into the three convolutional layers in the context-aware model to obtain the first feature, the second feature, and the third feature;

[0112] Using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature;

[0113] Using the context-aware model, a first grid context-aware feature map corresponding to the attribute feature information is obtained based on the weight feature map and the attribute feature information.

[0114] In some embodiments, the state determination unit 302 is configured to:

[0115] The first grid context-aware feature map is input into the depthwise separable convolutional layer in the channel augmentation model to obtain a context-aware feature map.

[0116] The context-aware feature map is input into the batch normalization layer of the channel amplification model to obtain the normalized context-aware feature map;

[0117] The normalized context-aware feature map is input into the function activation layer of the channel amplification model to obtain the channel amplification context-aware feature map.

[0118] In some embodiments, the state determination unit 302 is configured to:

[0119] The channel-amplified context-aware feature map is input into the segmented grid layer in the context-aware model to obtain several grid features of the attribute feature information.

[0120] Each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the channel-aware feature vector corresponding to each grid feature.

[0121] For each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer in the context-aware model to obtain the grid channel attention summation map corresponding to the grid feature.

[0122] Using the aforementioned context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature;

[0123] The grid channel attention feature map is input into the three convolutional layers in the context-aware model to obtain the first feature, the second feature, and the third feature;

[0124] Using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature;

[0125] Using the context-aware model, a second grid context-aware feature map corresponding to the attribute feature information is obtained based on the weighted feature map and the channel-amplified context-aware feature map.

[0126] In some embodiments, the state determination unit 302 is configured to:

[0127] The second grid context-aware feature map corresponding to the attribute feature information is used as the context-aware feature information corresponding to the attribute feature information.

[0128] In some embodiments, the state determination unit 302 is configured to:

[0129] For each attribute feature information, the context-aware feature information is input into the global average pooling layer to obtain the average pooling context-aware feature information corresponding to the attribute feature information; the dimension of the average pooling context-aware feature information corresponding to the attribute feature information is adjusted to obtain the adjusted average pooling context-aware feature information; the adjusted average pooling context-aware feature information is input into the fully connected layer to obtain the target attribute state corresponding to the attribute feature information.

[0130] The target attribute state corresponding to each of the aforementioned attribute feature information is taken as the target attribute state corresponding to the target video frame.

[0131] In some embodiments, the result determining unit 303 is configured to:

[0132] If the target attribute states corresponding to each of the plurality of target video frames satisfy the liveness attribute condition, then the liveness recognition result corresponding to the target video is determined to be that the target video includes a live person;

[0133] If the target attribute state corresponding to each of the plurality of target video frames does not meet the liveness attribute condition, then the liveness recognition result corresponding to the target video is determined to be that the target video does not include liveness.

[0134] The liveness attribute condition is that the target attribute states of at least two target video frames that are adjacent in playback order are different.

[0135] In some embodiments, the plurality of attribute feature information includes at least two attribute feature information, wherein the attributes corresponding to the two attribute feature information are eye attribute and mouth attribute, respectively.

[0136] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0137] Figure 4 This is a schematic diagram of computer device 4 provided in an embodiment of this disclosure. Figure 4 As shown, the computer device 4 in this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0138] Exemplarily, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to perform the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in computer device 4.

[0139] Computer device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other similar computer device. Computer device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 4 and does not constitute a limitation on computer device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0140] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0141] The memory 402 can be an internal storage unit of the computer device 4, such as a hard disk or RAM of the computer device 4. The memory 402 can also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 4. Furthermore, the memory 402 can include both internal and external storage units of the computer device 4. The memory 402 is used to store computer programs and other programs and data required by the computer device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this disclosure. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0145] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0149] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A liveness detection method, characterized in that, The method includes: Acquire the target video and extract several target video frames from the target video; For each target video frame, feature extraction is performed on the target video frame to obtain a first feature image; based on the first feature image, several attribute feature information is obtained, wherein the attribute corresponding to each attribute feature information is different; based on each attribute feature information, context-aware feature information corresponding to the attribute feature information is obtained; based on the context-aware feature information corresponding to each of the several attribute feature information, the target attribute state corresponding to the target video frame is determined. Based on the target attribute states corresponding to each of the target video frames, the liveness detection result corresponding to the target video is determined; The step of obtaining context-aware feature information corresponding to each attribute feature information includes: for each attribute feature information, inputting the attribute feature information into a trained context-aware model to obtain a first grid context-aware feature map corresponding to the attribute feature information; inputting the first grid context-aware feature map into a trained channel augmentation model to obtain a channel augmentation context-aware feature map; inputting the channel augmentation context-aware feature map into the context-aware model to obtain a second grid context-aware feature map corresponding to the attribute feature information; and obtaining the context-aware feature information corresponding to the attribute feature information based on the second grid context-aware feature map corresponding to the attribute feature information. Based on the target attribute states corresponding to each of the plurality of target video frames, the liveness detection result corresponding to the target video is determined, including: if the target attribute states corresponding to each of the plurality of target video frames satisfy the liveness attribute condition, then the liveness detection result corresponding to the target video is determined to be that the target video includes a live person; if the target attribute states corresponding to each of the plurality of target video frames do not satisfy the liveness attribute condition, then the liveness detection result corresponding to the target video is determined to be that the target video does not include a live person; wherein, the liveness attribute condition is that the target attribute states of at least two target video frames with adjacent playback order are different, and the target attribute state is the attribute state of one attribute of the face in the target video frame.

2. The method according to claim 1, characterized in that, The step of extracting features from the target video frame to obtain a first feature image includes: The target video frame is input into the trained base network to obtain the first feature image of the target video frame; wherein, the first feature image includes batch dimension, channel number dimension, height dimension and width dimension; The underlying network includes any of the following network models: residual network, convolutional network, or separable convolutional network.

3. The method according to claim 1, characterized in that, The step of obtaining several attribute feature information based on the first feature map includes: The first feature map is input into several attribute feature extraction models to obtain several attribute feature information; wherein, the attribute feature information extracted by each attribute feature extraction model corresponds to different attributes, and each attribute feature extraction model includes a depthwise separable convolutional layer, a batch quantization normalization layer and a function activation layer.

4. The method according to claim 1, characterized in that, The step of inputting the attribute feature information into a trained context-aware model to obtain a first grid context-aware feature map corresponding to the attribute feature information includes: The attribute feature information is input into the segmentation grid layer in the context-aware model to obtain several grid features of the attribute feature information; Each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the channel-aware feature vector corresponding to each grid feature. For each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer in the context-aware model to obtain the grid channel attention summation map corresponding to the grid feature. Using the aforementioned context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature; The grid channel attention feature map is input into the three convolutional layers in the context-aware model to obtain the first feature, the second feature, and the third feature; Using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature; Using the context-aware model, a first grid context-aware feature map corresponding to the attribute feature information is obtained based on the weight feature map and the attribute feature information.

5. The method according to claim 1, characterized in that, The step of inputting the first grid context-aware feature map into the trained channel augmentation model to obtain the channel augmentation context-aware feature map includes: The first grid context-aware feature map is input into the depthwise separable convolutional layer in the channel augmentation model to obtain a context-aware feature map. The context-aware feature map is input into the batch normalization layer of the channel amplification model to obtain the normalized context-aware feature map; The normalized context-aware feature map is input into the function activation layer of the channel amplification model to obtain the channel amplification context-aware feature map.

6. The method according to claim 1, characterized in that, The step of inputting the channel-amplified context-aware feature map into the context-aware model to obtain the second grid context-aware feature map corresponding to the attribute feature information includes: The channel-amplified context-aware feature map is input into the segmented grid layer in the context-aware model to obtain several grid features of the attribute feature information. Each grid feature is input into the inter-channel context-aware learning layer of the context-aware model to obtain the channel-aware feature vector corresponding to each grid feature. For each grid feature, the grid feature and its corresponding channel-aware feature vector are input into the channel importance weighting layer in the context-aware model to obtain the grid channel attention summation map corresponding to the grid feature. Using the aforementioned context-aware model, a grid channel attention feature map is obtained based on the grid channel attention summation map corresponding to each grid feature; The grid channel attention feature map is input into the three convolutional layers in the context-aware model to obtain the first feature, the second feature, and the third feature; Using the context-aware model, a weighted feature map is obtained based on the first feature, the second feature, and the third feature; Using the context-aware model, a second grid context-aware feature map corresponding to the attribute feature information is obtained based on the weighted feature map and the channel-amplified context-aware feature map.

7. The method according to claim 1, characterized in that, The step of obtaining the context-aware feature information corresponding to the attribute feature information based on the second grid context-aware feature map corresponding to the attribute feature information includes: The second grid context-aware feature map corresponding to the attribute feature information is used as the context-aware feature information corresponding to the attribute feature information.

8. The method according to claim 1, characterized in that, The step of determining the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the plurality of attribute feature information includes: For each attribute feature information, the context-aware feature information is input into the global average pooling layer to obtain the average pooling context-aware feature information corresponding to the attribute feature information; the dimension of the average pooling context-aware feature information corresponding to the attribute feature information is adjusted to obtain the adjusted average pooling context-aware feature information; the adjusted average pooling context-aware feature information is input into the fully connected layer to obtain the target attribute state corresponding to the attribute feature information. The target attribute state corresponding to each of the aforementioned attribute feature information is taken as the target attribute state corresponding to the target video frame.

9. The method according to any one of claims 1-8, characterized in that, The plurality of attribute feature information includes at least two attribute feature information, and the attributes corresponding to the two attribute feature information are eye attribute and mouth attribute, respectively.

10. A liveness detection device, characterized in that, The device includes: A video frame acquisition unit is used to acquire a target video and extract several target video frames from the target video. A state determination unit is configured to: extract features from each target video frame to obtain a first feature image; obtain several attribute feature information based on the first feature image, wherein each attribute feature information corresponds to a different attribute; obtain context-aware feature information corresponding to each attribute feature information; and determine the target attribute state corresponding to the target video frame based on the context-aware feature information corresponding to each of the several attribute feature information. The step of obtaining the context-aware feature information corresponding to each attribute feature information includes: for each attribute feature information, inputting the attribute feature information into a trained context-aware model to obtain a first grid context-aware feature image corresponding to the attribute feature information; inputting the first grid context-aware feature image into a trained channel augmentation model to obtain a channel augmentation context-aware feature image; inputting the channel augmentation context-aware feature image into the context-aware model to obtain a second grid context-aware feature image corresponding to the attribute feature information; and obtaining the context-aware feature information corresponding to the attribute feature information based on the second grid context-aware feature image corresponding to the attribute feature information. The result determination unit is configured to determine the liveness detection result corresponding to the target video based on the target attribute states corresponding to each of the plurality of target video frames; determining the liveness detection result corresponding to the target video based on the target attribute states corresponding to each of the plurality of target video frames includes: if the target attribute states corresponding to each of the plurality of target video frames satisfy the liveness attribute condition, then the liveness detection result corresponding to the target video is determined to be that the target video includes a live person; if the target attribute states corresponding to each of the plurality of target video frames do not satisfy the liveness attribute condition, then the liveness detection result corresponding to the target video is determined to be that the target video does not include a live person; wherein, the liveness attribute condition is that the target attribute states of at least two target video frames with adjacent playback order are different.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 9.

Images