A living body detection method, device and equipment

Abstract

The embodiment of the present specification discloses a kind of live detection method, device and equipment.The scheme can include: the semantic feature information and visual feature information of the image to be detected are acquired, the semantic feature information and visual feature information obtained are processed by feature fusion, to obtain the fusion feature vector of the image to be detected.Then the live detection model based on attention mechanism is used to process the fusion feature vector, and then the live detection result for the image to be detected is obtained.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method, apparatus and device for liveness detection. Background Technology

[0002] With the development of computer technology and optical imaging technology, image processing technology is increasingly being used for user identification in various scenarios. For example, in user authentication scenarios, attackers might place a photo, recorded video, or even a wax figure of a legitimate user in front of the camera capturing the user's image to cheat. Therefore, liveness detection is necessary for the captured user images to reduce the risks in the user identification process. Currently, this typically relies on a large number of training samples to train a convolutional neural network model for liveness detection. However, this approach depends on the proper distribution of training samples, which can affect the stability and accuracy of the liveness detection results.

[0003] Therefore, improving the stability and accuracy of the generated liveness detection results has become an urgent technical problem to be solved. Summary of the Invention

[0004] The embodiments of this specification provide a liveness detection method, apparatus, and device that can improve the stability and accuracy of the generated liveness detection results.

[0005] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0006] This specification provides an embodiment of a liveness detection method, comprising:

[0007] Obtain semantic and visual feature information of the image to be detected.

[0008] The visual feature information and the semantic feature information are subjected to feature fusion processing to obtain the fused feature vector of the image to be detected.

[0009] The fused feature vector is processed using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

[0010] This specification provides an embodiment of a liveness detection device, comprising:

[0011] The first acquisition module is used to acquire semantic feature information and visual feature information of the image to be detected.

[0012] The fusion module is used to perform feature fusion processing on the visual feature information and the semantic feature information to obtain the fused feature vector of the image to be detected.

[0013] The processing module is used to process the fused feature vector using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

[0014] This specification provides an embodiment of a liveness detection device, comprising:

[0015] At least one processor, and,

[0016] The memory communicatively connected to the at least one processor, wherein,

[0017] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0018] Obtain semantic and visual feature information of the image to be detected.

[0019] The visual feature information and the semantic feature information are subjected to feature fusion processing to obtain the fused feature vector of the image to be detected.

[0020] The fused feature vector is processed using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

[0021] At least one embodiment provided in this specification can achieve the following beneficial effects:

[0022] By fusing semantic and visual features of the image to be detected, a fused feature vector is obtained. Then, a liveness detection model based on an attention mechanism is used to process this fused feature vector to obtain a liveness detection result for the image. This scheme improves the stability and accuracy of liveness detection results by combining multimodal image information for liveness recognition. Furthermore, by utilizing an attention mechanism, the liveness detection model can focus on regions with strong differences between liveness and attacks, thus exhibiting better generalization and versatility. This reduces the dependence on the accuracy of the model's training data selection, further enhancing the stability and accuracy of liveness detection results. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of a liveness detection method provided in the embodiments of this specification;

[0025] Figure 2 A schematic flowchart of a liveness detection method provided in the embodiments of this specification;

[0026] Figure 3 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the swimlane flow of a liveness detection method;

[0027] Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a liveness detection device;

[0028] Figure 5 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a liveness detection device. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0030] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0031] In existing technologies, liveness detection of acquired user images is mainly achieved based on a trained convolutional neural network model. However, training a convolutional neural network model usually requires a large number of training samples, and the liveness prediction accuracy of the trained convolutional neural network model is highly dependent on the reasonable distribution of the training samples, which will affect the stability and accuracy of the generated liveness detection results.

[0032] To address the shortcomings of existing technologies, this solution provides the following embodiments:

[0033] Figure 1 This is a schematic diagram illustrating an application scenario of a liveness detection method provided in the embodiments of this specification.

[0034] like Figure 1As shown, the liveness detection device 100 may be equipped with a preset language model 101, a preset convolutional neural network model 102, and a liveness detection model 103. The preset language model 101 can extract semantic feature information of the image to be detected, and the preset convolutional neural network model 102 can extract visual feature information of the image to be detected. By performing positional encoding and feature fusion on the semantic feature information and visual feature information of the image to be detected, a fused feature vector of the image to be detected can be obtained.

[0035] Subsequently, the encoder based on the self-attention mechanism in the liveness detection model 103 can encode the fused feature vector of the image to be detected into an encoding vector, and input the encoding vector into the feature processing sub-model for channel attention processing and spatial attention processing. Finally, the processed encoding vector is input into the decoder based on the self-attention mechanism in the liveness detection model 103, and the decoder generates the liveness detection result for the image to be detected based on the encoding vector.

[0036] It is worth noting that, Figure 1 The number of neuron layers, connection results, and number of neurons per layer shown in the examples are for illustrative purposes only and are not intended to limit the specific models.

[0037] Figure 1 The proposed solution, on the one hand, improves the stability and accuracy of liveness detection results by combining multimodal image information for liveness recognition. On the other hand, it utilizes an attention mechanism to enable the liveness detection model to focus on the strongly differentiating regions between liveness and attacks, thereby achieving better generalization and versatility. This reduces the dependence on the accuracy of the model training data selection and further enhances the stability and accuracy of liveness detection results.

[0038] Next, a liveness detection method provided in the embodiments of the specification will be described in detail with reference to the accompanying drawings:

[0039] Figure 2 This is a flowchart illustrating a liveness detection method provided in an embodiment of this specification. From a programming perspective, the entity executing this process can be a liveness detection device, or an application program installed on the liveness detection device. Figure 2 As shown, the process may include the following steps:

[0040] Step 202: Obtain semantic and visual feature information of the image to be detected.

[0041] In the embodiments of this specification, the image to be detected can be a user image collected for a user performing liveness detection. The image to be detected can typically contain the user's facial information. Of course, the image to be detected can also contain the user's body movement information and surrounding environment information, etc.

[0042] In the embodiments of this specification, semantic feature information can be feature information used to reflect image description information for the image to be detected. For example, when the image to be detected is an image captured from a user holding a photo of a face, the image description information for the image to be detected can be used to reflect that "fingers are present at the edge of the photo," or it can be used to reflect that "the background around the face does not match the background of the scene." In the embodiments of this specification, the content of the image description information corresponding to the semantic feature information can be set according to requirements, and no specific limitation is made thereto.

[0043] In the embodiments of this specification, visual feature information can be image feature information extracted from the image to be detected, reflecting the visual information contained in the image. Specifically, the acquired visual feature information can be used to reflect features such as facial features, hairstyle, whether a mask is worn, whether glasses are worn, and whether a hat is worn in the image to be detected. The specific types of features reflected by the visual feature information are not limited here and can be set according to requirements.

[0044] In the embodiments of this specification, the extraction order of semantic feature information and visual feature information of the image to be detected is not limited. Semantic feature information can be extracted first, visual feature information can be extracted first, or semantic feature information and visual feature information can be extracted simultaneously.

[0045] Step 204: Perform feature fusion processing on the visual feature information and the semantic feature information to obtain the fused feature vector of the image to be detected.

[0046] In the embodiments of this specification, feature fusion refers to the combination of features from different layers or branches. It is a common operation in modern network architecture. In image processing feature fusion, for general multi-scale fusion, they can be directly added together or spliced ​​together.

[0047] In the embodiments of this specification, semantic feature information extracted from the image to be detected and visual feature information extracted from the image to be detected are fused according to preset rules. The fused feature vector of the image to be detected obtained after feature fusion processing can simultaneously reflect the descriptive information and the visual image information carried by the image to be detected.

[0048] Step 206: Using a liveness detection model built on an attention mechanism, process the fused feature vector to obtain the liveness detection result for the image to be detected.

[0049] In the embodiments of this specification, the attention mechanism originates from research on human vision. In cognitive science, due to the bottleneck of information processing, humans selectively focus on a portion of all information while ignoring other visible information. This mechanism is commonly referred to as the attention mechanism. The attention mechanism mainly includes two aspects: first, determining which part of the input needs attention; and second, allocating limited information processing resources to the important parts.

[0050] In the embodiments of this specification, the liveness detection model is a detection model used in some identity verification scenarios to determine the true physiological characteristics of an object, in order to verify whether the user is a real, living person. The liveness detection model is mainly used in scenarios such as mobile phone facial recognition unlocking, facial recognition payment, and remote identity verification.

[0051] In the embodiments of this specification, a liveness detection model based on an attention mechanism can be used to perform attention-based classification processing on the fused feature vector to obtain a liveness classification result for the image to be detected. If the liveness probability value corresponding to the liveness classification result is greater than a preset threshold, a liveness detection result can be generated reflecting that the user performing the liveness detection in the image to be detected is a live person. This liveness detection result can indicate that the user performing the liveness detection is performing a compliant operation to conduct the liveness detection; or, if the non-liveness probability value corresponding to the liveness classification result is greater than a preset threshold, a liveness detection result can be generated reflecting that the user performing the liveness detection in the image to be detected is a non-live person. This liveness detection result can indicate that the user performing the liveness detection is performing a fraudulent operation to cheat in the liveness detection.

[0052] In the embodiments of this specification, semantic and visual feature information is extracted from the image to be detected. By inputting more information from the image to be detected into the liveness detection model, the accuracy of the output results of the liveness detection model is improved. Furthermore, the liveness detection model is built on an attention mechanism, which allows the liveness detection model to focus on regions with strong differences between live targets and attackers. Therefore, the liveness detection model has better generalization and versatility, reducing its dependence on training samples, thereby further improving the stability and accuracy of the liveness detection results.

[0053] based on Figure 2 In addition to the method described in the embodiments of this specification, some specific implementation schemes of the method are also provided, which will be described below.

[0054] In the embodiments of this specification, a language model can be used to extract semantic feature information from the image to be detected. Based on this, obtaining the semantic feature information of the image to be detected may specifically include:

[0055] The image to be detected is processed by feature extraction using a preset language model to obtain the semantic feature vector of the image to be detected; wherein the preset language model is a deep learning model used to generate image description information for input images.

[0056] In the embodiments of this specification, the preset language model can be a language model built by the user according to their needs. For example, the preset language model can be implemented using the Generative Pre-Training-3 model (GPT3 model). By using the built preset language model to perform feature extraction processing on the image to be detected, the semantic feature vector of the image to be detected can be obtained. Specifically, according to user needs, the semantic feature vector can be extracted from the convolutional layer of the preset language model, and the resulting semantic feature vector is a two-dimensional feature vector. Alternatively, the semantic feature vector can be extracted from the fully connected layer of the preset language model, and the resulting semantic feature vector is a one-dimensional feature vector. However, if the semantic feature vector extracted from the preset language model is a two-dimensional feature vector, it needs to be converted into a one-dimensional feature vector for positional encoding processing.

[0057] In the embodiments of this specification, the preset language model is a deep learning model that needs to be trained. The user needs to specify the output information of the preset language model, select appropriate training samples and labels associated with the output information according to the specified output information, and train the preset language model to obtain a language model that can output the information required by the user.

[0058] In the embodiments described in this specification, the preset language model is built by the user according to their needs. Training samples are then selected appropriately based on the user's requirements for the output information to train the preset language model. This allows the user to extract the required semantic feature vectors from the image to be detected based on the preset language model, thereby improving user satisfaction.

[0059] The semantic feature vectors obtained by extracting features from the image to be detected using a pre-defined language model can reflect multi-dimensional image description information of the image. Typically, the image description information of these multiple dimensions reflects different content depending on the order in which the dimensions are arranged. For example, when a person wearing a mask undergoes liveness detection, the image description information corresponding to the semantic feature vector extracted by the pre-defined language model could be "mouth wearing a mask." If the positions of the dimensions representing the mouth and the mask are not specified, the text information described by the extracted semantic feature vector might become "mask wearing a mouth." This could cause the liveness detection model to interpret the semantic feature vector as a mask with a mouth pattern instead of a live face wearing a mask, leading to an error in the detection output. Therefore, the positions of the semantic feature vectors extracted from the image to be detected need to be specified.

[0060] Therefore, to ensure the accuracy of semantic feature information, the step of obtaining the semantic feature information of the image to be detected may further include:

[0061] Position encoding is performed on the semantic feature vector of the image to be detected to obtain the position-encoded target semantic feature vector.

[0062] In the embodiments of this specification, the mainstream positional encoding methods are mainly divided into two categories: absolute positional encoding and relative positional encoding. Absolute positional encoding informs the network architecture of the position of each element in the input sequence, similar to giving each element in the input sequence a "position label" indicating its absolute position. Relative positional encoding, on the other hand, informs the network architecture of the distance between pairs of elements.

[0063] In the embodiments of this specification, positional encoding of the semantic feature vector of the image to be detected can be performed by configuring a positional label for each dimension of the semantic feature vector to obtain a positional encoding vector of the semantic feature vector. The dimension of the semantic feature vector and the dimension of the positional encoding vector are the same. The semantic feature vector and the positional encoding vector with the same dimension are fused to realize the positional encoding operation of each semantic feature vector. In practical applications, an ADD operation can be performed on the semantic feature vector and the positional encoding vector. The data dimensions of the final target semantic feature vector, the semantic feature vector, and the positional encoding vector can all be the same. It is worth noting that it is usually necessary to sort the target semantic feature vectors according to the order reflected by the positional encoding vectors corresponding to the target semantic feature vectors, so as to use the sorted target semantic feature vectors to accurately reflect the image description text information of the image to be detected.

[0064] In the embodiments of this specification, positional encoding of the semantic feature vector of the image to be detected can also be performed by encoding the distance between each pair of dimensions in the semantic feature vector to obtain the positional encoding vector of the semantic feature vector. The dimension of the semantic feature vector is the same as the dimension of the positional encoding vector. An ADD operation is then performed on the semantic feature vector and the positional encoding vector with the same dimension, and the data dimension of the final target semantic feature vector remains unchanged. Similarly, it is usually necessary to sort the target semantic feature vectors according to the order reflected by the positional encoding vectors corresponding to the target semantic feature vectors, so as to accurately reflect the image description text information of the image to be detected using the sorted target semantic feature vectors.

[0065] In the embodiments of this specification, by performing position encoding operations on each semantic feature vector of the image to be detected, the semantic feature vectors can be sorted so that the sorted semantic feature vectors can reflect accurate text description information, thereby improving the accuracy of the input information of the liveness detection model.

[0066] In this embodiment of the specification, visual feature information extraction and position encoding of the image to be detected are also required to facilitate subsequent feature fusion processing. Therefore, obtaining the visual feature information of the image to be detected may specifically include:

[0067] The image to be detected is processed by feature extraction using a preset convolutional neural network model to obtain the visual feature vector of the image to be detected; wherein, the preset convolutional neural network model is a model used to perform processing operations including convolution processing on the input image.

[0068] Position encoding is performed on the visual feature vector of the image to be detected to obtain the position-encoded target visual feature vector.

[0069] In the embodiments of this specification, the extraction of visual feature information of the image to be detected can be accomplished using a preset convolutional neural network model. The preset convolutional neural network model can be constructed with network structures such as convolutional layers, pooling layers, fully connected layers, and output layers, depending on user needs. The number of neurons required for each network layer can be one or more, depending on requirements. The constructed preset convolutional neural network model is used to perform feature extraction processing on the image to be detected to obtain a visual feature vector for the image. Depending on user needs, the visual feature vector can be extracted from the convolutional layers of the constructed preset convolutional neural network model, resulting in a two-dimensional feature vector. Alternatively, the visual feature vector can be extracted from the fully connected layers of the constructed preset convolutional neural network model, resulting in a one-dimensional feature vector. However, if the extracted visual feature vector is a two-dimensional feature vector, it needs to be converted into a one-dimensional feature vector for positional encoding processing.

[0070] In the embodiments of this specification, the process of extracting visual feature information from the image to be detected using the preset convolutional neural network model involves processing the pixel values ​​of different parts of the image to be detected, extracting features based on the pixel values ​​of different parts, and obtaining the visual feature vector of the image to be detected. The convolutional layers of the preset convolutional neural network model can extract and process pixel values ​​of different parts of the input image without prior training. Therefore, the preset convolutional neural network model does not require pre-training. Of course, the preset convolutional neural network model can also be pre-trained to extract visual feature vectors, and this is not specifically limited.

[0071] In the embodiments of this specification, the preset convolutional neural network model extracts different feature vectors for different parts of the image to be detected during the visual feature vector extraction process. That is, the extracted visual feature vector is a multi-dimensional vector that can represent pixel information of different parts of the image to be detected. Typically, the order of the different dimensions of information varies, reflecting different overall dimensional information content. For example, when the image to be detected is a face, the eyes, nose, mouth, etc., each have their own fixed positional information. If the positions of the facial features are not specified, the positions of the eyes and mouth may be reversed. Therefore, the visual feature vectors extracted from the image to be detected should undergo positional encoding processing to obtain the target visual feature vector after positional encoding.

[0072] In the embodiments of this specification, positional encoding of the visual feature vector of the image to be detected can be performed by configuring a positional label for each dimension of the visual feature vector to obtain a positional encoding vector of the visual feature vector. The dimension of the visual feature vector is the same as the dimension of the positional encoding vector. The visual feature vector and the positional encoding vector with the same dimension are then subjected to feature fusion processing (e.g., an ADD operation) to obtain the target visual feature vector. At this time, the data dimension of the final target visual feature vector remains unchanged. In practical applications, it is usually necessary to sort the target visual feature vectors according to the order reflected by the positional encoding vectors corresponding to the target visual feature vectors, so as to use the sorted target visual feature vectors to accurately reflect the visual information carried by the image to be detected.

[0073] In the embodiments of this specification, positional encoding of the visual feature vectors of the image to be detected can also be performed by encoding the distance between each pair of dimensions in the visual feature vectors to obtain positional encoding vectors of the visual feature vectors. The dimensions of the visual feature vectors and the positional encoding vectors are the same. An ADD operation is then performed on the visual feature vectors and positional encoding vectors with the same dimensions, and the data dimension of the final target visual feature vector remains unchanged. Similarly, it is usually necessary to sort the target visual feature vectors according to the order reflected by the positional encoding vectors corresponding to the target visual feature vectors, so as to accurately reflect the visual information carried by the image to be detected using the sorted target visual feature vectors.

[0074] In the embodiments of this specification, the position-encoded visual feature vector can accurately reflect the visual information of the image. Therefore, when using the position-encoded visual feature vector for liveness detection, it is beneficial to improve the accuracy of the input information of the liveness detection model, and thus improve the accuracy of the liveness detection results output by the liveness detection model.

[0075] In the embodiments of this specification, it is generally necessary to fuse the semantic and visual feature information extracted from the image to be detected so that the liveness detection model can perform liveness recognition based on the fused feature information. Therefore, step 204: the feature fusion processing of the visual and semantic feature information to obtain the fused feature vector of the image to be detected may specifically include:

[0076] The target semantic feature vector and the target visual feature vector are subjected to feature concatenation processing to obtain the fused feature vector of the image to be detected.

[0077] Wherein, the dimension of the fused feature vector is the sum of the dimension of the target semantic feature vector and the dimension of the target visual feature vector; or,

[0078] The dimensions of the fused feature vector, the target semantic feature vector, and the target visual feature vector are all equal.

[0079] In the embodiments of this specification, the semantic feature vector obtained from the preset language model and the visual feature vector obtained from the preset convolutional neural network model generally contain the same feature dimensions. For example, if the semantic feature vector contains six different dimensions of features, then the visual feature vector should also contain six different dimensions of features to ensure consistency in the positional encoding of the semantic and visual feature vectors. Based on this, when building the preset language model and the preset convolutional neural network model, users can pre-plan the dimensional features of the output data of each layer to extract the required semantic and visual feature vectors from the corresponding network structure according to user needs.

[0080] In the embodiments of this specification, two methods can be used to perform feature concatenation processing on the semantic feature vector and visual feature vector of the image to be detected.

[0081] Method 1: Add semantic feature vectors before or after visual feature vectors so that the dimension of the resulting fused feature vector is the sum of the dimensions of the semantic and visual feature vectors. For example, arranging N-dimensional semantic feature vectors after or before M-dimensional visual feature vectors results in a fused feature vector with a dimension of N+M. In other words, the dimension of the fused feature vector obtained after feature concatenation is the sum of the dimensions of the semantic and visual feature vectors. In practical applications, feature concatenation using Method 1 can be performed using functions such as `dstack` and `stack` to expand the dimensionality of the features and increase the amount of information carried by the fused feature vector.

[0082] Method 2: The semantic feature vectors of the corresponding dimensions can be added to the visual feature vectors to obtain a fused feature vector with the same dimensions as the semantic and visual feature vectors. For example, the semantic feature vector of the i-th dimension can be added to the visual feature vector of the i-th dimension to obtain the fused feature vector of the i-th dimension, where i is greater than or equal to 1 and less than or equal to the number of dimensions of the semantic and visual feature vectors. In practical applications, feature concatenation using Method 2 can be processed using the `add` function, which is convenient and fast.

[0083] In the embodiments of this specification, after concatenating the target semantic feature vector and the target visual feature vector to obtain a fused feature vector, a corresponding liveness detection model needs to be built to detect the fused feature vector according to requirements. Since the self-attention mechanism allows us to focus more on features that are more relevant to the processed features, thereby improving the stability and accuracy of the network model, a liveness detection model can be built based on the attention mechanism.

[0084] Specifically, a liveness detection model based on an attention mechanism can include an encoder based on a self-attention mechanism and a decoder based on a self-attention mechanism. This allows the model to focus more on the distinguishing feature regions between live objects and attackers.

[0085] Correspondingly, step 206: The liveness detection model built based on the attention mechanism is used to process the fused feature vector to obtain the liveness detection result for the image to be detected, which may specifically include:

[0086] The fused feature vector is input into the encoder to obtain the encoded vector output by the encoder.

[0087] The decoder generates a liveness detection result for the image to be detected based on the encoded vector.

[0088] In the embodiments of this specification, the encoder and decoder built based on the self-attention mechanism can constitute an autoencoder. However, since the output layer of the decoder can be used to implement classification tasks, the liveness detection model built in the embodiments of this specification can be a classification model based on the principles of autoencoders and self-attention mechanisms, so as to improve the operational stability and accuracy of the liveness detection model.

[0089] Specifically, the encoder built on the self-attention mechanism, upon receiving the fused feature vector, can encode the fused feature vector to obtain a low-dimensional encoded vector, thereby enhancing the differential feature regions and weakening the similar feature regions. Subsequently, a decoder built on the self-attention mechanism can be used to decode the encoded vector, enabling the liveness detection model to pay more attention to the differential feature regions between live targets and attackers, which helps improve the accuracy and stability of the liveness detection model's output results.

[0090] Typically, when using a liveness detection model built on a self-attention mechanism to detect images, in order to limit the self-attention area to the main areas that people recognize, such as the phone frame, paper background texture, etc., other attention mechanisms need to be considered when building the liveness detection model.

[0091] Based on this, the liveness detection model built on the attention mechanism may further include: a feature processing sub-model built on the channel attention mechanism and the spatial attention mechanism.

[0092] Correspondingly, the step of using the decoder to generate a liveness detection result for the image to be detected based on the encoding vector may specifically include:

[0093] A two-dimensional feature matrix is ​​constructed based on the encoded vector to obtain the feature map to be processed.

[0094] The feature map to be processed is input into the feature processing sub-model to obtain the target feature map output by the feature processing sub-model.

[0095] The target feature map is segmented to obtain multiple target feature sub-maps.

[0096] The feature vectors extracted from each of the target feature sub-maps are concatenated to obtain the target feature vector.

[0097] The target feature vector is input into the decoder to obtain the liveness detection result of the image to be detected.

[0098] In the embodiments described in this specification, the feature processing sub-model built based on channel attention and spatial attention mechanisms enables the liveness detection model to pay more attention to the sensitive regions of the image to be detected. In practical applications, the feature processing sub-model built based on channel attention and spatial attention mechanisms can be implemented using Convolutional Block Attention Module (CBAM) technology, which mainly consists of a large number of convolutional layers.

[0099] Specifically, since the encoder based on the self-attention mechanism encodes the fused feature vector into a one-dimensional vector, while the input vector of the convolutional layer is usually a two-dimensional vector, a two-dimensional feature matrix needs to be constructed from the encoder's output encoding vector to transform the one-dimensional encoding vector into a two-dimensional feature map to be processed. Subsequently, the two-dimensional feature map to be processed is input into a feature processing sub-model based on channel attention and spatial attention mechanisms. This sub-model performs feature fusion processing on the two-dimensional feature map in both channel and spatial dimensions to obtain the target feature map output by the feature processing sub-model. This target feature map is also a two-dimensional feature vector. Therefore, before inputting this two-dimensional target feature vector into the decoder, it needs to be transformed into a one-dimensional target feature vector so that the decoder can perform decoding and liveness classification based on the one-dimensional target feature vector, thereby outputting the liveness detection result of the image to be detected.

[0100] In practical applications, the liveness detection model can be a model built based on Transformer. Therefore, the encoder is the encoder in the Transformer model, and the decoder is the decoder in the Transformer model; the feature processing sub-model can specifically include: a Channel Attention Module (CAM) and a Spatial Attention Module (SAM).

[0101] Specifically, the output of the encoder can be connected to the input of the channel attention module, the output of the channel attention module can be connected to the input of the spatial attention module, and the output of the spatial attention module can be connected to the input of the decoder.

[0102] In the embodiments described in this specification, the Transformer model is a model that uses a self-attention mechanism, completely eliminating its dependence on recurrent and convolutional neural networks. It is also the most fundamental technical support for the BERT model, exhibiting good operational stability and accuracy. Typically, the basic framework of the Transformer model mainly consists of an input part, an encoding part, a decoding part, and an output part.

[0103] In the embodiments described in this specification, since the encoder, channel attention module, spatial attention module, and decoder of the Transformer model need to be connected sequentially, and the encoder outputs a one-dimensional encoded vector, this one-dimensional encoded vector needs to be converted into a two-dimensional vector (i.e., the feature map to be processed mentioned above). This two-dimensional vector is then input into the channel attention module, allowing the channel attention module to constrain the two-dimensional vector based on the channel attention mechanism. The constrained feature vector is then input into the spatial attention module, which performs further constraints based on the spatial attention mechanism. Subsequently, the two-dimensional vector (i.e., the target feature map) obtained after processing by the spatial attention module needs to be flattened back into a one-dimensional vector (i.e., the target feature vector) so that the decoder in the Transformer model can perform decoding and classification processing on this one-dimensional vector. The output of the Transformer model then outputs the detection result of the image to be detected.

[0104] In summary, in the embodiments of this specification, the liveness detection model can be built based on various attention mechanisms, such as the self-attention mechanism of the Transformer model, the channel attention mechanism of the channel attention module, and the spatial attention mechanism of the spatial attention module. This helps to ensure the stability and accuracy of the liveness detection results generated by the liveness detection model and reduces the dependence on the accuracy of the training sample distribution.

[0105] In the embodiments of this specification, the liveness detection model built based on self-attention mechanism, channel attention mechanism, and spatial attention mechanism can be a binary classification model. The output result of the liveness detection model after detecting the image to be detected can be in the form of "the probability that the image to be detected is a live image is 85%, and the probability that it is not a live image is 15%", or it can be in the form of "the image to be detected is a live image, or the image to be detected is not a live image".

[0106] In the embodiments of this specification, a binary classification liveness detection model based on an attention mechanism is built, which enables the liveness detection model to output the detection result of whether the image to be detected is a live image in the first time, thereby improving the working efficiency of the liveness detection model.

[0107] To improve the accuracy of the liveness detection model, after building the liveness detection model based on the attention mechanism, it is necessary to train and learn the liveness detection model. Therefore, before step 206: processing the fused feature vector using the liveness detection model built on the attention mechanism to obtain the liveness detection result for the image to be detected, it may further include:

[0108] Obtain a training sample set, wherein the training samples in the training sample set are fused feature vector samples obtained by performing feature fusion processing on visual feature vector samples and semantic feature vector samples of sample images; the training samples carry classification label data for indicating whether a specified object in the sample image is a live object.

[0109] The initial liveness detection model based on the attention mechanism is trained using the training sample set to obtain the trained liveness detection model based on the attention mechanism.

[0110] In the embodiments of this specification, the training sample set for training the liveness detection model should typically contain multiple training samples. The number of training samples depends on the actual situation and is not limited here, as long as the detection accuracy of the liveness detection model reaches a preset result. Each training sample consists of a fused feature vector sample obtained by fusing visual feature vector samples and semantic feature vector samples of the sample image. The training sample also contains classification label data indicating whether a specified object in the sample image is live. The initial liveness detection model is trained using the training sample set. When the accuracy of the liveness detection model reaches the preset requirement, the final liveness detection model required by the user can be obtained.

[0111] In the embodiments of this specification, sample images may include liveness image samples and attack image samples (i.e., non-liveness image samples). Attack image samples mainly include images captured from a user when that user employs attack methods such as photos, videos, face swapping, masks, occlusion, 3D animation, and screen capture. Liveness image samples, on the other hand, can be images captured from a user who has not used any attack methods. Training samples generated from liveness image samples may carry classification label data indicating that a specified object in the sample image is live, while training samples generated from attack image samples may carry classification label data indicating that a specified object in the sample image is non-live.

[0112] It is worth noting that the visual feature vector samples and semantic feature vector samples of the sample image are usually feature vectors obtained after positional encoding. That is, the generation principle of the visual feature vector sample can be the same as the generation principle of the target visual feature vector, and the generation principle of the semantic feature vector sample can also be the same as the generation principle of the target semantic feature vector. This will not be elaborated further.

[0113] In the embodiments of this specification, the initial liveness detection model based on the attention mechanism is pre-trained, and the liveness detection model obtained after training is used to generate the liveness detection results of the image to be detected, which helps to improve the stability and accuracy of the liveness detection results.

[0114] Figure 3 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the swimlane flow chart for the liveness detection method in [the text]. (See attached diagram.) Figure 3 As shown, the liveness detection process can involve execution entities such as a preset language model, a preset convolutional neural network model, and a liveness detection model.

[0115] In the feature extraction stage, a pre-defined language model is used to extract features from the image to be detected, obtaining a semantic feature vector. This semantic feature vector is then positionally encoded to obtain a positionally encoded target semantic feature vector. Simultaneously, a pre-defined convolutional neural network model is used to extract features from the image to be detected, obtaining a visual feature vector. This visual feature vector is then positionally encoded to obtain a positionally encoded target visual feature vector. Finally, the target semantic feature vector and the target visual feature vector are concatenated and fused to obtain a fused feature vector.

[0116] During the detection phase, the fused feature vector is input into the encoder of the liveness detection model to obtain the encoded vector output by the encoder. The encoded vector is then subjected to channel attention constraints and spatial attention constraints using a feature processing sub-model based on CBAD technology to obtain the target feature vector. The target feature vector is then input into the decoder of the liveness detection model, where the decoder performs decoding and classification processing to obtain the liveness detection result of the image to be detected.

[0117] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a liveness detection device. Figure 4 As shown, the device may include:

[0118] The first acquisition module is used to acquire semantic feature information and visual feature information of the image to be detected.

[0119] The fusion module is used to perform feature fusion processing on the visual feature information and the semantic feature information to obtain the fused feature vector of the image to be detected.

[0120] The processing module is used to process the fused feature vector using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

[0121] based on Figure 4 The embodiments of this specification also provide some specific implementations of the device, which will be described below.

[0122] Optionally, the first acquisition module may specifically include:

[0123] The first extraction unit is used to perform feature extraction processing on the image to be detected using a preset language model to obtain the semantic feature vector of the image to be detected; wherein, the preset language model is a deep learning model used to generate image description information for input images.

[0124] Optionally, the first acquisition module may further include:

[0125] The first encoding unit is used to perform position encoding processing on the semantic feature vector of the image to be detected, so as to obtain the position-encoded target semantic feature vector.

[0126] Optionally, the first acquisition module may specifically include:

[0127] The second extraction unit is used to perform feature extraction processing on the image to be detected using a preset convolutional neural network model to obtain the visual feature vector of the image to be detected; wherein, the preset convolutional neural network model is a model used to perform processing operations including convolution processing on the input image.

[0128] The second encoding unit is used to perform position encoding processing on the visual feature vector of the image to be detected, so as to obtain the position-encoded target visual feature vector.

[0129] Optionally, the fusion module may specifically include:

[0130] The splicing unit is used to perform feature splicing processing on the target semantic feature vector and the target visual feature vector to obtain the fused feature vector of the image to be detected.

[0131] Wherein, the dimension of the fused feature vector is the sum of the dimension of the target semantic feature vector and the dimension of the target visual feature vector; or,

[0132] The dimensions of the fused feature vector, the target semantic feature vector, and the target visual feature vector are all equal.

[0133] Optionally, the liveness detection model based on the attention mechanism may include: an encoder based on the self-attention mechanism, and a decoder based on the self-attention mechanism.

[0134] Optionally, the processing module may specifically include:

[0135] The input unit is used to input the fused feature vector into the encoder to obtain the encoded vector output by the encoder.

[0136] The generation unit is used to generate a liveness detection result for the image to be detected based on the encoding vector using the decoder.

[0137] Optionally, the liveness detection model based on the attention mechanism may further include a feature processing sub-model based on channel attention mechanism and spatial attention mechanism.

[0138] Optionally, the generation unit may specifically include:

[0139] A sub-unit is constructed to build a two-dimensional feature matrix based on the encoding vector, thereby obtaining the feature map to be processed.

[0140] The first input sub-unit is used to input the feature map to be processed into the feature processing sub-model to obtain the target feature map output by the feature processing sub-model.

[0141] The molecular segmentation unit is used to segment the target feature map to obtain multiple target feature sub-maps.

[0142] The splicing subunit is used to splice the feature vectors extracted from each of the target feature sub-maps to obtain the target feature vector.

[0143] The second input subunit is used to input the target feature vector into the decoder to obtain the liveness detection result of the image to be detected output by the decoder.

[0144] Optionally, the encoder is the encoder in the Transformer model, and the decoder is the decoder in the Transformer model; the feature processing sub-model includes: a channel attention module and a spatial attention module.

[0145] The output of the encoder is connected to the input of the channel attention module, the output of the channel attention module is connected to the input of the spatial attention module, and the output of the spatial attention module is connected to the input of the decoder.

[0146] Optionally, the liveness detection model built based on the attention mechanism is a classification model; the liveness detection result output by the decoder for the image to be detected is a classification result; the classification result is used to indicate whether the target object in the image to be detected is a live object.

[0147] Optional, Figure 4 The device may also include:

[0148] The second acquisition module is used to acquire a training sample set. The training samples in the training sample set are fused feature vector samples obtained by fusing visual feature vector samples and semantic feature vector samples of the sample images. The training samples carry classification label data indicating whether a specified object in the sample image is a live object.

[0149] The training module is used to train the initial liveness detection model based on the attention mechanism using the training sample set, so as to obtain the trained liveness detection model based on the attention mechanism.

[0150] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0151] Figure 5 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a liveness detection device. (See diagram below.) Figure 5 As shown, device 500 may include:

[0152] At least one processor 510; and,

[0153] Memory 530 communicatively connected to the at least one processor; wherein,

[0154] The memory 530 stores instructions 520 that can be executed by the at least one processor 510, the instructions being executed by the at least one processor 510 to enable the at least one processor 510 to:

[0155] Obtain semantic and visual feature information of the image to be detected.

[0156] The visual feature information and the semantic feature information are subjected to feature fusion processing to obtain the fused feature vector of the image to be detected.

[0157] The fused feature vector is processed using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

[0158] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 5 As the device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0159] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0160] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0161] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0162] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0163] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0165] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0166] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0167] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0169] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital character versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0171] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0173] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting liveness, comprising: Obtain semantic and visual feature information of the image to be detected; The visual feature information and the semantic feature information are fused to obtain the fused feature vector of the image to be detected; Semantic feature information is used to reflect the feature information of the image description information obtained by using a preset language model to extract features from the image to be detected; The fused feature vector is processed using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

2. The method as described in claim 1, wherein obtaining the semantic feature information of the image to be detected specifically includes: The image to be detected is processed by feature extraction using a preset language model to obtain the semantic feature vector of the image to be detected; wherein the preset language model is a deep learning model used to generate image description information for input images.

3. The method as described in claim 2, wherein obtaining the semantic feature information of the image to be detected further includes: Position encoding is performed on the semantic feature vector of the image to be detected to obtain the position-encoded target semantic feature vector.

4. The method as described in claim 3, wherein obtaining the visual feature information of the image to be detected specifically includes: The image to be detected is processed by feature extraction using a preset convolutional neural network model to obtain the visual feature vector of the image to be detected; wherein, the preset convolutional neural network model is a model used to perform processing operations including convolution processing on the input image; Position encoding is performed on the visual feature vector of the image to be detected to obtain the position-encoded target visual feature vector.

5. The method as described in claim 4, wherein the step of performing feature fusion processing on the visual feature information and the semantic feature information to obtain the fused feature vector of the image to be detected specifically includes: The target semantic feature vector and the target visual feature vector are subjected to feature concatenation processing to obtain the fused feature vector of the image to be detected; Wherein, the dimension of the fused feature vector is the sum of the dimension of the target semantic feature vector and the dimension of the target visual feature vector; or, The dimensions of the fused feature vector, the target semantic feature vector, and the target visual feature vector are all equal.

6. The method according to any one of claims 1-5, wherein the liveness detection model based on the attention mechanism comprises: An encoder based on a self-attention mechanism, and a decoder based on a self-attention mechanism; The liveness detection model, built using an attention mechanism, processes the fused feature vector to obtain liveness detection results for the image to be detected, specifically including: The fused feature vector is input into the encoder to obtain the encoded vector output by the encoder; The decoder generates a liveness detection result for the image to be detected based on the encoded vector.

7. The method as described in claim 6, wherein the liveness detection model based on the attention mechanism further includes: Feature processing sub-model based on channel attention mechanism and spatial attention mechanism; The step of generating a liveness detection result for the image to be detected based on the encoded vector using the decoder specifically includes: A two-dimensional feature matrix is ​​constructed based on the encoded vector to obtain the feature map to be processed; The feature map to be processed is input into the feature processing sub-model to obtain the target feature map output by the feature processing sub-model. The target feature map is segmented to obtain multiple target feature sub-maps; The feature vectors extracted from each of the target feature sub-maps are concatenated to obtain the target feature vector; The target feature vector is input into the decoder to obtain the liveness detection result of the image to be detected.

8. The method of claim 7, wherein the encoder is an encoder in the Transformer model, and the decoder is a decoder in the Transformer model; The feature processing sub-model includes: Channel attention module and spatial attention module; The output of the encoder is connected to the input of the channel attention module, the output of the channel attention module is connected to the input of the spatial attention module, and the output of the spatial attention module is connected to the input of the decoder.

9. The method as described in claim 8, wherein the liveness detection model built based on the attention mechanism is a classification model; the liveness detection result output by the decoder for the image to be detected is a classification result; and the classification result is used to indicate whether the target object in the image to be detected is a live object.

10. The method of claim 9, further comprising, before processing the fused feature vector using a liveness detection model based on an attention mechanism to obtain a liveness detection result for the image to be detected, the method further comprising: Obtain a training sample set, wherein the training samples in the training sample set are fused feature vector samples obtained by performing feature fusion processing on visual feature vector samples and semantic feature vector samples of sample images; The training samples carry classification label data to indicate whether a specified object in the sample image is a live organism; The initial liveness detection model based on the attention mechanism is trained using the training sample set to obtain the trained liveness detection model based on the attention mechanism.

11. A liveness detection device, comprising: The first acquisition module is used to acquire semantic feature information and visual feature information of the image to be detected; The fusion module is used to perform feature fusion processing on the visual feature information and the semantic feature information to obtain the fused feature vector of the image to be detected; Semantic feature information is used to reflect the feature information of the image description information obtained by using a preset language model to extract features from the image to be detected; The processing module is used to process the fused feature vector using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

12. The apparatus of claim 11, wherein the first acquisition module specifically includes: The first extraction unit is used to perform feature extraction processing on the image to be detected using a preset language model to obtain the semantic feature vector of the image to be detected; wherein, the preset language model is a deep learning model used to generate image description information for input images.

13. The apparatus of claim 12, wherein the first acquisition module further comprises: The first encoding unit is used to perform position encoding processing on the semantic feature vector of the image to be detected, so as to obtain the position-encoded target semantic feature vector.

14. The apparatus of claim 13, wherein the first acquisition module further comprises: The second extraction unit is used to perform feature extraction processing on the image to be detected using a preset convolutional neural network model to obtain the visual feature vector of the image to be detected; wherein, the preset convolutional neural network model is a model used to perform processing operations including convolution processing on the input image; The second encoding unit is used to perform position encoding processing on the visual feature vector of the image to be detected, so as to obtain the position-encoded target visual feature vector.

15. The apparatus of claim 14, wherein the fusion module specifically comprises: The splicing unit is used to perform feature splicing processing on the target semantic feature vector and the target visual feature vector to obtain the fused feature vector of the image to be detected; Wherein, the dimension of the fused feature vector is the sum of the dimension of the target semantic feature vector and the dimension of the target visual feature vector; or, The dimensions of the fused feature vector, the target semantic feature vector, and the target visual feature vector are all equal.

16. The apparatus of any one of claims 11-15, wherein the liveness detection model based on the attention mechanism comprises: An encoder based on a self-attention mechanism, and a decoder based on a self-attention mechanism; The processing module specifically includes: An input unit is used to input the fused feature vector into the encoder to obtain the encoded vector output by the encoder; The generation unit is used to generate a liveness detection result for the image to be detected based on the encoding vector using the decoder.

17. The apparatus of claim 16, wherein the liveness detection model based on the attention mechanism further comprises: Feature processing sub-model based on channel attention mechanism and spatial attention mechanism; The generation unit specifically includes: A sub-unit is constructed to build a two-dimensional feature matrix based on the encoding vector, thereby obtaining the feature map to be processed. The first input sub-unit is used to input the feature map to be processed into the feature processing sub-model to obtain the target feature map output by the feature processing sub-model. A molecular segmentation unit is used to segment the target feature map to obtain multiple target feature sub-maps; The splicing subunit is used to splice the feature vectors extracted from each of the target feature sub-maps to obtain the target feature vector; The second input subunit is used to input the target feature vector into the decoder to obtain the liveness detection result of the image to be detected output by the decoder.

18. The apparatus of claim 17, wherein the encoder is an encoder in a Transformer model, and the decoder is a decoder in the Transformer model; The feature processing sub-model includes: Channel attention module and spatial attention module; The output of the encoder is connected to the input of the channel attention module, the output of the channel attention module is connected to the input of the spatial attention module, and the output of the spatial attention module is connected to the input of the decoder.

19. The apparatus of claim 18, wherein the liveness detection model based on the attention mechanism is a classification model; the liveness detection result output by the decoder for the image to be detected is a classification result; and the classification result is used to indicate whether the target object in the image to be detected is a live object.

20. The apparatus of claim 19, further comprising: The second acquisition module is used to acquire a training sample set, wherein the training samples in the training sample set are fused feature vector samples obtained by performing feature fusion processing on visual feature vector samples and semantic feature vector samples of sample images. The training samples carry classification label data to indicate whether a specified object in the sample image is a live organism; The training module is used to train the initial liveness detection model based on the attention mechanism using the training sample set, so as to obtain the trained liveness detection model based on the attention mechanism.

21. A liveness detection device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Obtain semantic and visual feature information of the image to be detected; The visual feature information and the semantic feature information are subjected to feature fusion processing to obtain the fused feature vector of the image to be detected; the semantic feature information is used to reflect the feature information of the image description information obtained by feature extraction of the image to be detected using a preset language model; The fused feature vector is processed using a liveness detection model based on an attention mechanism to obtain liveness detection results for the image to be detected.

Images