Liveness detection methods, devices, computer equipment, and storage media

Abstract

This application relates to the field of artificial intelligence, specifically to a liveness detection method, apparatus, computer device, and storage medium. The method includes: acquiring a target image corresponding to an object to be detected; determining input data for at least one processing unit; the input data for each processing unit includes the target image and at least one of the following: fusion features output by a pre-processing unit preceding the corresponding processing unit; extracting features from the input data of each processing unit to obtain global and local dimensional features, and performing feature fusion processing on the global and local dimensional features to obtain corresponding fusion features; and determining the liveness detection result corresponding to the target object based on the fusion features output by each processing unit. This method can be applied to intelligent transportation scenarios and can improve the accuracy of liveness detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for detecting liveness. Background Technology

[0002] With the development of technology, facial recognition systems are being used in increasingly widespread scenarios. Currently, there are many instances where individuals use photos or videos containing other people's faces, or wear masks to impersonate them, attempting to pass facial recognition and threatening the rights and safety of others. Therefore, before performing facial recognition, it is necessary to use liveness detection methods to determine whether the object to be detected in the image data is a real human face, and not a photo, video, or mask.

[0003] Traditional techniques assume that the difference between the image to be detected and the real image may conform to a certain distribution of noise. A neural network is then used to estimate the feature map of the noise, and liveness detection is performed on the image to be detected based on this feature map. However, when the image to be detected does not satisfy the premise that "the difference between the image to be detected and the real image may conform to a certain distribution of noise," detection errors will occur. Summary of the Invention

[0004] Therefore, it is necessary to provide a liveness detection method, apparatus, computer equipment, and storage medium that can improve the accuracy of liveness detection in response to the above-mentioned technical problems.

[0005] A method for detecting liveness, the method comprising:

[0006] Acquire the image to be detected corresponding to the object to be detected;

[0007] Determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected, and at least one of the fusion features output by the preprocessing unit preceding the corresponding processing unit;

[0008] Each of the aforementioned processing units extracts features from its respective input data to obtain global and local dimension features, and then performs feature fusion processing on the global and local dimension features to obtain corresponding fused features.

[0009] Based on the fusion features output by each of the processing units, a liveness detection result corresponding to the object to be detected is determined.

[0010] A liveness detection method apparatus, the apparatus comprising:

[0011] The data acquisition module is used to acquire the image to be detected corresponding to the object to be detected; determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected, and at least one of the fusion features output by the preprocessing unit before the corresponding processing unit;

[0012] The feature processing module is used to extract features from the input data of each of the processing units to obtain global dimension features and local dimension features, and to perform feature fusion processing on the global dimension features and local dimension features to obtain corresponding fused features.

[0013] The result output module is used to determine the liveness detection result corresponding to the object to be detected based on the fusion features output by each of the processing units.

[0014] In one embodiment, the data acquisition module includes an image acquisition module, used to acquire an original image and identify the facial region of the object to be detected in the original image; adjust the size of the facial region to obtain a target region; and crop the target region to obtain an image to be detected that includes the object to be detected.

[0015] In one embodiment, the data acquisition module is used to take the image to be detected as the input data of the first processing unit in at least one processing unit; and for each processing unit other than the first processing unit in at least one processing unit, the fusion features output by the preprocessing unit before the corresponding processing unit are taken as the input data of the corresponding processing unit.

[0016] In one embodiment, the feature processing module is used to extract features from the image to be detected through the first processing unit among multiple processing units to obtain a first global dimension feature and a first local dimension feature, and to perform feature fusion processing on the first global dimension feature and the first local dimension feature to obtain the fused feature output by the first processing unit; for each processing unit other than the first processing unit, feature extraction is performed on the fused feature output by the preprocessing unit before the corresponding processing unit to obtain a second global dimension feature and a second local dimension feature, and to perform feature fusion processing on the second global dimension feature and the second local dimension feature to obtain the fused feature output by the corresponding processing unit.

[0017] In one embodiment, the feature processing module includes a feature extraction module, configured to extract global initial features and local initial features from the corresponding input data for each of at least one processing unit; perform local pooling processing on the local initial features through the current processing unit to obtain local statistical features in the local processed features, and obtain corresponding local difference features based on the feature differences between the local initial features and the local statistical features; and perform feature supplementation processing on the global initial features and local difference features through the current processing unit to obtain corresponding global dimension features and local dimension features.

[0018] In one embodiment, the feature extraction module is further configured to extract global facial information from the local difference features and supplement the global facial information to the global initial features to obtain global dimensional features; and to extract local part information from the global initial features and supplement the local part information to the local difference features to obtain local dimensional features.

[0019] In one embodiment, the feature processing module further includes a feature fusion module, used to determine the feature weights corresponding to the global dimension features and the local dimension features respectively; and to perform feature fusion processing on the global dimension features and the local dimension features based on the feature weights corresponding to the global dimension features and the local dimension features respectively, to obtain the corresponding fused features.

[0020] In one embodiment, the feature fusion module is further configured to perform convolution processing on the global dimension features and the local dimension features respectively to obtain global dimension convolution results and local dimension convolution results, and to perform aggregation processing on the global dimension convolution results and the local dimension convolution results to obtain corresponding aggregated features; and to determine the feature weights corresponding to the global dimension features and the local dimension features respectively based on the aggregated features.

[0021] In one embodiment, the feature fusion module is further configured to determine a first matrix vector and a first normalization function corresponding to the global dimension feature, and to determine a second matrix vector and a second normalization function corresponding to the local dimension feature; perform global average pooling on the aggregated features to obtain a global pooling result; perform a first processing on the global pooling result using the first matrix vector and the first normalization function to obtain feature weights corresponding to the global dimension feature; and perform a second processing on the global pooling result using the second matrix vector and the second normalization function to obtain feature weights corresponding to the local dimension feature.

[0022] In one embodiment, the liveness detection device further includes a training module for acquiring a liveness detection model to be trained, sample images, and sample labels corresponding to the sample images; the liveness detection model includes at least one processing unit to be trained; the input samples corresponding to the at least one processing unit to be trained are determined; the input sample of the current processing unit to be trained in the at least one processing unit to be trained includes the sample image and at least one of the fusion prediction features output by the previous processing unit to be trained; each processing unit extracts features from its respective input samples to obtain global prediction features and local prediction features, and performs feature fusion processing on the global prediction features and local prediction features to obtain corresponding fusion prediction features; based on the fusion prediction features output by each processing unit to be trained, a prediction result for liveness prediction of the sample image is determined; based on the prediction result and the sample labels, the liveness detection model is trained until the training stops when the training stopping condition is met, resulting in a trained liveness detection model.

[0023] In one embodiment, the prediction result includes a predicted depth map and a liveness prediction label; the training module is further configured to perform depth map construction processing on the fused prediction features output by each of the training processing units using the depth map construction structure in the liveness detection model, to obtain a corresponding predicted depth map; and to perform liveness classification processing on the fused prediction features output by each of the training processing units using the liveness classification structure in the liveness detection model, to obtain a corresponding liveness prediction label.

[0024] In one embodiment, the sample label includes a target depth map and a liveness classification label; the training module is further configured to determine a first difference between the predicted depth map and the target depth map, and to determine a second difference between the liveness prediction label and the liveness classification label; the liveness detection model is trained using the first difference and the second difference until the training stops when the training stopping condition is met.

[0025] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the liveness detection methods provided in the embodiments of this application.

[0026] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the liveness detection methods provided in the embodiments of this application.

[0027] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the liveness detection methods provided in the embodiments of this application.

[0028] The aforementioned liveness detection method, apparatus, computer equipment, storage medium, and computer program, by acquiring the image to be detected and the fusion features output by the preprocessing unit, can determine the input data corresponding to each processing unit based on the image to be detected and the fusion features output by the preprocessing unit. Thus, each processing unit can perform feature extraction and feature fusion processing on its corresponding input data to obtain corresponding fusion features. By obtaining the fusion features output by each processing unit, the fusion features can be combined to obtain the liveness detection result corresponding to the object to be detected. Since it integrates global and local dimensional features to perform liveness detection on the image to be detected, compared to determining the liveness detection result through noise distribution, this application can still obtain accurate liveness detection results even when the image to be detected does not meet the premise that "the difference between the image to be detected and the real image may conform to a certain distribution of noise."

[0029] Furthermore, since no overly strong prior assumptions are imposed on global and local dimensional features, the impact on generalization is minimal, allowing it to generalize well to different data distributions. This enables the liveness detection model to identify forged images using ordinary RGB images as carriers. Simultaneously, this application requires no user facial movements or specific hardware; liveness detection can be performed based solely on a single image, significantly reducing the cost of liveness detection. Attached Figure Description

[0030] Figure 1 This is a diagram illustrating the application environment of a liveness detection method in one embodiment;

[0031] Figure 2 This is a flowchart illustrating a liveness detection method in one embodiment;

[0032] Figure 3 This is a schematic diagram illustrating the acquisition of the image to be detected in one embodiment;

[0033] Figure 4 This is a schematic diagram of the overall framework of a liveness detection model in one embodiment;

[0034] Figure 5 This is a schematic diagram of the feature extraction structure in one embodiment;

[0035] Figure 6This is a schematic diagram of the feature fusion structure in one embodiment;

[0036] Figure 7 This is a schematic diagram illustrating the training of a liveness detection model in one embodiment;

[0037] Figure 8 This is a flowchart illustrating a liveness detection method in a specific embodiment;

[0038] Figure 9 This is a flowchart illustrating the liveness detection method in another specific embodiment;

[0039] Figure 10 This is a structural block diagram of a liveness detection method apparatus in one embodiment;

[0040] Figure 11 This is a structural block diagram of the liveness detection method apparatus in another embodiment;

[0041] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] Figure 1 This is a diagram illustrating the application environment of a liveness detection method in one embodiment. (Refer to...) Figure 1 The liveness detection method is applied to a liveness detection system 100. The liveness detection system 100 includes a terminal 102 and a server 104. Both the terminal 102 and the server 104 can be used independently to execute the liveness detection method provided in this embodiment. The terminal 102 and server 104 can also be used collaboratively to execute the liveness detection method provided in this embodiment. Taking the collaborative execution of the terminal 102 and server 104 as an example, the terminal 102 can acquire an image to be detected and send the image to be detected to the server 104, so that the server 104 can call a liveness detection model, perform liveness detection on the image to be detected using the liveness detection model, obtain the liveness detection result, and return the liveness detection result to the terminal 102 for display.

[0044] The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these. The terminal 102 and the server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0045] This application also relates to the field of artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Immediately, this application specifically relates to computer vision (CV) technology within the field of AI. Computer vision is a science that studies how to enable machines to "see," and through computer vision technology, liveness detection can be performed on images.

[0046] It is worth noting that the embodiments of this application can also be applied to various scenarios such as cloud technology, artificial intelligence, and smart transportation. For example, when the terminal 102 is a vehicle terminal, the embodiments of this application can be applied to smart transportation scenarios, and when the server 104 is a cloud server that provides cloud computing, the embodiments of this application can be applied to cloud technology scenarios.

[0047] In one embodiment, such as Figure 2 As shown, a liveness detection method is provided, which can be applied to... Figure 1 The following explanation uses a computer device as an example, which specifically includes the aforementioned terminal and server.

[0048] The liveness detection method includes the following steps:

[0049] Step S202: Obtain the image to be detected corresponding to the object to be detected.

[0050] In this context, "living object" refers to an object possessing life, such as living animals, plants, human bodies, and their tissues. Specifically, a living object can be a human body, an eyeball, or an iris. In this embodiment, the object to be detected as a living object can be an animal, plant, human body, or its tissues. Liveness detection refers to a method for determining the true physiological characteristics of an object in identity verification and recognition scenarios, i.e., a method for determining whether the detected object is a living object. For example, when performing liveness detection on a human body, the face is typically detected. By comparing the detected facial features with the physiological characteristics of a real face, it is determined whether the user is a real living object. It is readily understood that in this embodiment, the object to be detected may or may not be a living object.

[0051] Specifically, when liveness detection is required, computer devices can acquire an image to be detected. For example, when a user needs to log in to a target application, to improve the security of the target application, the application can acquire an image to be detected for login verification. The liveness detection result of the image can include whether the object to be detected in the image is a live person or not. When the liveness detection result indicates that the object to be detected in the image is a live person, the image can be considered to have been acquired for a real live person; when the liveness detection result indicates that the object to be detected in the image is not a live person, the image can be considered to have been acquired for a photograph or video of another person's face, or an image obtained by overlaying another person's facial mask.

[0052] In one embodiment, a user can take a picture of their own face using the terminal to obtain an image to be detected; a user can also take a picture of another person's photo using the terminal to obtain an image to be detected; or a user can take a picture of another person's face mask using the terminal to obtain an image to be detected. This embodiment is not limited thereto.

[0053] In one embodiment, obtaining a detection image corresponding to the object to be detected includes: obtaining an original image and identifying the facial region of the object to be detected in the original image; adjusting the size of the facial region to obtain a target region; and cropping the target region to obtain a detection image including the object to be detected.

[0054] Specifically, the computer device can acquire the original image and identify the facial region of the object to be detected in the original image using a preset face recognition strategy, thereby obtaining a face recognition bounding box used to select the facial region. The face recognition strategy can be freely set according to needs; for example, it can be a face recognition algorithm built into the computer device, or it can be face recognition performed through a trained machine learning model. This embodiment of the application does not limit this. Furthermore, in order to obtain an image of the object to be detected that includes a complete face, the computer device can also enlarge the size of the face recognition bounding box to adjust the size of the facial region, obtain the corresponding target region, and crop the target region to obtain an image of the object to be detected that includes its face.

[0055] For example, refer to Figure 3 , Figure 3 A schematic diagram of image acquisition in one embodiment is shown. When a face recognition box 302 for selecting the face of the object to be detected is obtained, the computer device can enlarge the face recognition box 302 with its center point as the center to obtain an enlarged target recognition box 304, and use the image area selected by the target recognition box 304 as the image area corresponding to the image to be detected.

[0056] In this embodiment, by adjusting the size of the facial region, it is possible to make the image to be detected after the size adjustment include the complete facial region, so that the detection results of subsequent liveness detection based on the complete facial region can be more accurate.

[0057] Step S204: Determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected and at least one of the fusion features output by the preprocessing unit prior to the corresponding processing unit.

[0058] Specifically, a liveness detection model can be used to detect objects in the image to be detected. The liveness detection model can be a pre-trained, general-purpose machine learning model with liveness detection capabilities. This liveness detection model can specifically include at least one processing unit, each processing its own input data to obtain its own output fused features. The input data of the current processing unit includes at least one of the image to be detected and the fused features output by the pre-processing unit.

[0059] In one embodiment, determining the input data corresponding to each of the at least one processing unit includes: for the first processing unit among the at least one processing unit, using the image to be detected as the input data of the first processing unit; for each processing unit other than the first processing unit among the at least one processing unit, using the fusion features output by the preprocessing unit before the corresponding processing unit as the input data of the corresponding processing unit.

[0060] Specifically, the processing units in the liveness detection model can be connected end-to-end to form a processing unit sequence. The computer device uses the image to be detected as the input data for the first processing unit in the processing unit sequence, that is, the image to be detected is used as the input data for the first processing unit in at least one processing unit sequence. For each processing unit in the processing unit sequence other than the first processing unit, the corresponding fusion feature output by the preprocessing unit is used as the input data for the current processing unit. That is, the corresponding fusion feature output by the preprocessing unit is used as the input data for the current processing unit other than the first processing unit. Here, the preprocessing unit refers to the processing unit adjacent to and preceding the current processing unit.

[0061] For example, refer to Figure 4 The liveness detection model may include three processing units, namely processing unit 1, processing unit 2 and processing unit 3. The input data of processing unit 1 is the image to be detected, the input data of processing unit 2 is the fusion feature output by processing unit 1, and the input data of processing unit 3 is the fusion feature output by processing unit 2. Figure 4 A schematic diagram of the overall framework of a liveness detection model in one embodiment is shown.

[0062] In this embodiment, by determining the input data corresponding to each processing unit, each processing unit can process its own input data to obtain the output data of each processing unit.

[0063] Step S206: Each processing unit extracts features from its own input data to obtain global and local features, and then performs feature fusion processing on the global and local features to obtain the corresponding fused features.

[0064] The processing unit includes a feature extraction structure and a feature fusion structure. The feature extraction structure refers to the model structure used to extract global and local features from the input data; the feature fusion structure refers to the model structure used to fuse global and local features. Both feature extraction and feature fusion structures can be complex network structures formed by interconnected multi-layered networks. For example, a feature extraction structure can be a network structure composed of convolutional layers, local average pooling layers, and sampling layers. A feature fusion structure can be a network structure composed of convolutional layers, global average pooling layers, normalization layers, and fully connected layers.

[0065] Global dimensional features refer to features that reflect the overall information of the object to be detected. For example, global dimensional features can reflect large-scale facial information, such as facial contour features and facial texture features. Facial texture features can include the color and brightness distribution of pixels in the facial region of the image to be detected. Local dimensional features refer to features that reflect local information of the object to be detected. For example, local dimensional features can reflect features of specific facial regions, such as hairstyle, glasses, nose, mouth, and the distances between various facial features, among others.

[0066] Specifically, the computer device can perform feature extraction processing on its respective input data through each processing unit. For example, for the current processing unit in at least one processing unit, the computer device can extract global and local dimension features from the input data through the feature extraction structure in the current processing unit. Furthermore, the computer device can also perform feature fusion processing on the extracted global and local dimension features through each processing unit to obtain corresponding fused features. For example, for the current processing unit in at least one processing unit, the computer device can perform feature fusion processing on the global and local dimension features extracted by the feature extraction structure in the current processing unit through the feature fusion structure in the current processing unit to obtain the fused features output by the current processing unit.

[0067] In a straightforward manner, when the liveness detection model includes only one processing unit, steps S202 and S204 are executed sequentially; when the liveness detection model includes more than one processing unit, steps S202 and S204 are executed alternately. That is, the input data of the first processing unit is first determined, and the input data is processed by the first processing unit to obtain the fusion feature output by the first processing unit. Then, the input data of the next sequential processing unit is determined, and the input data is processed by the next sequential processing unit to obtain the fusion feature output by the next sequential processing unit. This process continues until the fusion feature output by the last sequential processing unit is obtained.

[0068] In one embodiment, each processing unit extracts features from its own input data to obtain global and local features, and then performs feature fusion processing on the global and local features to obtain corresponding fused features. This includes: the first processing unit extracts features from the image to be detected to obtain first global and first local features, and then performs feature fusion processing on the first global and first local features to obtain the fused features output by the first processing unit; for each processing unit other than the first processing unit, features are extracted from the fused features output by the preprocessing unit before the corresponding processing unit to obtain second global and second local features, and then the second global and second local features are performed feature fusion processing to obtain the fused features output by the corresponding processing unit.

[0069] Specifically, since the input data of the first processing unit is the image to be detected, the computer device performs feature extraction processing on the image to be detected through the first processing unit to obtain first global dimension features and first local dimension features. Then, it performs feature fusion processing on the first global dimension features and the first local dimension features to obtain the fused features output by the first processing unit. For example, the first processing unit performs a weighted summation of the first global dimension features and the first local dimension features to obtain the corresponding fused features. Further, the liveness detection model uses the fused features output by the first processing unit as the input data of the second sequential processing unit, and the second sequential processing unit performs further feature extraction and feature fusion processing on this input data to obtain the fused features output by the second sequential processing unit. This process continues until the fused features output by the final sequential processing unit are obtained.

[0070] In one embodiment, the computer device acquires an image to be detected corresponding to the object to be detected, and performs feature extraction on the image to be detected by the first processing unit among multiple processing units to obtain a first global dimension feature and a first local dimension feature. The first global dimension feature and the first local dimension feature are then fused to obtain the fused feature output by the first processing unit. For each processing unit other than the first unit, feature extraction is performed on the fused feature output by the preprocessing unit before the corresponding processing unit to obtain a second global dimension feature and a second local dimension feature. The second global dimension feature and the second local dimension feature are then fused to obtain the fused feature output by the corresponding processing unit. Based on the fused feature output by each processing unit, the liveness detection result corresponding to the object to be detected is determined.

[0071] In this embodiment, by deploying multiple processing units and using the output of the preprocessing unit as the input of the current processing unit, the fusion features output by each processing unit in the processing unit sequence can become increasingly abstract, thereby obtaining a better feature representation.

[0072] Step S208: Determine the liveness detection result corresponding to the object to be detected based on the fusion features output by each processing unit.

[0073] The liveness detection model may also include a liveness classification structure, which can output liveness detection results. For example, when the liveness classification structure outputs a target value, the image to be detected can be considered as an image captured from a real live subject. When the liveness classification structure outputs a non-target value, the image to be detected can be considered as a photo or video of another person's face, or an image obtained by applying a face mask to another person.

[0074] Specifically, when the fused features output by each processing unit are obtained, the liveness detection model can input these features into the liveness classification structure, which then outputs a liveness detection score for the image to be detected. Further, the liveness detection model determines whether the detection score is greater than or equal to a preset score threshold. If it is greater than or equal to the predicted score threshold, the image to be detected is considered to be an image captured for a real live subject; if it is less than the predicted score threshold, the image to be detected is considered to be an image captured for a non-real live subject.

[0075] refer to Figure 4 Since each processing unit, except for the first processing unit, takes the output of the preprocessing unit as the input of the current processing unit and further processes the output of the preprocessing unit to obtain more abstract fusion features, the level of abstraction of the fusion features output by each processing unit is different. This makes the liveness detection results obtained by combining fusion features with different levels of abstraction more accurate.

[0076] In the aforementioned liveness detection method, by acquiring the image to be detected and the fusion features output by the preprocessing unit, the input data corresponding to each processing unit can be determined based on the image to be detected and the fusion features output by the preprocessing unit. Thus, each processing unit can perform feature extraction and feature fusion processing on its corresponding input data to obtain the corresponding fusion features. By obtaining the fusion features output by each processing unit, the fusion features can be combined to obtain the liveness detection result corresponding to the object to be detected. Since it integrates global and local dimensional features to perform liveness detection on the image to be detected, compared to determining the liveness detection result through noise distribution, this application can still obtain accurate liveness detection results even when the image to be detected does not meet the premise that "the difference between the image to be detected and the real image may conform to a certain distribution of noise".

[0077] Furthermore, since no overly strong prior assumptions are imposed on global and local dimensional features, the impact on generalization is minimal, allowing it to generalize well to different data distributions. This enables the liveness detection model to identify forged images using ordinary RGB images as carriers. Simultaneously, this application requires no user facial movements or specific hardware; liveness detection can be performed based solely on a single image, significantly reducing the cost of liveness detection.

[0078] In one embodiment, each processing unit extracts features from its respective input data to obtain global and local dimension features, including: for each processing unit, extracting global initial features and local initial features from the corresponding input data; performing local pooling on the local initial features to obtain local statistical features, and obtaining corresponding local difference features based on the feature differences between the local initial features and the local statistical features; and performing feature supplementation on the global initial features and local difference features respectively to obtain the corresponding global and local dimension features.

[0079] Specifically, to obtain more accurate liveness detection results, the feature extraction structure in each processing unit can extract multi-dimensional features from the corresponding input data, resulting in global and local dimensional features. (Reference) Figure 5 , Figure 5 A schematic diagram of a feature extraction structure in one embodiment is shown. To better describe the embodiments of this application, the following description uses any one of the at least one processing unit to illustrate feature extraction. Figure 5As shown, the feature extraction structure in the processing unit may include a global feature extraction channel and a local feature extraction channel. When the processing unit obtains input data, it can input the input data into the global feature extraction channel and the local feature extraction channel, so that the global feature extraction channel performs convolution processing on the input data to obtain the global initial feature (X). C This involves convolving the local feature extraction channels with the input data to obtain local initial features (X). F ).

[0080] To further extract detailed features from the input data and obtain more pure local dimensional features, the local feature extraction channel in the feature extraction structure can also perform local average pooling (LAP) on the local initial features to obtain local statistical features in the local initial features, and determine the feature differences between the local initial features and the local statistical features. Based on the feature differences, more pure local dimensional features are obtained, that is, local difference features are obtained.

[0081] Since the global initial features may include detailed features from the input data, and the local difference features may include global features from the input data, the feature extraction structure can perform feature supplementation processing on the global initial features and the local difference features respectively to obtain global dimension features and local dimension features.

[0082] In one embodiment, the feature size of the global initial feature differs from the feature size of the local initial feature. Specifically, the feature sizes of the two can differ by a factor of two; for example, the global initial feature X... C Can be Local initial feature X F It can be X F ∈R e*h*w Where R represents that each parameter in the feature is a real number; e represents the number of channels in the feature; h represents the height of the feature; and w represents the width of the feature.

[0083] In one embodiment, the processing unit can utilize the formula LAP(X) F ;) Local pooling is performed on the initial local features to obtain local statistical features from the processed features. Here, α represents the receptive field size during local average pooling. For example, the processing unit can determine the size of the sliding window, slide the sliding window across the feature map corresponding to the initial local features, and extract the sub-regions selected by the sliding window in the feature map. The average value of these sub-regions is then calculated. This process is repeated until the average value of the last sub-region is obtained.

[0084] In one embodiment, the processing unit can be configured using a formula. Determine the characteristics of local regional differences By directly subtracting the local initial features from the local statistical features, the corresponding local differential features can be obtained quickly, thereby improving the efficiency of determining local differential features.

[0085] In one embodiment, reference Figure 5 The processing unit may include a feature extraction structure for extracting global and local dimensional features from the input data. This feature extraction structure may include a global feature extraction channel and a local feature extraction channel. The global feature extraction channel extracts initial global features from the input data and performs feature enhancement processing to obtain global dimensional features. The local feature extraction channel extracts initial local features from the input data and performs local pooling processing to obtain local statistical features. Furthermore, based on the feature differences between the initial local features and the local statistical features, local difference features are obtained, and these difference features are further enhanced to obtain local dimensional features. Since the local feature extraction channel extracts local dimensional features from the input data, primarily focusing on detailed features, while the global feature extraction channel extracts global dimensional features, subtracting local statistical features from the initial local features yields better detailed features. The global feature extraction channel, however, does not need to extract detailed features; therefore, it does not require the feature difference determination process.

[0086] In the above embodiments, by determining the feature differences between local initial features and local statistical features, local difference features containing purer detailed features can be obtained, thereby making the subsequent local dimensional features obtained based on the local difference features more accurate. Furthermore, by removing local statistical features from the local initial features, information irrelevant to the liveness detection model can be eliminated, thus reducing the risk of fitting irrelevant information and improving the generalization ability of the liveness detection model.

[0087] In one embodiment, feature supplementation processing is performed on the global initial features and local difference features respectively to obtain corresponding global dimension features and local dimension features, including: extracting global facial information from the local difference features and supplementing the global facial information to the global initial features to obtain global dimension features; extracting local part information from the global initial features and supplementing the local part information to the local difference features to obtain local dimension features.

[0088] Specifically, local difference features may include global facial information, and correspondingly, global initial features may include local part information. Therefore, to obtain local dimensional features containing more detailed information, the processing unit can also extract local part information from the global initial features and supplement it into the local difference features to obtain local dimensional features. Similarly, to obtain global dimensional features containing more global information, the current processing unit can extract global facial information from the local difference features and supplement it into the global initial features to obtain global dimensional features. Here, global facial information reflects the overall information of the object to be detected, while local part information reflects the information of local parts of the object to be detected.

[0089] In one embodiment, the processing unit can determine local dimensional features using the following formula.

[0090]

[0091] in, This represents a convolution operation whose input is local differential features. The parameter of the convolution kernel is W. F →F W F→F This represents the fine-grained convolution kernel parameters; This represents a convolution operation, whose input is the global initial feature X. C The parameters of the convolution kernel are W. C→F W C→F This represents the convolution kernel parameters from coarse-grained to fine-grained; upsample represents upsampling.

[0092] In one embodiment, the processing unit can determine the global dimensional features using the following formula.

[0093]

[0094] Among them, X C Represents the initial global features. Representing local differences, f(X) C W C→C () represents a convolution operation, whose input is X. C The parameters of the convolution kernel are W. C→C W C→C This represents the coarse-to-coarse convolution kernel parameters; This represents a convolution operation, whose input is... The parameter of the convolution kernel is W. F→C W F→c This represents the convolution kernel parameters, ranging from fine-grained to coarse-grained. This represents pooling, where α is the size of the receptive field during pooling.

[0095] In one embodiment, reference Figure 5 The processing unit can perform convolution processing on the initial global features through the global feature extraction channel to obtain the first convolution result, and perform convolution processing and upsampling processing on the local difference features to obtain global facial information. Furthermore, the global feature extraction channel can fuse the global facial information and the first convolution result to supplement the initial global features with the global facial information, obtaining global dimensional features. For example, global feature extraction can obtain the corresponding global dimensional features by superimposing the global facial information and the first convolution result.

[0096] The processing unit can perform convolution processing on local differential features through the local feature extraction channel to obtain the second convolution result, and perform convolution processing on the global initial features to obtain local part information. The local part information is then fused with the second convolution result to supplement the local differential features with the local part information to obtain local dimension information.

[0097] In the above embodiments, by supplementing the global facial information in the local difference features into the global initial features, a global dimensional feature containing more overall information about the object to be detected can be obtained; by supplementing the local part information in the global initial features into the local difference features, a local dimensional feature containing more local part information about the object to be detected can be obtained; by obtaining both the global dimensional feature containing more overall information about the object to be detected and the local dimensional feature containing more local part information about the object to be detected, the liveness detection result determined based on the global dimensional feature and the local dimensional feature can be more accurate.

[0098] In one embodiment, feature fusion processing is performed on global and local dimension features to obtain corresponding fused features, including: determining the feature weights corresponding to the global and local dimension features respectively; and performing feature fusion processing on the global and local dimension features based on the feature weights corresponding to the global and local dimension features respectively to obtain corresponding fused features.

[0099] Specifically, for each of the at least one processing unit, feature fusion processing can be performed on the corresponding global dimension features and local dimension features through the feature fusion structure in each processing unit to obtain the corresponding fused features. To better describe this embodiment, the following explanation uses any one of the at least one processing units as an example:

[0100] When the feature extraction structure in the processing unit extracts global and local dimension features from the input data, the feature fusion structure in the processing unit can determine the feature weights corresponding to each global and local dimension feature. Based on these feature weights, the global and local dimension features are fused to obtain the corresponding fused features. For example, the feature fusion structure in the processing unit can perform a weighted summation of the global and local dimension features to obtain the fused features.

[0101] In one embodiment, reference Figure 6 The feature weights corresponding to the global dimension features are C. C The feature weights corresponding to the time and local dimensional features are F. C At that time, the feature fusion structure can integrate global dimensional features. Multiplied by feature weight C C The first weighted feature is obtained. And the feature fusion structure can integrate local dimensional features Multiplied by feature weight F C The second weighted feature is obtained. The first and second weighted features are then convolved separately, and the results are superimposed to obtain the final output fused feature X. fina .in, Figure 6 A schematic diagram of a feature fusion structure in one embodiment is shown.

[0102] In the above embodiments, by assigning different weight values ​​to features of different dimensions, the feature fusion structure can adaptively select effective multi-dimensional information, thereby enabling the fused features obtained based on the selected effective multi-dimensional information to better reflect the liveness characteristics.

[0103] In one embodiment, determining the feature weights corresponding to the global and local dimension features includes: performing convolution processing on the global and local dimension features respectively to obtain global and local dimension convolution results, and aggregating the global and local dimension convolution results to obtain corresponding aggregated features; and determining the feature weights corresponding to the global and local dimension features based on the aggregated features.

[0104] Specifically, when the processing unit needs to determine the feature weights corresponding to global and local dimension features, the feature fusion structure in the processing unit can perform convolution processing on the global and local dimension features respectively, obtaining global and local convolution results, and then aggregate the global and local convolution results to obtain the basis for determining the feature weights. For example, refer to... Figure 6The feature fusion structure can be obtained through the formula The aggregated features are obtained. Among them... This represents the result of local dimension convolution. X represents the result of global dimension convolution. all This indicates aggregation features.

[0105] Furthermore, the feature fusion structure determines the feature weights for global and local dimensions based on the aggregated features. This allows for the subsequent selection of more generalizable features from both local and global dimensions using these determined feature weights, preventing liveness detection from fitting useless information and thus improving the generalization ability of the liveness detection model.

[0106] In one embodiment, determining the feature weights corresponding to the global dimension features and local dimension features based on the aggregated features includes: determining a first matrix vector and a first normalization function corresponding to the global dimension features, and determining a second matrix vector and a second normalization function corresponding to the local dimension features; performing global average pooling on the aggregated features to obtain a global pooling result; performing a first processing on the global pooling result using the first matrix vector and the first normalization function to obtain the feature weights corresponding to the global dimension features; and performing a second processing on the global pooling result using the second matrix vector and the second normalization function to obtain the feature weights corresponding to the local dimension features.

[0107] Specifically, the feature fusion structure in the processing unit can determine a first matrix vector and a first normalization function corresponding to the global dimension features, and a second matrix vector and a second normalization function corresponding to the local dimension features. The first matrix vector, the first normalization function, the second matrix vector, and the second normalization function can all be obtained through training a liveness detection model. For example, the liveness detection model can be trained through the last iteration to obtain the first matrix vector, the first normalization function, the second matrix vector, and the second normalization function. The first matrix vector and the second matrix vector can each be a fully connected layer. Similarly, the first normalization function and the second normalization function can each be a normalization layer.

[0108] Furthermore, when the aggregated features are obtained, the feature fusion structure can perform global average pooling (GAP) on these aggregated features to obtain a global pooling result. For example, the feature fusion structure can perform an average operation on the entire feature map corresponding to the aggregated features to obtain a global pooling result. The feature fusion structure inputs the global pooling result to the fully connected layer corresponding to the first matrix vector, and inputs the data output from the fully connected layer corresponding to the first matrix vector to the normalization layer corresponding to the first normalization function, so as to perform a first processing on the global pooling result through the first matrix vector and the first normalization function to obtain the feature weights corresponding to the global dimension features. Correspondingly, the feature fusion structure inputs the global pooling result to the fully connected layer corresponding to the second matrix vector, and inputs the data output from the fully connected layer corresponding to the second matrix vector to the normalization layer corresponding to the second normalization function, so as to perform a second processing on the global pooling result through the second matrix vector and the second normalization function to obtain the feature weights corresponding to the local dimension features.

[0109] In one embodiment, the feature fusion structure in the processing unit can determine the feature weights corresponding to the global dimension features using the following formula:

[0110] C C =relu(BN) C (W C ·GAP(X all )))

[0111] Where ReLU represents the activation function, and BN... C Let w represent the first normalization function. C GAP(X) represents the first matrix vector. all ) represents the result of global pooling.

[0112] In one embodiment, the feature fusion structure in the processing unit can determine the feature weights corresponding to local dimensional features using the following formula:

[0113] F C =relu(BN) F (W F ·GAP(X all )))

[0114] Where ReLU represents the activation function, and BN... F W represents the second normalization function. F GAP(X) represents the second matrix vector. all ) represents the result of global pooling.

[0115] In the above embodiments, the corresponding feature weights can be determined simply by performing fully connected layer processing and normalization on the global pooling results, thus greatly improving the efficiency of feature weight determination.

[0116] In one embodiment, when the liveness detection model includes only one processing unit, the computer device acquires the image to be detected and uses it as input data for the processing unit. The processing unit extracts features from the input data to obtain global and local features, and then performs feature fusion processing on these global and local features to obtain fused features. Further, the liveness detection model inputs the fused features output by the processing unit into a liveness classification structure, and outputs the liveness detection result for the image to be detected through the liveness classification structure.

[0117] In one embodiment, when the liveness detection model includes multiple processing units, the computer device acquires an image to be detected and uses this image as input data for the first processing unit. The first processing unit performs feature extraction and feature fusion processing on the input data to obtain the fused features output by the first processing unit. The liveness detection model uses the fused features output by the first processing unit as input data for the second sequential processing unit, so that the second sequential processing unit performs feature extraction and feature fusion processing on the input data to obtain the fused features output by the second sequential processing unit. This process continues until the fused features output by each processing unit are obtained. Thus, the liveness classification structure in the liveness detection model can obtain the liveness detection result based on the fused features output by each processing unit.

[0118] Traditional techniques assume that the difference between the image to be detected and the real image may conform to a certain distribution of noise. A neural network is then used to estimate the feature map of the noise, and the image to be detected is classified based on this feature map. However, this method only uses a portion of the information in the image to be detected (noise information) for liveness detection, discarding non-noise information. Therefore, when the image to be detected does not meet the premise that "the difference between the image to be detected and the real image may conform to a certain distribution of noise," detection errors occur. The embodiments of this application, by adaptively selecting and fusing target features from multi-dimensional feature information, can reduce the probability of detection errors due to insufficient information in a single dimension, while also avoiding interference from redundant information. This effectively improves the liveness detection model's ability to detect different types of images to be detected and significantly enhances the model's generalization ability.

[0119] In one embodiment, the liveness detection method is executed by a liveness detection model, which is trained through a model training step. The model training step includes: acquiring a liveness detection model to be trained, sample images, and sample labels corresponding to the sample images; the liveness detection model includes at least one processing unit to be trained; determining the input samples corresponding to each of the at least one processing unit to be trained; the input sample of the current processing unit to be trained includes the sample image and at least one of the fused prediction features output by the previous processing unit to be trained; extracting features from the input samples of each processing unit to obtain global prediction features and local prediction features, and performing feature fusion processing on the global prediction features and local prediction features to obtain corresponding fused prediction features; determining the prediction result of liveness prediction for the sample image based on the fused prediction features output by each processing unit to be trained; and training the liveness detection model based on the prediction result and sample labels until the training stopping condition is met, thus obtaining a trained liveness detection model.

[0120] Specifically, before performing liveness detection on the image to be detected using the liveness detection model, the model needs to be trained. The computer can acquire a large number of sample images and their corresponding labeling. Based on these images, the computer can iteratively train the liveness detection model. In each iteration, the computer uses the sample image as the input sample for the first processing unit in at least one training unit. This first unit extracts features from the input sample, obtaining global and local prediction features. These features are then fused to obtain the fused prediction features output by the first unit. Further, when the liveness detection model includes multiple processing units, the model can use the fused prediction features output by the first unit as the input sample for the second unit. This allows the second unit to extract and fuse features from the input sample, obtaining the fused prediction features output by the second unit. Furthermore, the liveness detection model uses the fused prediction features output by the second sequential training unit as the input sample for the third sequential training unit, and so on, until the fused prediction features output by the final sequential training unit are obtained. The process by which the training units output fused prediction features can refer to the process described above for the processing units to output fused features.

[0121] Furthermore, the liveness detection model determines the prediction result for liveness prediction of the sample image based on the fused prediction features output by each processing unit to be trained, and trains the liveness detection model based on the difference between the prediction result and the corresponding sample label. When a preset number of iterations of training have been performed, or when the output result of the liveness detection model meets the preset requirements, the training stopping condition can be considered to have been met. At this time, the computer device stops training the liveness detection model, and the trained liveness detection model is obtained.

[0122] In this embodiment, the liveness detection model to be trained is trained so that the detection results output by the trained liveness detection model can be more accurate.

[0123] In one embodiment, the prediction result includes a predicted depth map and a liveness prediction label. Based on the fused prediction features output by each training processing unit, the prediction result for liveness prediction of the sample image is determined, including: using the depth map construction structure in the liveness detection model to perform depth map construction processing on the fused prediction features output by each training processing unit to obtain the corresponding predicted depth map; and using the liveness classification structure in the liveness detection model to perform liveness classification processing on the fused prediction features output by each training processing unit to obtain the corresponding liveness prediction label.

[0124] The predicted depth map refers to the depth map output by the liveness detection model to be trained, which reflects the depth information of the object to be detected in the sample image. For example, the predicted depth map may specifically be the depth map of the face of the object to be detected in the sample image. The liveness prediction label refers to the label output by the liveness detection model to be trained, which reflects whether the training sample was taken for a live subject. For example, the liveness prediction label may specifically be "the sample image is an image captured for a real live subject", or "the sample image is a photo or video of another person's face, or an image captured by applying another person's face mask".

[0125] Specifically, the liveness detection model to be trained may include a depth map construction structure and a liveness classification structure. When the fused prediction features output by each processing unit are obtained, the liveness detection structure can input these features into the depth map construction structure and the liveness classification structure. Thus, the depth map construction structure can obtain a predicted depth map corresponding to the sample image based on the fused prediction features output by each processing unit, and the liveness classification structure can obtain a liveness prediction label corresponding to the sample image based on the fused prediction features output by each processing unit. The liveness classification structure can specifically be a binary classification network. In one embodiment, the depth map construction structure may only be used to train the liveness detection model and not participate in the detection process of the image to be detected.

[0126] In one embodiment, reference Figure 7 , Figure 7 The diagram illustrates the training of a liveness detection model in one embodiment. The liveness detection model to be trained can output corresponding fused prediction features through each training processing unit, thereby enabling the depth map construction structure and liveness classification structure to obtain corresponding predicted depth maps and liveness prediction labels based on the fused prediction features.

[0127] In the above embodiments, by obtaining the predicted depth map and the liveness prediction label, the liveness detection model to be trained can be trained based on the predicted depth map and the liveness prediction label.

[0128] In one embodiment, the sample labels include a target depth map and a liveness classification label; the liveness detection model is trained based on the prediction results and the sample labels until a training stopping condition is met, including: determining a first difference between the predicted depth map and the target depth map, and determining a second difference between the liveness prediction label and the liveness classification label; the liveness detection model is trained using the first and second differences until a training stopping condition is met.

[0129] Here, the target depth map refers to a standard depth map, and the liveness classification label refers to an accurate classification label. When acquiring sample images, model trainers can perform labeling processing on the sample images to obtain the corresponding target depth map and liveness classification label. In one embodiment, the target depth map of a sample image acquired from a non-live object can be a completely black image.

[0130] Specifically, the computer device can determine a first difference between the predicted depth map and the corresponding target depth map, and a second difference between the liveness prediction label and the liveness classification label. It can construct a first loss function based on the first difference and a second loss function based on the second difference. The liveness detection model is trained using the constructed first and second loss functions until the training stops when the stopping condition is met.

[0131] In one embodiment, the computer device can determine the first loss function L using the following function. Dep :

[0132]

[0133] Where Data is the sample set, x i Let dep represent the i-th sample image in the sample set. i Dep(Ext(x) represents the target depth map corresponding to the i-th sample in the sample set. i )) represents the predicted depth map, Ext(x i) represents at least one fusion prediction feature corresponding to the i-th sample, and Dep(.) represents the depth map construction structure for depth map construction processing.

[0134] In one embodiment, the computer device can determine the second loss function L using the following function. Cls :

[0135]

[0136] Where Data is the sample set, x i Let dep represent the i-th sample image in the sample set. i Ext(x) represents the liveness classification label corresponding to the i-th sample in the sample set. i ) represents at least one fusion prediction feature corresponding to the i-th sample, and BC(.) represents the normalization layer.

[0137] In one embodiment, the computer device can also determine the corresponding LBP (Local Binary Pattern) features and reflectance map features through the fusion prediction features output by each device, and train the liveness detection model to be trained using the determined LBP features and reflectance map features.

[0138] In the above embodiments, by constructing a predicted depth map and a liveness prediction label, the liveness detection model can be trained by combining the first difference between the predicted depth map and the target depth map, and the second difference between the liveness prediction label and the liveness classification label, so that the trained liveness detection model can be more accurate.

[0139] This application also provides an application scenario in which the above-described liveness detection method is applied. Specifically, the liveness detection method is applied in this scenario as follows:

[0140] To enhance application security, when logging into the application or transferring resources through it, the application can capture a facial image to be detected. A liveness detection model is then used to process the captured facial image, yielding the corresponding liveness detection result. If the liveness detection result determines that the facial image was captured from a natural person, the application will either perform a login operation or a resource transfer operation. If the liveness detection result determines that the facial image is a photograph or video of another person's face, or an image captured using a mask of another person's face, the application will pause the login operation or the resource transfer operation and display a corresponding risk warning message.

[0141] This application also provides another application scenario in which the above-described liveness detection method is applied. Specifically, the liveness detection method is applied in this scenario as follows:

[0142] When remote account opening is required, in order to verify the true identity of the account holder, the terminal can capture a facial image including the account holder's face through the front-facing camera, and input the facial image into the liveness detection model. The liveness detection model performs liveness detection, and when the liveness detection result indicates that the facial image is an image captured from a natural person, the liveness detection result is determined to be successful.

[0143] The above application scenarios are merely illustrative. It is understood that the application of the liveness detection method provided in the embodiments of this application is not limited to the above scenarios. For example, it can also be used in community access control verification scenarios, driver remote authentication scenarios, etc.

[0144] In one specific embodiment, referring to Question 8, the liveness detection method includes:

[0145] S802, acquire the original image and identify the facial region of the object to be detected in the original image; adjust the size of the facial region to obtain the target region (for example, enlarge the size of the face recognition box to obtain the target region including the complete face and part of the background); crop the target region to obtain the image to be detected including the object to be detected.

[0146] S804, for the first processing unit in at least one processing unit, the image to be detected is used as the input data of the first processing unit.

[0147] S806, for each processing unit other than the first processing unit in at least one processing unit, the fusion features output by the preceding processing unit before the corresponding processing unit are used as the input data of the corresponding processing unit, wherein the preceding processing unit refers to a processing unit that is adjacent to the current processing unit and located before the current processing unit.

[0148] S808, the processing unit includes a feature extraction structure. For each processing unit in at least one processing unit, the global initial features and local initial features in the corresponding input data are extracted through the feature extraction structure in the current processing unit. Local pooling is performed on the local initial features to obtain local statistical features in the local processed features. Based on the feature difference between the local initial features and the local statistical features, the corresponding local difference features are obtained.

[0149] S810 extracts global facial information from local difference features through the feature extraction structure in the current processing unit, and supplements the global facial information to the global initial features to obtain global dimensional features, such as facial contour features and facial texture features; it also extracts local part information from the global initial features and supplements the local part information to the local difference features to obtain local dimensional features, such as hairstyle, glasses, nose, mouth, and distances between various facial organs, or one or more of these features.

[0150] S812, the processing unit also includes a feature fusion structure, which performs convolution processing on global dimension features and local dimension features respectively to obtain global dimension convolution results and local dimension convolution results, and then performs aggregation processing on the global dimension convolution results and local dimension convolution results to obtain aggregated features.

[0151] S814, through the feature fusion structure, determines the first matrix vector and the first normalization function corresponding to the global dimension features, and determines the second matrix vector and the second normalization function corresponding to the local dimension features. For example, the first matrix vector and the second matrix vector can each be a fully connected layer, and the first normalization function and the second normalization function can each be a normalization layer.

[0152] S816 performs global average pooling on aggregated features through a feature fusion structure to obtain a global pooling result; for example, the feature fusion structure can perform an average operation on the entire feature map corresponding to the aggregated features to obtain a global pooling result.

[0153] S818 uses a feature fusion structure and performs a first processing on the global pooling result based on a first matrix vector and a first normalization function to obtain feature weights corresponding to global dimension features; then, it performs a second processing on the global pooling result using a second matrix vector and a second normalization function to obtain feature weights corresponding to local dimension features.

[0154] S820 uses a feature fusion structure and performs feature fusion processing on global and local features based on their respective feature weights to obtain the corresponding fused features.

[0155] S822, based on the fusion features output by each processing unit, determine the liveness detection result corresponding to the object to be detected. For example, input the fusion features output by each processing unit into the liveness classification structure, and output the detection score of the image to be detected for liveness detection through the liveness classification structure.

[0156] In the aforementioned liveness detection method, by acquiring the image to be detected and the fusion features output by the preprocessing unit, the input data corresponding to each processing unit can be determined based on the image to be detected and the fusion features output by the preprocessing unit. Thus, each processing unit can perform feature extraction and feature fusion processing on its corresponding input data to obtain the corresponding fusion features. By obtaining the fusion features output by each processing unit, the various fusion features can be combined to obtain the liveness detection result corresponding to the object to be detected. Since it integrates global and local dimensional features to perform liveness detection on the image to be detected, compared to determining the liveness detection result through noise distribution, this application can still obtain accurate liveness detection results even when the image to be detected does not meet the premise that "the difference between the image to be detected and the real image may conform to a certain distribution of noise".

[0157] In one specific embodiment, reference Figure 9 Liveness detection methods include:

[0158] S902, acquire the liveness detection model to be trained, sample images, and sample labels corresponding to the sample images; the liveness detection model includes at least one processing unit to be trained.

[0159] S904, determine the input samples corresponding to at least one processing unit to be trained. For example, the input sample corresponding to the first processing unit to be trained is a sample image, and the input samples corresponding to the other processing units besides the first processing unit to be trained are fusion prediction features. The input sample of the current processing unit to be trained in at least one processing unit to be trained includes at least one of the sample image and the fusion prediction features output by the previous processing unit to be trained.

[0160] S906: Each processing unit extracts features from its respective input samples to obtain global prediction features and local prediction features, and then performs feature fusion processing on the global prediction features and local prediction features to obtain the corresponding fused prediction features.

[0161] S908 uses the depth map construction structure in the liveness detection model to perform depth map construction processing on the fusion prediction features output by each training processing unit to obtain the corresponding prediction depth map, such as obtaining the depth map of the face of the object to be detected in the sample image.

[0162] S910 uses the liveness classification structure in the liveness detection model to perform liveness classification processing on the fusion prediction features output by each training processing unit to obtain the corresponding liveness prediction label. For example, it obtains the label "the sample image is an image collected for a real live body" or "the sample image is a photo or video of another person's face, or an image collected by applying another person's face mask".

[0163] S912, determine the first difference between the predicted depth map and the target depth map, for example, by subtracting the predicted depth map from the target depth map to obtain the first difference, and determine the second difference between the liveness prediction label and the liveness classification label; train the liveness detection model using the first difference and the second difference until the training stops when the stopping condition is met, for example, until the preset number of training iterations is reached.

[0164] S914, acquire the image to be detected corresponding to the object to be detected; determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected and at least one of the fusion features output by the preprocessing unit before the corresponding processing unit.

[0165] S916 extracts features from its respective input data through each processing unit to obtain global and local features, and then performs feature fusion processing on the global and local features to obtain the corresponding fused features.

[0166] S918 determines the liveness detection result corresponding to the object to be detected based on the fusion features output by each processing unit.

[0167] It should be understood that, although Figure 2 , Figures 8-9 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 , Figures 8-9 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0168] In one embodiment, such as Figure 10 As shown, a liveness detection device 1000 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a data acquisition module 1002, a feature fusion module 1004, and a result output module 1006, wherein:

[0169] The data acquisition module 1002 is used to acquire the image to be detected corresponding to the object to be detected; determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected and at least one of the fusion features output by the preprocessing unit before the corresponding processing unit.

[0170] The feature processing module 1004 is used to extract features from the input data of each processing unit to obtain global dimension features and local dimension features, and to perform feature fusion processing on the global dimension features and local dimension features to obtain corresponding fused features.

[0171] The result output module 1006 is used to determine the liveness detection result corresponding to the object to be detected based on the fusion features output by each processing unit.

[0172] In one embodiment, reference Figure 11 The data acquisition module 1002 includes an image acquisition module 1021, which is used to acquire the original image and identify the facial region of the object to be detected in the original image; adjust the size of the facial region to obtain the target region; and crop the target region to obtain the image to be detected that includes the object to be detected.

[0173] In one embodiment, the data acquisition module 1002 is used to take the image to be detected as the input data of the first processing unit in at least one processing unit; and for each processing unit other than the first processing unit in at least one processing unit, the fusion features output by the preprocessing unit before the corresponding processing unit are taken as the input data of the corresponding processing unit.

[0174] In one embodiment, the feature processing module 1004 is used to extract features from the image to be detected through the first processing unit among a plurality of processing units, to obtain a first global dimension feature and a first local dimension feature, and to perform feature fusion processing on the first global dimension feature and the first local dimension feature to obtain the fused feature output by the first processing unit; for each processing unit other than the first processing unit, feature extraction is performed on the fused feature output by the preprocessing unit before the corresponding processing unit to obtain a second global dimension feature and a second local dimension feature, and to perform feature fusion processing on the second global dimension feature and the second local dimension feature to obtain the fused feature output by the corresponding processing unit.

[0175] In one embodiment, the feature processing module 1004 includes a feature extraction module 1041, configured to extract global initial features and local initial features from the corresponding input data for each of the at least one processing unit; perform local pooling processing on the local initial features through the current processing unit to obtain local statistical features in the local processed features, and obtain corresponding local difference features based on the feature differences between the local initial features and the local statistical features; and perform feature supplementation processing on the global initial features and local difference features through the current processing unit to obtain corresponding global dimension features and local dimension features.

[0176] In one embodiment, the feature extraction module 1041 is further configured to extract global facial information from local difference features and supplement the global facial information to the global initial features to obtain global dimensional features; and extract local part information from the global initial features and supplement the local part information to the local difference features to obtain local dimensional features.

[0177] In one embodiment, the feature processing module 1004 further includes a feature fusion module 1042, which is used to determine the feature weights corresponding to the global dimension features and the local dimension features respectively; and to perform feature fusion processing on the global dimension features and the local dimension features based on the feature weights corresponding to the global dimension features and the local dimension features respectively, so as to obtain the corresponding fused features.

[0178] In one embodiment, the feature fusion module 1042 is further configured to perform convolution processing on global dimension features and local dimension features respectively to obtain global dimension convolution results and local dimension convolution results, and to perform aggregation processing on global dimension convolution results and local dimension convolution results to obtain corresponding aggregated features; and to determine the feature weights corresponding to global dimension features and local dimension features respectively based on the aggregated features.

[0179] In one embodiment, the feature fusion module 1042 is further configured to determine a first matrix vector and a first normalization function corresponding to the global dimension features, and to determine a second matrix vector and a second normalization function corresponding to the local dimension features; perform global average pooling on the aggregated features to obtain a global pooling result; perform a first processing on the global pooling result using the first matrix vector and the first normalization function to obtain feature weights corresponding to the global dimension features; and perform a second processing on the global pooling result using the second matrix vector and the second normalization function to obtain feature weights corresponding to the local dimension features.

[0180] In one embodiment, the liveness detection device 1000 further includes a training module 1008, used to acquire a liveness detection model to be trained, sample images, and sample labels corresponding to the sample images; the liveness detection model includes at least one processing unit to be trained; the input samples corresponding to the at least one processing unit to be trained are determined; the input sample of the current processing unit to be trained in the at least one processing unit to be trained includes a sample image and at least one of the fusion prediction features output by the previous processing unit to be trained; each processing unit extracts features from its respective input samples to obtain global prediction features and local prediction features, and performs feature fusion processing on the global prediction features and local prediction features to obtain corresponding fusion prediction features; based on the fusion prediction features output by each processing unit to be trained, the prediction result for liveness prediction of the sample image is determined; based on the prediction result and sample labels, the liveness detection model is trained until the training stopping condition is met, and the trained liveness detection model is obtained.

[0181] In one embodiment, the prediction result includes a predicted depth map and a liveness prediction label; the training module 1008 is further configured to construct a depth map on the fused prediction features output by each training processing unit using the depth map construction structure in the liveness detection model, thereby obtaining the corresponding predicted depth map; and to perform liveness classification on the fused prediction features output by each training processing unit using the liveness classification structure in the liveness detection model, thereby obtaining the corresponding liveness prediction label.

[0182] In one embodiment, the sample labels include a target depth map and a liveness classification label; the training module 1008 is further configured to determine a first difference between the predicted depth map and the target depth map, and to determine a second difference between the liveness prediction label and the liveness classification label; the liveness detection model is trained using the first difference and the second difference until the training stops when the training stopping condition is met.

[0183] Specific limitations regarding the liveness detection device can be found in the limitations of the liveness detection method described above, and will not be repeated here. Each module in the aforementioned liveness detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0184] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores liveness detection data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a liveness detection method.

[0185] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0186] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0187] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0188] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0189] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0190] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting liveness, characterized in that, The method includes: Acquire the image to be detected corresponding to the object to be detected; Determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected, and at least one of the fusion features output by the preprocessing unit preceding the corresponding processing unit; Each of the aforementioned processing units extracts features from its respective input data to obtain global and local dimension features, and then performs feature fusion processing on the global and local dimension features to obtain corresponding fused features. Based on the fusion features output by each of the processing units, the liveness detection result corresponding to the object to be detected is determined; The process involves each processing unit extracting features from its respective input data to obtain global and local features, and then fusing these global and local features to obtain corresponding fused features, including: The first processing unit among multiple processing units extracts features from the image to be detected to obtain a first global dimension feature and a first local dimension feature. The first global dimension feature and the first local dimension feature are then fused to obtain the fused feature output by the first processing unit. For each processing unit other than the first processing unit, feature extraction is performed on the fused features output by the preprocessing unit before the corresponding processing unit to obtain the second global dimension feature and the second local dimension feature. Then, feature fusion processing is performed on the second global dimension feature and the second local dimension feature to obtain the fused features output by the corresponding processing unit.

2. The method according to claim 1, characterized in that, The step of acquiring the image to be detected corresponding to the object to be detected includes: Acquire the original image and identify the facial region of the object to be detected in the original image; The facial area is resized to obtain the target area; The target region is cropped to obtain an image to be detected that includes the object to be detected.

3. The method according to claim 1, characterized in that, The step of determining the input data corresponding to at least one processing unit includes: For the first processing unit in at least one processing unit, the image to be detected is used as the input data of the first processing unit; For each processing unit other than the first processing unit, the fused features output by the preprocessing unit preceding the corresponding processing unit are used as the input data for the corresponding processing unit.

4. The method according to claim 1, characterized in that, The step involves each processing unit extracting features from its respective input data to obtain global and local dimension features, including: For each of the at least one processing unit, global initial features and local initial features are extracted from the corresponding input data through the current processing unit; The current processing unit performs local pooling on the local initial features to obtain local statistical features in the local processed features, and obtains corresponding local difference features based on the feature differences between the local initial features and the local statistical features. The current processing unit performs feature supplementation processing on the global initial features and local difference features respectively to obtain the corresponding global dimension features and local dimension features.

5. The method according to claim 4, characterized in that, The step of performing feature supplementation processing on the global initial features and local difference features respectively to obtain corresponding global dimension features and local dimension features includes: Global facial information is extracted from the local difference features, and the global facial information is added to the global initial features to obtain global dimensional features; Local part information is extracted from the global initial features, and the local part information is added to the local difference features to obtain local dimension features.

6. The method according to claim 1, characterized in that, The feature fusion process of the global and local dimensional features to obtain the corresponding fused features includes: Determine the feature weights corresponding to the global and local dimension features respectively; Based on the feature weights corresponding to the global and local dimensional features, feature fusion processing is performed on the global and local dimensional features to obtain the corresponding fused features.

7. The method according to claim 6, characterized in that, Determining the feature weights corresponding to the global dimension features and the local dimension features includes: The global and local features are convolved separately to obtain global and local convolution results, and the global and local convolution results are aggregated to obtain the corresponding aggregated features. Based on the aggregated features, the feature weights corresponding to the global dimension features and the local dimension features are determined.

8. The method according to claim 7, characterized in that, The step of determining the feature weights corresponding to the global dimension feature and the local dimension feature based on the aggregated features includes: Determine a first matrix vector and a first normalization function corresponding to the global dimension features, and determine a second matrix vector and a second normalization function corresponding to the local dimension features; The aggregated features are subjected to global average pooling to obtain the global pooling result; The global pooling result is processed by the first matrix vector and the first normalization function to obtain the feature weights corresponding to the global dimension features. The global pooling result is processed by the second matrix vector and the second normalization function to obtain the feature weights corresponding to the local dimensional features.

9. The method according to any one of claims 1 to 8, characterized in that, The liveness detection method is executed by a liveness detection model, which is trained through a model training step, which includes: Obtain a liveness detection model to be trained, sample images, and sample labels corresponding to the sample images; the liveness detection model includes at least one processing unit to be trained. Determine the input samples corresponding to at least one training processing unit; the input sample of the current training processing unit in the at least one training processing unit includes the sample image and at least one of the fused prediction features output by the previous training processing unit; Each of the aforementioned processing units extracts features from its respective input samples to obtain global prediction features and local prediction features, and then performs feature fusion processing on the global prediction features and local prediction features to obtain corresponding fused prediction features. Based on the fused prediction features output by each of the processing units to be trained, the prediction result for liveness prediction of the sample image is determined. Based on the prediction results and the sample labels, the liveness detection model is trained until the training stops when the training stopping condition is met, thus obtaining a fully trained liveness detection model.

10. The method according to claim 9, characterized in that, The prediction result includes a predicted depth map and a liveness prediction label; determining the liveness prediction result for the sample image based on the fused prediction features output by each of the training processing units includes: By constructing a depth map in the liveness detection model, the fusion prediction features output by each of the training processing units are processed to obtain a corresponding prediction depth map. Using the liveness classification structure in the liveness detection model, the fused prediction features output by each of the training processing units are subjected to liveness classification processing to obtain the corresponding liveness prediction labels.

11. The method according to claim 10, characterized in that, The sample labels include target depth maps and liveness classification labels; the step of training the liveness detection model based on the prediction results and the sample labels until the training stops when the stopping condition is met includes: Determine a first difference between the predicted depth map and the target depth map, and determine a second difference between the liveness prediction label and the liveness classification label; The liveness detection model is trained using the first difference and the second difference until the training stops when the training cessation condition is met.

12. A liveness detection device, characterized in that, The device includes: The data acquisition module is used to acquire the image to be detected corresponding to the object to be detected; determine the input data corresponding to at least one processing unit; the input data of each processing unit includes the image to be detected, and at least one of the fusion features output by the preprocessing unit before the corresponding processing unit; The feature processing module is used to extract features from the input data of each of the processing units to obtain global dimension features and local dimension features, and to perform feature fusion processing on the global dimension features and local dimension features to obtain corresponding fused features. The result output module is used to determine the liveness detection result corresponding to the object to be detected based on the fusion features output by each of the processing units. The feature processing module is further configured to extract features from the image to be detected through the first processing unit among multiple processing units, to obtain a first global dimension feature and a first local dimension feature, and to perform feature fusion processing on the first global dimension feature and the first local dimension feature to obtain the fused feature output by the first processing unit; for each processing unit other than the first processing unit, feature extraction is performed on the fused feature output by the preprocessing unit before the corresponding processing unit to obtain a second global dimension feature and a second local dimension feature, and feature fusion processing is performed on the second global dimension feature and the second local dimension feature to obtain the fused feature output by the corresponding processing unit.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images