A living body detection method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of living body detection, in particular to a living body detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring each first feature map corresponding to a target image, detecting at least one detection object contained in each first feature map, and obtaining total detection results corresponding to each first feature map respectively. Then, based on the confidence of each sub-detection result in each total detection result, a to-be-calibrated sub-detection result corresponding to a target detection object is screened out from each total detection result. Further, the to-be-calibrated sub-detection result is calibrated to obtain a calibrated total detection result, and a living body attribute corresponding to the target image is determined according to the calibrated total detection results. Through the above method, object detection and living body classification are simultaneously performed, and the detected positive sample object and negative sample object can be further calibrated, so that the living body detection precision is improved.

Description

A method, apparatus, electronic device and storage medium for liveness detection Technical Field

[0001] This application relates to the field of liveness detection technology, specifically to a liveness detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, the video surveillance field has developed rapidly. Facial recognition, coupled with deep learning, has been widely applied to various authorized terminal devices due to its ease of deployment and strong advantages in user experience, detection speed, and accuracy. However, with the rapid development of multimedia devices and the internet, obtaining high-quality facial images has become increasingly convenient. Therefore, facial recognition systems also face significant challenges.

[0003] Currently, common technical attacks primarily originate from screen attacks and photo attacks. If high-quality images or videos of authorized users are used to attack recognition systems, it poses a significant challenge to the security of users' personal information and assets. Therefore, to protect users' information and financial rights, liveness detection technology has emerged. In facial recognition scenarios, liveness detection modules can detect fraudulent activities, thereby protecting user rights.

[0004] Currently, there are two main types of liveness detection: silent and interactive. Interactive liveness detection requires user interaction based on commands, determining whether a face is an intrusion based on the user's actions. This is primarily used in scenarios with stringent facial security requirements. Silent liveness detection, on the other hand, does not require user cooperation and makes a judgment based on five senses. From a hardware perspective, liveness detection involves using visible light cameras, multispectral cameras, and depth cameras to acquire facial images, infrared images, and facial depth images, respectively. Among these, using only visible light cameras for facial recognition is a more mainstream approach.

[0005] In related technologies, face liveness detection is usually performed in the following two ways: deploying a face detection unit and a liveness detection unit. In the deployment, the frame image is first input into the face detection unit. When a face is detected in the frame image, the detected face data is input into the liveness detection unit for liveness detection judgment.

[0006] The above approach, where face detection and liveness detection are developed and updated as two independent units, has several drawbacks in practical applications. First, the two independent modules perform data processing and forward inference separately, resulting in a longer overall running time. Second, the face detection unit, as a preceding unit to the liveness detection unit, exhibits performance differences across different scenarios, which can affect the quality of input data for the subsequent liveness detection unit, thereby impacting classification accuracy. Summary of the Invention

[0007] This application provides a liveness detection method, apparatus, electronic device, and storage medium to address the problem of low accuracy in current liveness detection methods.

[0008] In a first aspect, this application provides a method for detecting liveness, the method comprising:

[0009] Determine at least one first feature map corresponding to the acquired target image;

[0010] For each first feature map, at least one detection object in the first feature map is detected to obtain the total detection result corresponding to the first feature map; the total detection result includes the sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects, and object objects associated with the sample objects;

[0011] Based on the confidence level of each sub-detection result in each total detection result, the sub-detection results to be calibrated corresponding to the target detection object are selected from the total detection results. The detection objects of the target detection object include positive sample objects and negative sample objects.

[0012] Based on the positional relationship between each target detection object and its associated object, the detection results of each sub-detection to be calibrated are calibrated to obtain the calibrated total detection result;

[0013] Based on the calibrated overall detection results, the liveness attribute corresponding to the target image is determined.

[0014] The above method enables simultaneous target detection and localization with target liveness classification. Furthermore, it allows for further calibration of detected live and virtual targets, improving liveness detection accuracy.

[0015] In one possible embodiment, obtaining each of the first feature maps corresponding to the target image includes:

[0016] Acquire the target image;

[0017] The target image is used to extract features through a ResNet50 network to obtain various second feature maps.

[0018] The first feature maps are obtained by fusing and convolving the second feature maps using a Feature Pyramid Network (FPN).

[0019] By using the above methods, based on the network characteristics of ResNet50 and FPN networks, the detection box can be adapted to the characteristic information it contains, improving the applicability of the detection module and thus increasing the target detection speed.

[0020] In one possible embodiment, before selecting the sub-detection results to be calibrated corresponding to each target detection object from the total detection results based on the confidence level of each sub-detection result in each total detection result, the method further includes:

[0021] Based on the margins between each feature point in each first feature map and its corresponding detection box, calculate the centrality of each feature point in each first feature map.

[0022] Multiply the centrality of each feature point by its corresponding detection result to obtain the confidence level for each feature point;

[0023] The confidence level of each sub-detection result in each total detection result is obtained based on the confidence level corresponding to each feature point in each first feature map.

[0024] By using the above method to calculate the confidence level corresponding to each sub-detection result, a basis can be provided for subsequently determining the sub-detection results to be calibrated.

[0025] In one possible embodiment, the step of selecting the sub-detection results to be calibrated corresponding to each target detection object from the total detection results based on the confidence level of each sub-detection result in each total detection result includes:

[0026] Based on the confidence level of each sub-detection result in each overall detection result, obtain the confidence level corresponding to each target detection object;

[0027] The confidence level of each target detection object is compared with the first threshold to obtain each first confidence level that is greater than or equal to the first threshold.

[0028] Each of the first confidence levels is compared with a second threshold to obtain a second confidence level that is less than the second threshold, wherein the second threshold is greater than the first threshold;

[0029] The sub-detection results of the target detection object corresponding to each of the second confidence levels are used as the sub-detection results to be calibrated.

[0030] The above method can identify sub-detection results to be calibrated from the current liveness detection results, thereby helping to improve the accuracy of liveness detection.

[0031] In one possible embodiment, calibrating each sub-detection result to be calibrated based on the positional relationship between each target detection object and its associated object includes:

[0032] Calculate the intersection of the detection bounding box of the target detection object corresponding to each detection result of the sub-calibrator and the detection bounding box of the object to obtain the first detection bounding box;

[0033] Compare the size of the detection box of the target detection object corresponding to each of the detection results of the sub-calibrators with the size of the detection box of the object, and take the smaller detection box as the second detection box;

[0034] Calculate the ratio between the first detection frame and the second detection frame;

[0035] If the ratio is equal to the set value, the detection object corresponding to the first detection box is determined to be a negative sample object; otherwise, the detection object corresponding to the first detection box is determined to be a positive sample object, thereby realizing the calibration of the detection result of the sub-calibrator.

[0036] Using the above method, the sub-detection results to be calibrated are calibrated based on the confidence levels of each sub-detection result previously calculated, thereby improving the accuracy of liveness detection.

[0037] Secondly, this application provides a liveness detection device, the device comprising:

[0038] The first determining module is used to determine at least one first feature map corresponding to the acquired target image;

[0039] The detection module is used to detect at least one detection object in each first feature map to obtain a total detection result corresponding to the first feature map; the total detection result includes sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects, and object objects associated with the sample objects;

[0040] The filtering module is used to filter out the sub-detection results to be calibrated corresponding to the target detection object from each total detection result based on the confidence level of each sub-detection result in each total detection result. The detection objects of the target detection object include positive sample objects and negative sample objects.

[0041] The calibration module is used to calibrate the detection results of each sub-detection based on the positional relationship between each target detection object and its associated object, and to obtain the total detection result after calibration.

[0042] The second determining module is used to determine the liveness attribute corresponding to the target image based on the calibrated total detection results.

[0043] In one possible embodiment, the detection module is specifically used for:

[0044] Acquire the target image;

[0045] The target image is used to extract features through a ResNet50 network to obtain various second feature maps.

[0046] The first feature maps are obtained by fusing and convolving the second feature maps using a Feature Pyramid Network (FPN).

[0047] In one possible embodiment, the filtering module is further configured to:

[0048] Based on the margins between each feature point in each first feature map and its corresponding detection box, calculate the centrality of each feature point in each first feature map.

[0049] Multiply the centrality of each feature point by its corresponding detection result to obtain the confidence level for each feature point;

[0050] The confidence level of each sub-detection result in each total detection result is obtained based on the confidence level corresponding to each feature point in each first feature map.

[0051] In one possible embodiment, the filtering module is further configured to:

[0052] Based on the confidence level of each sub-detection result in each overall detection result, obtain the confidence level corresponding to each target detection object;

[0053] The confidence level of each target detection object is compared with the first threshold to obtain each first confidence level that is greater than or equal to the first threshold.

[0054] Each of the first confidence levels is compared with a second threshold to obtain a second confidence level that is less than the second threshold, wherein the second threshold is greater than the first threshold;

[0055] The sub-detection results of the target detection object corresponding to each of the second confidence levels are used as the sub-detection results to be calibrated.

[0056] In one possible embodiment, the calibration module is specifically used for:

[0057] Calculate the intersection of the detection bounding box of the target detection object corresponding to each detection result of the sub-calibrator and the detection bounding box of the object to obtain the first detection bounding box;

[0058] Compare the size of the detection box of the target detection object corresponding to each of the detection results of the sub-calibrators with the size of the detection box of the object, and take the smaller detection box as the second detection box;

[0059] Calculate the ratio between the first detection frame and the second detection frame;

[0060] If the ratio is equal to the set value, the detection object corresponding to the first detection box is determined to be a negative sample object; otherwise, the detection object corresponding to the first detection box is determined to be a positive sample object, thereby realizing the calibration of the detection result of the sub-calibrator.

[0061] Thirdly, this application provides an electronic device, comprising:

[0062] Memory, used to store program instructions;

[0063] A processor is configured to invoke program instructions stored in the memory and execute the steps included in the liveness detection method described in any one of the first aspects according to the obtained program instructions.

[0064] Fourthly, this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the liveness detection method described in any one of the first aspects.

[0065] The beneficial effects of this application are as follows:

[0066] The liveness detection method, apparatus, computer device, and storage medium provided in this application, after acquiring each first feature map corresponding to the target image, detect at least one detection object in each first feature map to obtain a total detection result corresponding to each first feature map. The total detection result includes sub-detection results corresponding to at least one detection object, where each detection object includes positive sample objects, negative sample objects, and object objects. Thus, when detecting the target image, the sub-detection results corresponding to positive sample objects, negative sample objects, and object objects in the obtained total detection result reflect liveness classification, achieving simultaneous object detection and liveness classification. Compared to the traditional method of separating object detection and liveness classification, this effectively reduces the runtime consumption of the liveness detection system and improves its operating efficiency.

[0067] Furthermore, the confidence level of each sub-detection result in each total detection result is calculated, and based on each confidence level, the sub-detection results corresponding to the target detection object are selected from each total detection result. The target detection objects include positive and negative sample objects. Further, based on the positional relationship between the target detection object and the object corresponding to each sub-detection result to be calibrated, the calibrated total detection results corresponding to each first feature map are obtained. Finally, based on the calibrated total detection results, the liveness attribute corresponding to the target image is determined. In this way, the sub-detection results corresponding to the target detection object can be calibrated, thereby re-determining the liveness attribute of positive or negative samples and improving the accuracy of liveness detection. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 is a schematic diagram of a possible application scenario provided by an embodiment of this application;

[0070] Figure 2 is a flowchart of a liveness detection method provided in an embodiment of this application;

[0071] Figure 3 is a schematic diagram of the structure of an FCOS network provided in an embodiment of this application;

[0072] Figure 4 is an example diagram of the positional relationship of a detection frame provided in an embodiment of this application;

[0073] Figure 5 is a structural diagram of a liveness detection device provided in an embodiment of this application;

[0074] Figure 6 is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0076] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0077] The data collection, dissemination, and use in this application all comply with relevant national laws and regulations.

[0078] To facilitate understanding by those skilled in the art, some of the nouns and terms involved in the embodiments of this application will be briefly described and explained as follows:

[0079] Liveness detection: A method used in identity verification scenarios to determine the true physiological characteristics of an individual. In facial recognition applications, liveness detection verifies whether a user is a real, living person by detecting combinations of actions such as blinking, opening their mouth, shaking their head, and nodding, using technologies like facial landmark localization and face tracking. It effectively defends against common attacks using photos, videos, face swaps, masks, occlusions, 3D animations, and screen captures, thus helping users identify fraudulent activities and protecting their interests.

[0080] ResNet50 network: a convolutional neural network used for feature extraction.

[0081] FPN: Feature Pyramid Networks.

[0082] Backbone network: Chinese name: backbone network, is a high-speed network used to connect multiple regions or areas. Each backbone network has at least one connection point that can interconnect with other backbone networks.

[0083] Head network: A type of network that acquires the output of a network. The head uses features extracted by the previous network to make predictions.

[0084] Neck network: Located between the backbone network and the head network, it is designed to better utilize the features extracted by the backbone network.

[0085] Anchor-free: There is an object detection algorithm, and another object detection algorithm is anchor-based. The difference between the two is whether or not anchors are used to extract candidate detection boxes.

[0086] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0087] The technical methods provided in this application involve liveness detection and machine learning technologies. The FCOS network proposed in this application is an anchor-free detection model, including a Backbone network, a Neck network, and a Head network. The Backbone network is mainly used to extract feature maps from the acquired target image using a ResNet50 network, obtaining various second feature maps. The Neck network is mainly used to fuse and convolve the various second feature maps using an FPN, obtaining various first feature maps. The Head network is mainly used to perform target detection on the various first feature maps, obtaining a total detection result containing four categories. This total detection result includes four dimensions, including sub-detection results for live targets, fake targets, paper, and screen. Thus, the tasks of target detection and liveness classification are performed simultaneously.

[0088] It should be noted that the training and use of the aforementioned FCOS network are divided into two parts: a training part and an application part. The training part involves the fields of liveness detection and machine learning. In this part, the FCOS network uses liveness detection technology to train samples for object detection, obtaining total detection results including positive sample objects, negative sample objects, and object objects across different dimensions. Machine learning is then used to train the FCOS network, continuously adjusting model parameters through optimization algorithms until the model converges, obtaining the corresponding network parameters. The application part also involves liveness detection and machine learning. In this part, the FCOS network uses liveness detection technology to train samples for target detection, obtaining total detection results including positive sample objects, negative sample objects, and object objects across different dimensions. Based on machine learning, the FCOS network uses the network parameters obtained in the training part to perform liveness detection on the input target image, obtaining the total detection results for each first feature map corresponding to the target image. This total detection result includes sub-detection results across multiple dimensions, where each dimension includes at least positive sample objects, negative sample objects, and object objects. In addition, it should be noted that the artificial neural network model in the embodiments of this application can be trained online or offline, and no specific limitation is made here. This article uses offline training as an example for illustration.

[0089] After obtaining the overall detection results corresponding to the target image based on the applied FCOS model, the sub-detection results corresponding to the positive or negative samples whose confidence does not meet the requirements in each overall detection result are further calibrated, and the liveness attribute is judged based on the calibrated results, thereby improving the accuracy of liveness detection.

[0090] The design concept of the embodiments of this application is briefly introduced below:

[0091] In the field of liveness detection, common attack methods include screen attacks and photo attacks. For example, using a face displayed on a mobile phone screen or a face in a photograph to perform facial recognition can be used to attack the identity verification system. Related technologies typically employ two methods for liveness detection: deploying a face detection unit and a liveness detection unit. During deployment, a frame image is first input into the face detection unit. Once a face is detected in the frame image, the detected face data is input into the liveness detection unit for liveness determination.

[0092] The above approach, where face detection and liveness detection are developed and updated as two independent units, has several drawbacks in practical applications. First, the two independent modules perform data processing and forward inference separately, resulting in a longer overall running time. Second, the face detection unit, as a preceding unit to the liveness detection unit, exhibits performance differences across different scenarios, which can affect the quality of input data for the subsequent liveness detection unit, thereby impacting classification accuracy.

[0093] Therefore, the liveness detection method, apparatus, computer device, and storage medium provided in this application, after acquiring each first feature map corresponding to the target image, detect at least one detection object in each first feature map to obtain a total detection result corresponding to each first feature map. The total detection result includes sub-detection results corresponding to at least one detection object, where each detection object includes positive sample objects, negative sample objects, and object objects. Thus, when detecting the target image, the sub-detection results corresponding to positive sample objects, negative sample objects, and object objects in the obtained total detection result reflect liveness classification, thereby achieving simultaneous object detection and liveness classification. Compared to the traditional method of separating object detection and liveness classification, this effectively reduces the runtime consumption of the liveness detection system and improves its operating efficiency.

[0094] Furthermore, the confidence level of each sub-detection result in each total detection result is calculated, and based on each confidence level, the sub-detection results corresponding to the target detection object are selected from each total detection result. The target detection objects include positive and negative sample objects. Further, based on the positional relationship between the target detection object and the object corresponding to each sub-detection result to be calibrated, the calibrated total detection results corresponding to each first feature map are obtained. Finally, based on the calibrated total detection results, the liveness attribute corresponding to the target image is determined. In this way, the sub-detection results corresponding to the target detection object can be calibrated, thereby re-determining the liveness attribute of positive or negative samples and improving the accuracy of liveness detection.

[0095] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0096] Figure 1 illustrates an application scenario according to an embodiment of this application. In this scenario, the system may include an image acquisition device 101, terminal devices (102a, 102b), and a server 103. The image acquisition device 101 and the terminal devices (102a, 102b) interact with the server 103 via a communication network. The communication network may employ wireless communication or wired communication.

[0097] For example, the image acquisition device 101 and the target terminal (102a, 102b) can access the network and communicate with the server 103 through cellular mobile communication technology, including 5th Generation Mobile Networks (5G) technology.

[0098] For example, the image acquisition device 101 and the terminal devices (102a, 102b) can access the network and communicate with the server 103 through short-range wireless communication, which includes Wireless Fidelity (Wi-Fi) technology.

[0099] The image acquisition device 101 described above can be installed on the terminal devices (102a, 102b), or it can be installed separately from the terminal devices (102a, 102b).

[0100] This application embodiment does not impose any limitation on the number of the above-mentioned devices. As shown in Figure 1, only the terminal devices (102a, 102b) and the server 103 are described as examples. The following is a brief introduction to each of the above-mentioned devices and their respective functions.

[0101] Image acquisition device 101 is a device that can provide users with image acquisition and forwarding, including: cameras, cloud cameras or other terminal devices with image acquisition functions, used to acquire target images corresponding to various application scenarios.

[0102] For example, in access control scenarios, image acquisition device 101 is used to acquire facial images for identity verification; in warning and deportation application scenarios, image acquisition device 101 is used to acquire live objects within the warning range to issue a warning to deport them; in the field of surveillance, image acquisition device 101 is used to acquire the driving trajectory of target objects.

[0103] Terminal devices (102a, 102b) are devices that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connectivity, vehicle-mounted terminal devices, etc.

[0104] For example, terminal devices (102a, 102b) include, but are not limited to: mobile phones, tablets, laptops, handheld computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0105] Furthermore, the terminal devices (102a, 102b) may have a liveness detection-related client installed. This client can be software (e.g., an app, browser, short video app, etc.), or a webpage, mini-program, etc. In this embodiment, the user can use the aforementioned liveness detection-related client through the terminal devices (102a, 102b) and can interact with the server 103 to exchange information related to the liveness detection scenario.

[0106] Furthermore, server 103 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0107] Furthermore, in this embodiment, the server 103 may be equipped with a liveness detection service platform corresponding to the client. This liveness detection service platform is used to receive the target image acquired by the image acquisition device 101 and then perform liveness detection services. Service triggering includes automatic triggering and manual triggering.

[0108] The following describes the content recommendation method provided by exemplary embodiments of this application in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0109] Based on the above application scenarios, the liveness detection method provided in this application embodiment will be described and explained from the server side with reference to the accompanying drawings. As shown in Figure 2, a liveness detection method is provided in this application embodiment, which specifically includes the following steps:

[0110] S201, determine at least one first feature map corresponding to the acquired target image;

[0111] S202, for each first feature map, at least one detection object in the first feature map is detected to obtain the total detection result corresponding to the first feature map; the total detection result includes the sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects and object objects associated with the sample objects;

[0112] The overall detection result includes at least one sub-detection result corresponding to each detection object; at least one detection object includes positive sample objects, negative sample objects, and object objects; positive sample objects can be target faces, and negative sample objects can be fake faces; object objects include paper and screens.

[0113] In this embodiment of the application, after the server acquires the target image captured by the image acquisition device, it starts to call the trained FCOS network, thereby acquiring the first feature maps corresponding to the target image through the FCOS network, specifically including:

[0114] Acquire the target image;

[0115] Feature extraction of the target image is performed using a ResNet50 network to obtain various second feature maps;

[0116] The first feature maps are obtained by fusing and convolving the second feature maps using the Feature Pyramid Network (FPN).

[0117] Furthermore, the trained FCOS network is used to detect at least one object in each first feature map, and the total detection results corresponding to each first feature map are obtained.

[0118] As exemplarily shown in Figure 3, this application provides a schematic diagram of an anchor-free FCOS network structure, comprising three parts: a Backbone network, a Neck network, and a Head network. The Backbone network, after acquiring the target image, extracts features from the target image using a ResNet50 network, thereby outputting feature maps C1, C2, and C3.

[0119] Furthermore, the Neck network performs fusion convolutions on C1, C2, and C3 using the FPN network. Specifically, this includes convolving C3 to obtain feature map P3, convolving P3 to obtain feature map P4, and then convolving P4 to obtain P5. This part is a "bottom-up" convolution. Correspondingly, P3 is sampled to adjust the feature map to the same scale as C2, and then the result of convolving it with C2 is summed pixel by pixel to obtain feature map P2. Similarly, P2 is adjusted to the same scale as C1, and the result of convolving it with C1 is summed pixel by pixel to obtain feature map P1. This part is a "top-down" convolution process.

[0120] Furthermore, the Head network detects multiple objects in P1 to P3, which mainly includes two branches: regression analysis and object classification. The regression branch performs regression analysis on the bounding boxes of each feature map to obtain regression results. Object classification detects multiple objects in the target image, yielding detection results with four dimensions, each corresponding to a category of object: positive objects, negative objects, paper, and screen. This allows for simultaneous object detection and liveness classification. In addition, object classification obtains the centerness of each feature point in each feature map. The purpose of calculating centerness is to strengthen the central constraint and reduce the weight of bounding boxes far from the target center. The specific calculation formula is as follows:

[0121]

[0122] In formula (1), l, r, t, and b are the distances of the feature point relative to the four sides of the detection box, respectively. The centerness value is distributed between [0, 1], and a binary cross-entropy loss is used for optimization during training. The specific loss function is:

[0123]

[0124] In formula (2), l*, r*, t*, and b* represent the labeled results corresponding to l, r, t, and b, respectively; centerness * The result after labeling centerness; p * x,y For p x,y The labeled result, where p x,y The detection results for the feature points include four dimensions; t * x,y For t x,y The corresponding labeled result, where tx,y The regression results for the detection boxes corresponding to the feature points; N ops p represents the number of positive samples. x,y The confidence score of the classification result corresponding to the feature point, λ1 and λ2 represent the weight values, and L, respectively. cls FocalLoss classification loss function, L reg IOULOss is the regression loss function, and Lcenterness is the centerness loss, specifically the BCELOss function.

[0125] S203, based on the confidence level of each sub-detection result in each total detection result, select the sub-detection results to be calibrated for each target detection object in each total detection result;

[0126] The target detection objects include positive sample objects and negative sample objects.

[0127] After obtaining the overall detection results corresponding to each first feature map through the FCOS network, the confidence of the sub-detection results corresponding to each target object in each overall detection result is further determined. The specific calculation method is as follows:

[0128] The confidence score for each sub-detection result is obtained by multiplying the sub-detection result in the total detection result branch of FCOS by the regression result of the detection box of the corresponding detection object. The higher the confidence score, the more accurate the classification of the detection object corresponding to the sub-detection result.

[0129] For example, the total detection result corresponding to each first feature map is a sub-detection result containing four dimensions, namely A1, A2, A3 and A4. The regression results of the detection boxes of the detection objects corresponding to the four sub-detection results are t1, t2, t3 and t4 respectively. Then, the confidence scores of these four sub-detection results are A1×t1, A2×t2, A3×t3 and A4×t4 respectively.

[0130] Furthermore, based on the confidence level corresponding to each sub-detection result, the sub-detection results to be calibrated corresponding to the target detection object are selected from each total detection result. Specifically:

[0131] The confidence scores of each positive or negative sample object are compared with the first threshold to obtain the first confidence scores that are greater than or equal to the first threshold.

[0132] For example, if the sub-detection results for the four dimensions are A1, A2, A3, and A4, where A1 corresponds to a positive sample object, A2 corresponds to a negative sample object, A3 corresponds to a paper object, and A4 corresponds to a screen object, then A1 and A2 are compared with a first threshold. If A1 or A2 is less than the first threshold, the detection result corresponding to A1 or A2 is considered invalid; if A1 or A2 is greater than or equal to the first threshold, then A1 or A2 is placed in the first queue.

[0133] Furthermore, each first confidence level in the first queue is compared with a second threshold to obtain each second confidence level that is less than the second threshold, wherein the second threshold is greater than the first threshold.

[0134] For example, in the first queue, there are sub-detection results A1 and A2. A1 corresponds to a positive sample object, and A2 corresponds to a negative sample object. A1 and A2 are compared with a second threshold. If A1 or A2 is greater than or equal to the second threshold, then the confidence level of A1 or A2 meets the requirement, meaning the sample object corresponding to A1 is a positive sample object, or the sample object corresponding to A2 is a negative sample object. However, if A1 or A2 is less than the second threshold, then the detection results corresponding to A1 or A2 need further calibration. In this case, A1 or A2 represents each second confidence level less than or equal to the second threshold, and the sub-detection results corresponding to the positive or negative sample objects for each second confidence level are the sub-detection results to be calibrated.

[0135] The above method can be used to screen out sub-detection results that need further calibration from the current liveness detection results, thereby helping to improve the accuracy of liveness detection.

[0136] S204. Based on the positional relationship between each target detection object and its associated object, the detection results of each sub-detection to be calibrated are calibrated to obtain the calibrated total detection result.

[0137] After detecting the results of each sub-sub ...

[0138] The intersection of the detection bounding box of the target detection object corresponding to each detection result of the sub-calibrator and the detection bounding box of the object is calculated to obtain the first detection bounding box. In this embodiment, the object includes paper and / or screen. As shown in Figure 4, P is the detection bounding box corresponding to the positive sample object or the negative sample object, Q1 is the detection bounding box corresponding to the paper, and Q2 is the detection bounding box corresponding to the screen. Q1 and Q2 are both detection bounding boxes of the object. Then, the intersection between P and the detection bounding boxes of the object includes Z1 and Z2, and Z1 and Z2 are both the first detection bounding boxes.

[0139] The detection bounding boxes of the target objects corresponding to each detection result of the calibrator are compared with the detection bounding boxes of the object objects, and the smaller detection bounding box is used as the second detection bounding box. As shown in Figure 4, the size of the object object detection bounding box is Z1 or Z2. If the area of ​​detection bounding box P is less than the area of ​​Z1 or Z2, then detection bounding box P is used as the second detection bounding box. If the area of ​​P is greater than or equal to the area of ​​Z1 or Z2, then Z1 or Z2 is used as the second detection bounding box.

[0140] The ratio between the first and second detection boxes is calculated. If the ratio equals a set value, the object corresponding to the first detection box is determined to be a negative sample object; otherwise, it is determined to be a positive sample object, thus calibrating the sub-detection result to be calibrated. The set value is 1. If the ratio between the first and second detection boxes is 1, it indicates that the object's detection box contains the first detection box, meaning the object corresponding to the first detection box is a negative sample object. In this case, if the object corresponding to the first detection box is itself a negative sample object, no modification to the sub-detection result corresponding to the first detection box is needed. If the object corresponding to the first detection box is a positive sample object, it needs to be classified as a negative sample object, and the sub-detection result corresponding to the first detection box needs adjustment.

[0141] S205, based on the calibrated total detection results, determine the liveness attribute corresponding to the target image.

[0142] After calibrating the sub-detection results corresponding to the positive and negative sample objects respectively, the total detection results corresponding to each first feature map after calibration are obtained. Then, by fusing and analyzing the total detection results, the liveness attribute corresponding to the target image is determined.

[0143] Based on the above-described liveness detection method, after obtaining the first feature maps corresponding to the target image, at least one detection object in each first feature map is detected to obtain the total detection result corresponding to each first feature map. The total detection result includes sub-detection results corresponding to at least one detection object, which includes positive sample objects, negative sample objects, and object objects. Thus, when detecting the target image, the sub-detection results corresponding to positive sample objects, negative sample objects, and object objects in the obtained total detection result reflect liveness classification, achieving simultaneous object detection and liveness classification. Compared to the traditional method of separating object detection and liveness classification, this effectively reduces the runtime overhead of the liveness detection system and improves its operational efficiency.

[0144] Furthermore, the confidence level of each sub-detection result in each total detection result is calculated, and based on each confidence level, the sub-detection results corresponding to the target detection object are selected from each total detection result. The target detection objects include positive and negative sample objects. Further, based on the positional relationship between the target detection object and the object corresponding to each sub-detection result to be calibrated, the calibrated total detection results corresponding to each first feature map are obtained. Finally, based on the calibrated total detection results, the liveness attribute corresponding to the target image is determined. In this way, the sub-detection results corresponding to the target detection object can be calibrated, thereby re-determining the liveness attribute of positive or negative samples and improving the accuracy of liveness detection.

[0145] Based on the same inventive concept, this application provides a liveness detection device. Referring to Figure 5, the device includes:

[0146] The first determining module 501 is used to determine at least one first feature map corresponding to the acquired target image;

[0147] The detection module 502 is used to detect at least one detection object in each first feature map to obtain a total detection result corresponding to the first feature map; the total detection result includes sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects and object objects associated with the sample objects;

[0148] The filtering module 503 is used to filter out the sub-detection results to be calibrated for each target detection object from each total detection result based on the confidence level of each sub-detection result in each total detection result;

[0149] The calibration module 504 is used to calibrate each sub-detection result to be calibrated according to the positional relationship between each target detection object and the associated object, so as to obtain the calibrated total detection result, wherein the object includes;

[0150] The second determining module 505 is used to determine the liveness attribute corresponding to the target image based on the calibrated total detection results.

[0151] In one possible embodiment, the detection module 502 is specifically used for:

[0152] Acquire the target image;

[0153] The target image is used to extract features through a ResNet50 network to obtain various second feature maps.

[0154] The first feature maps are obtained by fusing and convolving the second feature maps using a Feature Pyramid Network (FPN).

[0155] In one possible embodiment, the filtering module 503 is further configured to:

[0156] Based on the margins between each feature point in each first feature map and its corresponding detection box, calculate the centrality of each feature point in each first feature map.

[0157] Multiply the centrality of each feature point by its corresponding detection result to obtain the confidence level for each feature point;

[0158] The confidence level of each sub-detection result in each total detection result is obtained based on the confidence level corresponding to each feature point in each first feature map.

[0159] In one possible embodiment, the filtering module 503 is further configured to:

[0160] Based on the confidence level of each sub-detection result in each overall detection result, obtain the confidence level corresponding to each target detection object;

[0161] The confidence level of each target detection object is compared with the first threshold to obtain each first confidence level that is greater than or equal to the first threshold.

[0162] Each of the first confidence levels is compared with a second threshold to obtain a second confidence level that is less than the second threshold, wherein the second threshold is greater than the first threshold;

[0163] The sub-detection results of the target detection object corresponding to each of the second confidence levels are used as the sub-detection results to be calibrated.

[0164] In one possible embodiment, the calibration module 504 is specifically used for:

[0165] Calculate the intersection of the detection bounding box of the target detection object corresponding to each detection result of the sub-calibrator and the detection bounding box of the object to obtain the first detection bounding box;

[0166] Compare the size of the detection box of the target detection object corresponding to each of the detection results of the sub-calibrators with the size of the detection box of the object, and take the smaller detection box as the second detection box;

[0167] Calculate the ratio between the first detection frame and the second detection frame;

[0168] If the ratio is equal to the set value, the detection object corresponding to the first detection box is determined to be a negative sample object; otherwise, the detection object corresponding to the first detection box is determined to be a positive sample object, thereby realizing the calibration of the detection result of the sub-calibrator.

[0169] Based on the aforementioned liveness detection device, after acquiring the first feature maps corresponding to the target image, target object detection is performed on each first feature map. Specifically, the detected target objects include live targets, fake targets, paper, and screens, thereby obtaining the total detection result corresponding to each first feature map. The total detection result includes the classification result corresponding to each target object. In this way, target detection and liveness classification are performed simultaneously. Compared to the traditional method of separating target detection and liveness detection, this effectively reduces the runtime consumption of the liveness detection system and improves its operating efficiency.

[0170] Furthermore, the confidence level of each sub-detection result in each total detection result is calculated. Based on each confidence level, sub-detection results corresponding to live targets or false targets are selected from each total detection result to be calibrated. According to the positional relationship between the live target or false target and the object corresponding to each sub-detection result to be calibrated, the calibrated total detection results corresponding to each first feature map are obtained. The object includes paper and / or a screen. Based on the calibrated total detection results, the liveness attribute corresponding to the target image is determined. In this way, further calibration of facial liveness attributes can be achieved, improving the accuracy of liveness detection.

[0171] Based on the same inventive concept, this application also provides an electronic device that can perform the functions of the aforementioned liveness detection method device. Referring to FIG6, the electronic device includes:

[0172] At least one processor 601 and a memory 602 connected to at least one processor 601 are included. In this embodiment, the specific connection medium between the processor 601 and the memory 602 is not limited. Figure 6 illustrates an example where the processor 601 and the memory 602 are connected via a bus 600. The bus 600 is represented by a thick line in Figure 6. The connection methods between other components are for illustrative purposes only and are not intended to be limiting. The bus 600 can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in Figure 6, but this does not indicate that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller; the name is not limited.

[0173] In this embodiment, the memory 602 stores instructions executable by at least one processor 601. By executing the instructions stored in the memory 602, the at least one processor 601 can perform the liveness detection method described above. The processor 601 can implement the functions of each module in the device shown in FIG5.

[0174] The processor 601 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 602 and calling data stored in memory 602, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0175] In one possible design, processor 601 may include one or more processing units. Processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 601. In some embodiments, processor 601 and memory 602 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0176] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the liveness detection method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0177] Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 602 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 602 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0178] By designing and programming the processor 601, the code corresponding to the liveness detection method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the liveness detection method shown in Figure 2 during operation. How to design and program the processor 601 is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0179] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium. The computer program product includes computer program code, which, when executed on a computer, causes the computer to perform any of the liveness detection methods discussed above. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to that of the liveness detection method, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0180] Based on the same inventive concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the liveness detection methods discussed above. Since the principle by which the above-described computer program product solves the problem is similar to that of the liveness detection method, the implementation of the above-described computer program product can be referred to the implementation of the method, and repeated details will not be described again.

[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable liveness detection device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable liveness detection device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable liveness detection device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0184] These computer program instructions may also be loaded onto a computer or other programmable liveness detection device to cause a series of user-operated steps to be performed on the computer or other programmable device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0185] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting liveness, characterized in that, The method includes: determining at least one first feature map corresponding to the acquired target image; detecting at least one detection object in each first feature map to obtain a total detection result corresponding to the first feature map; the total detection result includes sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects, and object objects associated with the sample objects; wherein, the object objects include paper and / or screen; based on the confidence level of each sub-detection result in each total detection result, selecting sub-detection results to be calibrated corresponding to each target detection object from the total detection results; the target detection objects include the positive sample objects and the negative sample objects; calibrating each sub-detection result to be calibrated according to the positional relationship between each target detection object and the associated object object to obtain a calibrated total detection result. The step of calibrating each sub-detection result to be calibrated based on the positional relationship between each target detection object and the associated object includes: calculating the intersection between the detection box of the target detection object corresponding to each sub-detection result to be calibrated and the detection box of the object to obtain a first detection box; comparing the size between the detection box of the target detection object corresponding to each sub-detection result to be calibrated and the detection box of the object to be calibrated, and taking the smaller detection box as the second detection box; calculating the ratio between the first detection box and the second detection box; if the ratio is equal to a set value, then determining that the detection object corresponding to the first detection box is a negative sample object, otherwise, determining that the detection object corresponding to the first detection box is a positive sample object, thereby realizing the calibration of the sub-detection results to be calibrated; and determining the liveness attribute corresponding to the target image based on the calibrated total detection results.

2. The method as described in claim 1, characterized in that, Before selecting the sub-detection results to be calibrated for each target detection object from the total detection results, the method further includes: calculating the centrality of each feature point in each first feature map based on the margins between each feature point and its corresponding detection box; multiplying the centrality of each feature point by its corresponding detection result to obtain the confidence of each feature point; and obtaining the confidence of each sub-detection result in the total detection results based on the confidence of each feature point in each first feature map.

3. The method as described in claim 1, characterized in that, The step of selecting the sub-detection results to be calibrated for each target detection object from the total detection results based on the confidence level of each sub-detection result in each total detection result includes: obtaining the confidence level corresponding to each target detection object based on the confidence level of each sub-detection result in each total detection result; comparing the confidence level corresponding to each target detection object with a first threshold to obtain a first confidence level greater than or equal to the first threshold; comparing each first confidence level with a second threshold to obtain a second confidence level less than the second threshold, wherein the second threshold is greater than the first threshold; and using the sub-detection results of the target detection objects corresponding to each second confidence level as the sub-detection results to be calibrated.

4. A liveness detection device, characterized in that, The apparatus includes: a first determining module, configured to determine at least one first feature map corresponding to an acquired target image; a detection module, configured to detect at least one detection object in each first feature map to obtain a total detection result corresponding to the first feature map; the total detection result includes sub-detection results corresponding to each of the at least one detection object; the at least one detection object includes positive sample objects, negative sample objects, and object objects associated with the sample objects; wherein, the object objects include paper and / or a screen; a filtering module, configured to filter out sub-detection results to be calibrated corresponding to the target detection object from each total detection result based on the confidence level of each sub-detection result in each total detection result; the target detection object includes the positive sample objects and the negative sample objects; and a calibration module, configured to calibrate each sub-detection result to be calibrated according to the positional relationship between each target detection object and the associated object object to obtain calibrated results. The overall detection result; wherein, when the calibration module calibrates each sub-detection result to be calibrated according to the positional relationship between each target detection object and the associated object, it is specifically used to calculate the intersection between the detection box of the target detection object corresponding to each sub-detection result to be calibrated and the detection box of the object to obtain a first detection box; compare the size between the detection box of the target detection object corresponding to each sub-detection result to be calibrated and the detection box of the object to be calibrated, and take the smaller detection box as the second detection box; calculate the ratio between the first detection box and the second detection box; if the ratio is equal to a set value, then determine that the detection object corresponding to the first detection box is a negative sample object, otherwise, determine that the detection object corresponding to the first detection box is a positive sample object, thereby realizing the calibration of the sub-detection result to be calibrated; the second determination module is used to determine the liveness attribute corresponding to the target image according to the calibrated overall detection results; 5. The apparatus as described in claim 4, characterized in that, The filtering module is further configured to: calculate the centrality of each feature point in each first feature map based on the margins between each feature point in each first feature map and its corresponding detection box; multiply the centrality of each feature point by its corresponding detection result to obtain the confidence level of each feature point; and obtain the confidence level of each sub-detection result in each total detection result based on the confidence level of each feature point in each first feature map.

6. The apparatus as claimed in claim 4, characterized in that, The filtering module is further configured to: obtain the confidence level corresponding to each target detection object based on the confidence level of each sub-detection result in each total detection result; compare the confidence level corresponding to each target detection object with a first threshold to obtain each first confidence level greater than or equal to the first threshold; compare each first confidence level with a second threshold to obtain each second confidence level less than the second threshold, wherein the second threshold is greater than the first threshold; and use the sub-detection result of the target detection object corresponding to each second confidence level as the sub-detection result to be calibrated.

7. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1-3 in accordance with the obtained program instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-3.

Images