Target detection method and device, electronic terminal and computer readable storage medium
By jointly training and feature fusion of the liveness detection network, the problem of low accuracy in liveness detection is solved, and effective defense against paper and screen attacks is achieved.
Patent Information
- Application Number
- CN202510875029.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-11-11
AI Technical Summary
Current technologies have low accuracy in detecting live targets, making it difficult to effectively defend against attacks on paper printing and mobile device displays.
A liveness detection network is employed, which improves detection accuracy by jointly training a texture feature extractor, a depth feature extractor, and a classifier, and by fusing depth and texture features.
It improves the accuracy of liveness detection, effectively identifies live and non-live targets, and enhances the defense against paper and screen attacks.
Smart Images
Figure CN120932313A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image technology, and in particular to a target detection method, apparatus, electronic terminal, and computer-readable storage medium. Background Technology
[0002] The widespread application of target recognition technology in the digital society has brought convenience to everyone's lives, but it has also brought challenges to target information security. Liveness detection technology has emerged to address this challenge, providing protection for target information security. In the most widespread applications, the most common information spoofing attacks come from paper printing and mobile device display attacks. These attack methods are inexpensive, but given the numerous characteristics of paper and ink, as well as the potential attacks from a large number of mobile device models, defending against attacks using a single frame of color image captured by a single color camera is a challenging task. Summary of the Invention
[0003] The main technical problem addressed by this application is to provide a target detection method, device, electronic terminal, and computer-readable storage medium, thereby solving the problem of low accuracy in live target detection in the prior art.
[0004] To solve the above-mentioned technical problems, the first technical solution adopted in this application is: to provide a target detection method, the target detection method comprising:
[0005] Acquire the image to be detected containing the target to be identified;
[0006] The image to be detected is input into the trained liveness detection network, which outputs the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier.
[0007] When training the deep feature extractor, the deep feature extractor, texture feature extractor, and classifier are trained using sample images containing the target; the sample images have reference feature maps; the target has category labels and domain labels; the category labels include live or non-live categories; the domain labels include one of live, paper attack, screen attack, and mask attack.
[0008] The process involves inputting the image to be detected into the trained liveness detection network, which then outputs the liveness detection results for the target to be identified, including:
[0009] Perform target detection on the image to be detected to obtain a target detection box containing the target to be identified;
[0010] The depth features of the image to be detected are obtained by using a depth feature extractor to extract features from the image to be detected.
[0011] The texture features of the target to be identified are obtained by extracting features from the target detection box containing the target to be identified using a texture feature extractor.
[0012] The depth features of the image to be detected and the texture features of the target to be identified are fused to obtain the fused features of the target to be identified.
[0013] The classifier identifies the fused features of the target to be identified, and obtains the liveness detection result of the target; the liveness detection result includes live or non-live objects.
[0014] The training methods for the liveness detection network include:
[0015] The sample image is subjected to feature extraction by a depth feature extractor to obtain the predicted depth feature map of the sample image;
[0016] Texture extraction is performed on the target in the sample image using a texture feature extractor to obtain the predicted texture feature map of the target and the probability of the target belonging to each domain category; the domain categories include living creature, paper attack, screen attack and mask attack;
[0017] The predicted depth feature map and the predicted texture feature map are fused to obtain the predicted fused feature map.
[0018] The predicted category information of the target is obtained by identifying the predicted fusion feature map through a classifier;
[0019] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0020] The training methods for the liveness detection network also include:
[0021] Deep features are extracted from the sample images to obtain the semantic feature maps of the sample images;
[0022] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label. This includes:
[0023] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the semantic feature map and the predicted deep feature map, the loss value between the probability of the target belonging to each domain category and the domain label, and the loss value between the predicted category information and the category label.
[0024] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0025] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the weighted sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each domain category and the domain label, and the loss value between the predicted category information and the category label.
[0026] The deep feature extractor includes multiple cascaded feature extraction layers; the liveness detection network also includes a depth estimator.
[0027] The sample image is processed by a depth feature extractor to obtain a predicted depth feature map of the sample image, including:
[0028] The sample image is input into the deep feature extractor to obtain the predicted feature map output by each feature extraction layer;
[0029] The depth estimator fuses the features of each predicted feature map corresponding to the sample image to obtain the predicted depth feature map of the sample image.
[0030] The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label. This includes:
[0031] The depth feature extractor, depth estimator, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted depth feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0032] Remove the depth estimator from the trained liveness detection network.
[0033] The process involves fusing the depth features of the image to be detected and the texture features of the target to be identified to obtain the fused features of the target, including:
[0034] The texture features corresponding to the image to be detected and the depth features of the image to be detected are superimposed to obtain the fused features of the target to be identified.
[0035] To solve the above-mentioned technical problems, the second technical solution adopted in this application is: to provide a target detection device, the target detection device comprising:
[0036] The acquisition module is used to acquire the image to be detected containing the target to be identified;
[0037] The detection module is used to input the image to be detected into the trained liveness detection network and output the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. Among them, when training the depth feature extractor, the depth feature extractor, texture feature extractor, and classifier are trained using sample images containing the target. The sample images have reference feature maps. The target has a category label and a domain label. The category label includes a liveness category or a non-liveness category. The domain label includes one of the following: live object, paper attack, screen attack, and mask attack.
[0038] To solve the above-mentioned technical problems, the third technical solution adopted in this application is: to provide an electronic terminal, which includes a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory, and the processor being used to execute program data to implement the steps in the above-mentioned target detection method.
[0039] To solve the above-mentioned technical problems, the fourth technical solution adopted in this application is: to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the target detection method described above.
[0040] The beneficial effects of this application are as follows: Unlike existing technologies, this application provides a target detection method, apparatus, electronic terminal, and computer-readable storage medium. The target detection method includes: acquiring a target image to be detected containing the target to be identified; inputting the target image into a trained liveness detection network; and outputting the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. Specifically, when training the depth feature extractor, the depth feature extractor, texture feature extractor, and classifier are trained using sample images containing the target. The sample images have reference feature maps; the target has category labels and domain labels; the category labels include liveness categories or non-liveness categories; and the domain labels include one of liveness, paper attack, screen attack, and mask attack. This application improves the detection accuracy of the liveness detection network by training it with sample images containing domain labels and category labels; and improves the accuracy of the target liveness detection result by using the liveness detection network to perform liveness detection on the target to be identified in the target image. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 This is a flowchart illustrating a specific embodiment of the target detection method provided in this application;
[0043] Figure 2 yes Figure 1 A flowchart illustrating a specific embodiment of the liveness detection network training method provided in the document;
[0044] Figure 3 yes Figure 1 A flowchart illustrating a specific embodiment of step S2 in the target detection method provided in the document;
[0045] Figure 4 This is a schematic diagram of the framework of an embodiment of the target detection device provided in this application;
[0046] Figure 5 This is a schematic diagram of the framework of an embodiment of the electronic terminal provided in this application;
[0047] Figure 6 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0048] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0049] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.
[0050] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.
[0051] To enable those skilled in the art to better understand the technical solution of this application, the target detection method provided in this application will be described in further detail below with reference to the accompanying drawings and specific embodiments.
[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0053] The target detection method provided in this application can be implemented by a server or terminal alone, or by a server and terminal working together. In some embodiments, the terminal or server can implement the target detection method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports virtual scenes, such as a game APP; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.
[0054] The target detection method provided in this application embodiment is illustrated below using a server implementation as an example.
[0055] Please see Figure 1 , Figure 1 This is a flowchart illustrating a specific embodiment of the target detection method provided in this application.
[0056] This embodiment provides a target detection method applicable to live target detection based on a single image of the target to be identified, and also applicable to live target detection based on at least two images of the target to be identified. The target detection method includes the following steps.
[0057] S1: Obtain the image to be detected containing the target to be identified;
[0058] S2: Input the image to be detected into the trained liveness detection network and output the liveness detection result of the target to be identified; the liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier; wherein, when training the depth feature extractor, the depth feature extractor, texture feature extractor, and classifier are trained using sample images containing the target; the sample images have reference feature maps; the target has a category label and a domain label; the category label includes the liveness category or the non-liveness category; the domain label includes one of the following: live object, paper attack, screen attack, and mask attack.
[0059] In this embodiment, the liveness detection network is trained using sample images with domain and category labels to improve its detection accuracy; the liveness detection network is then used to perform liveness detection on the target to be identified in the image to be detected, thereby improving the accuracy of the liveness detection results.
[0060] Specifically, the specific implementation method for obtaining the image to be detected containing the target to be identified in step S1 is as follows.
[0061] In one embodiment, a detection image containing the target to be identified is obtained by acquiring an image of the target to be identified using a single image acquisition device. The detection image can be an RGB image or an infrared image. The target to be identified can be a living object, a target in paper, a target on a screen, a 2D mask, a 3D mask, etc. The detection image can contain one target to be identified or multiple targets to be identified.
[0062] In one embodiment, by using multiple image acquisition devices to acquire images of the target to be identified, multiple images of the target to be identified can be fused together to obtain a detection image containing the target to be identified.
[0063] Specifically, the implementation method for inputting the image to be detected into the trained liveness detection network in step S2 and outputting the liveness detection result of the target to be identified is as follows.
[0064] Specifically, the liveness detection network needs to be trained in advance. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier.
[0065] Specifically, a deep feature extractor, a texture feature extractor, and a classifier are trained using sample images containing the target; the target has a category label and a domain label; the category label includes a live object category or a non-live object category; the domain label includes one of the following: live object, paper attack, screen attack, and mask attack.
[0066] Please see Figure 2 , Figure 2yes Figure 1 The flowchart of a specific embodiment of the liveness detection network training method provided in the document is shown in the figure.
[0067] In one embodiment, training a depth feature extractor, a texture feature extractor, and a classifier using sample images containing the target specifically includes the following steps.
[0068] S201: Obtain a sample image containing the target, wherein the target in the sample image has a category label and a domain label; the category label includes a live object category or a non-live object category; the domain label includes one of the following: live object, paper attack, screen attack, and masked attack.
[0069] Specifically, target detection is performed on the sample image to obtain a detection box containing the target.
[0070] In one embodiment, sample images can be preprocessed to improve the stability of the liveness detection network. Preprocessing of sample images includes image content enhancement and cropping / scaling. Image content enhancement includes, but is not limited to, adding various types of noise, such as Gaussian noise, Poisson noise, etc., and adjusting the image's colors. The cropping / scaling method for the sample images is the same as that for the images to be detected, and will not be described further here.
[0071] S202: Extract features from the sample image using a depth feature extractor to obtain the predicted depth feature map of the sample image.
[0072] The deep feature extractor consists of multiple cascaded feature extraction layers.
[0073] In one embodiment, a sample image is input into a depth feature extractor, and features are extracted from the sample image sequentially through each feature extraction layer in the depth feature extractor. The predicted feature map output by the last feature extraction layer is then used as the predicted depth feature map.
[0074] In one embodiment, the liveness detection network further includes a depth estimator. A sample image is input into a depth feature extractor, where each feature extraction layer sequentially extracts features from the sample image, resulting in a predicted feature map output by each layer. The depth estimator then fuses these predicted feature maps to obtain a predicted depth feature map of the sample image.
[0075] Attacks targeting paper printing and screen display can often detect fake backgrounds that do not match the real background. Under realistic forgery attacks, these backgrounds can be extremely realistic, making it difficult for the model to make a judgment at times, thus misjudging these fake backgrounds and leading to misjudgment.
[0076] To improve the feature extraction accuracy of the depth feature extractor, the depth features of the sample image can be supervised based on the depth features of the sample image.
[0077] Specifically, depth features are extracted from the sample image to obtain its semantic feature map. In one embodiment, the DINOv2 network can be used to extract depth features from the sample image to obtain its semantic feature map. By aligning the predicted depth feature map with the semantic feature map of the sample image, the predicted depth feature map is constrained to contain correct background semantic information. This improves the accuracy of depth estimation and enhances the defensive capability against attacks with false backgrounds.
[0078] S203: Extract texture from the target in the sample image using a texture feature extractor to obtain the predicted texture feature map of the target and the probability of the target belonging to each domain category; the domain categories include living creature, paper attack, screen attack and mask attack.
[0079] Specifically, a texture feature extractor is used to extract the texture of the target within the detection box, resulting in a predicted texture feature map of the target. Based on the predicted texture features of the target, the probability of the target belonging to each domain category is predicted.
[0080] S204: Perform feature fusion on the predicted depth feature map and the predicted texture feature map to obtain a fused feature map.
[0081] Specifically, the texture features and depth features corresponding to the image to be detected can be superimposed to obtain the fused features of the target to be identified; or the texture features and depth features corresponding to the image to be detected can be spliced together to obtain the fused features of the target to be identified.
[0082] S205: The fused feature map is identified by a classifier to obtain the predicted category information of the target.
[0083] Specifically, a classifier identifies the fused feature map to obtain the predicted category and probability value of the target. The predicted category includes two types: live and non-live.
[0084] S206: The deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0085] In one embodiment, a deep feature extractor is trained based on the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, and the loss values between the deep feature map and the predicted deep feature map. A texture feature extractor is trained based on the loss values between the probability of the target belonging to each domain category and the domain label. A classifier is trained based on the loss values between the predicted category information and the category label, thereby achieving separate training of the deep feature extractor, texture feature extractor, and classifier.
[0086] Specifically, the loss value L between the predicted depth feature map and the reference feature map corresponding to the sample image is calculated using the following formula. l .
[0087]
[0088] ρ(d p ,d g )=|d p -d g | (Formula 2)
[0089]
[0090] In the formula: t(d) represents the median depth value of the predicted depth feature map; s(d) represents the maximum value minus the minimum value of the predicted depth feature map.
[0091] Specifically, the loss value L between the depth feature map corresponding to the sample image and the predicted depth feature map is calculated using the following formula. f .
[0092]
[0093] In the formula: F s Fd represents the depth feature map; Fd represents the predicted depth feature map.
[0094] Specifically, the loss value L between the probability of the target belonging to each domain category and the domain label is calculated using the following formula. d .
[0095]
[0096] In the formula: x represents the image in the detection box; G represents the texture feature extractor; and D represents the domain discriminator.
[0097] Specifically, the loss value L between the predicted category information and the category label is calculated using the following formula. c .
[0098]
[0099] In the formula: yi Indicates the predicted category; p i This represents the probability value.
[0100] In one embodiment, the deep feature extractor, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the deep feature map and the predicted deep feature map, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0101] In one specific embodiment, in order to enable the deep feature extractor to extract more accurate and richer deep feature maps from sample images with false backgrounds, the deep feature extractor, texture feature extractor, and classifier are jointly trained based on the weighted sum of the loss values between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label.
[0102] Specifically, the final loss value L is calculated using the following formula: a weighted sum of the loss values between the predicted deep feature map and the reference feature map, the loss values between the deep feature map and the predicted deep feature map, the loss value between the probability of the target belonging to each domain category and the domain label, and the loss value between the predicted category information and the category label.
[0103] L=λcLc+λdLd+λ f L f +λlLl (Formula 7)
[0104] In one embodiment, λ c =1,λ f =0.1, λ l =0.5. In one embodiment, when the domain label of the target in the sample image is paper attack or screen attack, λ d =2,λ f =0.1, when the domain label of the target in the sample image is a living object. d =1,λ f =0.05, etc., thus enabling more accurate extraction of depth and foreground / background features even with false backgrounds.
[0105] The depth feature extractor, depth estimator, texture feature extractor, and classifier are jointly trained based on the sum of the loss values between the predicted depth feature map and the reference feature map corresponding to the sample image, the loss values between the probability of the target belonging to each domain category and the domain label, and the loss values between the predicted category information and the category label; the depth estimator is then removed from the trained liveness detection network.
[0106] In one specific embodiment, the image to be detected can be input into a liveness detection network; the liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. The texture feature extractor extracts texture features from the target within the target detection box to obtain the texture features of the target; the depth feature extractor extracts features from the image to be detected to obtain the depth features of the image. The classifier identifies the fused features of the target to be detected to obtain the liveness detection result; the liveness detection result includes whether the target is live or not.
[0107] Please see Figure 3 , Figure 3 yes Figure 1 The flowchart of a specific embodiment of step S2 in the target detection method provided is shown in the figure.
[0108] In one embodiment, the liveness detection network trained through the above steps performs target detection on the image to be detected.
[0109] S211: Perform target detection on the image to be detected to obtain a target detection box containing the target to be identified.
[0110] Specifically, an object detection model is used to detect objects in the image to be detected, resulting in object detection boxes containing the objects to be identified. These object detection boxes may contain only the feature regions of the objects to be identified.
[0111] To facilitate detection, the target detection bounding box can be preprocessed. For example, preprocessing may include cropping and scaling. Specifically, when the target detection bounding box is a rectangular area, the image within it can be expanded to a square area to obtain the complete feature region of the target to be identified, and then the image within the square target detection bounding box can be extracted. To facilitate detection, the extracted image within the target detection bounding box is scaled to a preset size. The preset size can be 112*112, and the specific size can be set according to the actual situation.
[0112] To improve image processing speed, the image to be detected can be preprocessed. Specifically, the size of the image to be detected is adjusted to a set size. This set size can be 224*224, but the specific size should be determined based on the actual situation.
[0113] S212: The depth features of the image to be detected are obtained by using a depth feature extractor to extract features from the image to be detected.
[0114] Specifically, feature extraction algorithms are used to extract features from the image to be detected, thereby obtaining the depth features of the image. Feature extraction algorithms can be VGG (Visual Geometry Group), CNN (Convolutional Neural Networks), etc.
[0115] S213: Extract features from the target detection box containing the target to be identified using a texture feature extractor to obtain the texture features of the target to be identified.
[0116] In one embodiment, texture features are extracted from the feature regions of the target to be identified within the target detection bounding box using a texture feature algorithm to obtain the texture features of the target. The texture feature algorithm can be the LBP method, gray-level co-occurrence matrix, etc. The LBP method (Local Binary Patterns) is a method used in computer vision for image feature classification.
[0117] S214: Perform feature fusion on the depth features of the image to be detected and the texture features of the target to be identified to obtain the fused features of the target to be identified.
[0118] In one embodiment, texture features and depth features corresponding to the image to be detected can be superimposed to obtain fused features of the target to be identified.
[0119] In one embodiment, the texture features and depth features corresponding to the image to be detected can be stitched together to obtain the fused features of the target to be identified.
[0120] S215: The classifier identifies the fused features of the target to be identified, and the liveness detection result of the target to be identified is obtained.
[0121] Specifically, a classifier identifies the fused features of the target to be identified, resulting in a liveness detection result. This result includes the detection category and category confidence. The detection category can be either a liveness category or a non-liveness category.
[0122] If the detection category of the target to be identified is a live target and the corresponding category confidence is not less than the threshold, then the target to be identified is determined to be a live target.
[0123] In one embodiment, a liveness detection network can be used to perform liveness detection on an image containing the target to be identified, thereby obtaining the liveness detection result of the target to be identified.
[0124] The target detection method provided in this embodiment obtains the texture features of the target and the depth features of the image by extracting features from the image to be detected. The texture features of the target and the depth features of the image are then fused, so that the fused features contain both the feature information of the target and the feature information of the background, thus having more comprehensive and richer features. The target is detected based on the fused features, thereby improving the accuracy of the target liveness detection results.
[0125] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the target detection device provided in this application. This embodiment provides a target detection device 60, which includes an acquisition module 61 and a detection module 62.
[0126] The acquisition module 61 is used to acquire the image to be detected containing the target to be identified.
[0127] The detection module 62 is used to input the image to be detected into the trained liveness detection network and output the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. When training the depth feature extractor, the depth feature extractor, texture feature extractor, and classifier are trained using sample images containing the target. The sample images have reference feature maps. The target has a category label and a domain label. The category label includes a liveness category or a non-liveness category. The domain label includes one of the following: live object, paper attack, screen attack, and mask attack.
[0128] The target detection device provided in this embodiment trains the liveness detection network with sample images having domain labels and category labels, thereby improving the detection accuracy of the liveness detection network; and performs liveness detection on the target to be identified in the image to be detected through the liveness detection network, thereby improving the accuracy of the liveness detection result.
[0129] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of an embodiment of the electronic terminal provided in this application. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described target detection method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.
[0130] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described target detection method embodiments. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.
[0131] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described embodiments of the target detection method.
[0132] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0133] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0134] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0135] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0136] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0137] The above are merely embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application’s specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.
Claims
1. A target detection method, characterized in that, The target detection method includes: Acquire the image to be detected containing the target to be identified; The image to be detected is input into the trained liveness detection network, which outputs the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. Specifically, when training the deep feature extractor, the deep feature extractor, the texture feature extractor, and the classifier are trained using sample images containing the target; the sample images have reference feature maps; the target has category labels and domain labels; the category labels include either a live object category or a non-live object category; the domain labels include one of the following: live object, paper attack, screen attack, and mask attack.
2. The target detection method according to claim 1, characterized in that, The step of inputting the image to be detected into the trained liveness detection network and outputting the liveness detection result of the target to be identified includes: Target detection is performed on the image to be detected to obtain a target detection box containing the target to be identified; The depth features of the image to be detected are obtained by extracting features from the image using the depth feature extractor. The texture feature extractor extracts features from the target detection box containing the target to be identified, thereby obtaining the texture features of the target to be identified; The depth features of the image to be detected and the texture features of the target to be identified are fused to obtain the fused features of the target to be identified; The classifier identifies the fusion features of the target to be identified, and obtains the liveness detection result of the target to be identified; the liveness detection result includes live or non-live objects.
3. The target detection method according to claim 1, characterized in that, The training method for the liveness detection network includes: The sample image is subjected to feature extraction by the depth feature extractor to obtain the predicted depth feature map of the sample image; The texture feature extractor extracts the texture of the target in the sample image to obtain a predicted texture feature map of the target and the probability of the target belonging to each of the domain categories; the domain categories include living creature, paper attack, screen attack and mask attack; The predicted depth feature map and the predicted texture feature map are fused to obtain a predicted fused feature map. The predicted category information of the target is obtained by identifying the predicted fusion feature map through the classifier. The deep feature extractor, the texture feature extractor, and the classifier are jointly trained based on the sum of the loss value between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label.
4. The target detection method according to claim 3, characterized in that, The training method for the liveness detection network further includes: Deep feature extraction is performed on the sample image to obtain the semantic feature map of the sample image; The joint training of the deep feature extractor, the texture feature extractor, and the classifier based on the sum of the loss value between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label includes: The deep feature extractor, the texture feature extractor, and the classifier are jointly trained based on the sum of the loss values between the predicted depth feature map and the reference feature map corresponding to the sample image, the loss value between the semantic feature map and the predicted depth feature map, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label.
5. The target detection method according to claim 3, characterized in that, The joint training of the deep feature extractor, the texture feature extractor, and the classifier based on the sum of the loss value between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label includes: The deep feature extractor, the texture feature extractor, and the classifier are jointly trained based on the weighted sum of the loss value between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label.
6. The target detection method according to claim 3, characterized in that, The deep feature extractor includes multiple cascaded feature extraction layers; the liveness detection network also includes a depth estimator. The step of extracting features from the sample image using the depth feature extractor to obtain a predicted depth feature map of the sample image includes: The sample image is input into the deep feature extractor to obtain the predicted feature map output by each feature extraction layer; The predicted depth feature map of the sample image is obtained by fusing features of each predicted feature map corresponding to the sample image using a depth estimator. The joint training of the deep feature extractor, the texture feature extractor, and the classifier based on the sum of the loss value between the predicted deep feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label includes: The depth feature extractor, the depth estimator, the texture feature extractor, and the classifier are jointly trained based on the sum of the loss value between the predicted depth feature map and the reference feature map corresponding to the sample image, the loss value between the probability of the target belonging to each of the domain categories and the domain label, and the loss value between the predicted category information and the category label. Remove the depth estimator from the trained liveness detection network.
7. The target detection method according to claim 2, characterized in that, The feature fusion of the depth features of the image to be detected and the texture features of the target to be identified to obtain the fused features of the target to be identified includes: The texture features corresponding to the image to be detected and the depth features of the image to be detected are superimposed to obtain the fused features of the target to be identified.
8. A target detection device, characterized in that, The target detection device includes: The acquisition module is used to acquire the image to be detected containing the target to be identified; The detection module is used to input the image to be detected into a trained liveness detection network and output the liveness detection result of the target to be identified. The liveness detection network includes a texture feature extractor, a depth feature extractor, and a classifier. When training the depth feature extractor, the depth feature extractor, the texture feature extractor, and the classifier are trained using sample images containing the target. The sample images have reference feature maps. The target has category labels and domain labels. The category labels include a liveness category or a non-liveness category. The domain labels include one of the following: live object, paper attack, screen attack, and mask attack.
9. An electronic terminal, characterized in that, The electronic terminal includes a memory and a processor coupled to each other. The processor is used to execute program instructions stored in the memory and to execute program data to implement the steps in the target detection method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target detection method as described in any one of claims 1 to 7.