Liveness detection method and system

Abstract

The liveness detection method and system provided in this specification, after obtaining the target user image, inputs the target user image into a liveness detection model to obtain the attack probability of the target user. The liveness detection model is a lightweight model obtained by decoupling and distilling easy and difficult knowledge based on a twin-teacher model group. Based on the attack probability, the liveness detection result of the target user is determined and output. This scheme can improve the accuracy of liveness detection.

Description

Technical Field

[0001] This specification relates to the field of image recognition, and in particular to a liveness detection method and system. Background Technology

[0002] In recent years, with the rapid development of internet technology, liveness detection has become an indispensable part of facial recognition systems, effectively blocking non-live attack samples. Existing liveness detection methods often perform liveness detection using liveness detection models on the cloud or edge.

[0003] In the process of researching and practicing existing technologies, the inventors of this invention discovered that when a liveness detection model is deployed on the cloud side, user images need to be uploaded to the cloud side, which relies on the network. Liveness detection cannot be performed in environments without a network or with a weak network. When liveness detection is performed on the edge side using a liveness detection model, the detection performance is limited by the computing power and storage space of the edge side, resulting in a low accuracy rate of liveness detection. Summary of the Invention

[0004] This manual provides a more accurate method and system for liveness detection.

[0005] In a first aspect, this specification provides a liveness detection method, comprising: obtaining a target user image of a target user; inputting the target user image into a liveness detection model to obtain an attack probability of the target user, wherein the liveness detection model is a lightweight model obtained by decoupling and distilling easy and difficult knowledge based on a twin teacher model group; and determining a liveness detection result of the target user based on the attack probability, and outputting the liveness detection result.

[0006] In some embodiments, the knowledge of difficulty includes simple knowledge and difficult knowledge, wherein the simple knowledge includes knowledge corresponding to salient features, and the difficult knowledge includes knowledge corresponding to features other than the salient features.

[0007] In some embodiments, the training process of the liveness detection model includes the following steps: obtaining a first user image sample and inputting the first user image sample into the twin teacher model group to obtain teacher image features and teacher predicted categories; inputting the first user image sample, the teacher image features, and the teacher predicted categories into a preset liveness detection model to obtain student image features, student predicted categories, and decoupling weights corresponding to the easy and difficult knowledge; and distilling the preset liveness detection model based on the teacher image features, the teacher predicted categories, the student image features, the student predicted categories, and the decoupling weights to obtain the trained liveness detection model.

[0008] In some embodiments, the preset liveness detection model includes a student network and a meta-network corresponding to the easy and difficult knowledge; and the step of inputting the first user image sample, the teacher image features, and the teacher predicted category into the preset liveness detection model to obtain the student image features, the student predicted category, and the decoupling weights corresponding to the easy and difficult knowledge includes: inputting the first user image sample into the student network to obtain the student image features and the student predicted category corresponding to the first user image sample, and inputting the teacher image features, the teacher predicted category, the student image features, and the student predicted category into the meta-network to obtain the decoupling weights corresponding to the easy and difficult knowledge.

[0009] In some embodiments, distilling the preset liveness detection model to obtain the trained liveness detection model includes: comparing the teacher's predicted category with the student's predicted category to obtain categorical distillation loss information; comparing the teacher's image features with the student's image features based on the decoupling weights to obtain decoupling distillation loss information; and fusing the categorical distillation loss information and the decoupling distillation loss information, and converging the preset liveness detection model based on the fused target distillation loss information to obtain the liveness detection model.

[0010] In some embodiments, comparing the teacher image features with the student image features to obtain decoupled distillation loss information includes: extracting salient features from the teacher image features and the student image features respectively to obtain simple knowledge distillation loss information corresponding to simple knowledge; comparing the absolute values ​​of the teacher image features and the absolute values ​​of the student image features to obtain difficult knowledge distillation loss information corresponding to difficult knowledge; and performing weighted fusion of the simple knowledge distillation loss information and the difficult knowledge distillation loss information based on the decoupling weights to obtain the decoupled distillation loss information.

[0011] In some embodiments, the step of extracting salient features from the teacher image features and the student image features respectively to obtain simple knowledge distillation loss information corresponding to simple knowledge includes: extracting salient features from the teacher image features to obtain teacher simple knowledge corresponding to the twin teacher model group; extracting the salient features from the student image features to obtain student simple knowledge corresponding to the student network; and comparing the teacher simple knowledge with the student simple knowledge to obtain the simple knowledge distillation loss information.

[0012] In some embodiments, the teacher image features include multiple image sub-features; and the step of extracting salient features from the teacher image features to obtain simple teacher knowledge corresponding to the twin teacher model group includes: determining the feature value of each image sub-feature among the multiple image sub-features, sorting the multiple image sub-features from largest to smallest based on the feature value, and selecting image sub-features corresponding to a preset sorting range among the multiple image sub-features based on the sorting result to obtain the salient features, and using the salient features as the simple teacher knowledge.

[0013] In some embodiments, the number of the plurality of image sub-features is N, and the preset sorting range is the first N / 2 image sub-features.

[0014] In some embodiments, comparing the absolute values ​​of the teacher image features with the absolute values ​​of the student image features to obtain the difficult knowledge distillation loss information corresponding to the difficult knowledge includes: determining the absolute values ​​of the teacher image features to obtain the teacher's difficult knowledge, and determining the absolute values ​​of the student image features to obtain the student's difficult knowledge; obtaining the feature difference between the teacher's difficult knowledge and the student's difficult knowledge; and determining the difficult knowledge distillation loss information corresponding to the difficult knowledge based on the feature difference.

[0015] In some embodiments, after distilling the preset liveness detection model to obtain the trained liveness detection model, the method further includes: obtaining statistical features of students' simple knowledge in the simple knowledge, obtaining a first statistical feature, and determining a first feature distribution of the students' simple knowledge based on the first statistical feature; and obtaining statistical features of students' difficult knowledge in the difficult knowledge, obtaining a second statistical feature, and determining a second feature distribution of the students' difficult knowledge based on the second statistical feature.

[0016] In some embodiments, the statistical feature includes at least one of mean and variance.

[0017] In some embodiments, the method further includes: when the running time of the liveness detection model reaches a preset time period, calculating the data distribution probability of the user image set during the running time, the data distribution probability including a first distribution probability and a second distribution probability of the user image in the user image set being in the first feature distribution and being in the second feature distribution; and selecting one of a first scheme, a second scheme, and a third scheme based on the first distribution probability and the second distribution probability, wherein: the first scheme includes: determining that the first distribution probability is less than a first preset probability threshold, and using the liveness detection model as the updated target liveness detection model; the second scheme includes: determining that the second distribution probability is less than a second preset probability threshold, and training the liveness detection model based on the user image to obtain the updated target liveness detection model; and the third scheme includes: determining that the first distribution probability is greater than the first preset probability threshold and the second distribution probability is greater than the second preset probability threshold, and sending the user image to a remote server so that the remote server updates the liveness detection model and receives the updated target liveness detection model returned by the remote server.

[0018] In some embodiments, the step of statistically analyzing the data distribution probability of the user image set within the running time includes: obtaining the user image set within the running time, and determining a first initial distribution probability and a second initial distribution probability of each user image in the user image set being in the first feature distribution and being in the second feature distribution; determining the mean of the first initial distribution probability to obtain a first distribution probability, and determining the mean of the second initial distribution probability to obtain a second distribution probability; and using the first distribution probability and the second distribution probability as the data distribution probability of the user image set.

[0019] In some embodiments, the training process of the twin teacher model group includes the following steps: obtaining a second user image sample; inputting the second user image sample into a preset twin teacher model group to obtain the sample image features and predicted user category output by each teacher model in the preset twin teacher model group; and converging the preset twin teacher model group based on the sample image features and the predicted user category to obtain the trained twin teacher model group.

[0020] In some embodiments, the preset twin-teacher model group includes a first teacher model and a second teacher model with identical structures; and the convergence of the preset twin-teacher model group to obtain the trained twin-teacher model group includes: obtaining the labeled user category of the second user image sample, comparing the labeled user category with the predicted user category to obtain single classification loss information, comparing the sample image features of the first teacher model with the sample image features of the second teacher model to obtain feature complementarity loss information, fusing the predicted user category of the first teacher model with the predicted category of the second teacher model to obtain fused classification loss information, and fusing the single classification loss information, the feature complementarity loss information and the fused classification loss information, and converging the preset twin-teacher model group based on the fused target teacher loss information to obtain the twin-teacher model group.

[0021] In some embodiments, comparing the sample image features of the first teacher model with the sample image features of the second teacher model to obtain feature complementarity loss information includes: extracting a first target sample image feature output by each network layer of the first teacher model from the sample image features of the first teacher model; extracting a second target sample image feature output by each network layer of the second teacher model from the sample image features of the second teacher model; and determining the similarity between the first target sample image feature and the corresponding second target sample image feature to obtain feature complementarity loss information for each network layer, wherein the constraint condition of the feature complementarity loss information is that the similarity between the target sample image features of corresponding network layers of different teacher models is less than a preset similarity threshold.

[0022] In some embodiments, fusing the predicted user categories of the first teacher model and the predicted user categories of the second teacher model to obtain fused classification loss information includes: extracting a first predicted probability corresponding to each candidate user category from the predicted user categories of the first teacher model; extracting a second predicted probability corresponding to each candidate user category from the predicted user categories of the second teacher model; adding the first predicted probability to the corresponding second predicted probability to obtain a target predicted probability for each candidate user category; and determining the fused target user category based on the target predicted probability, and comparing the target user category with the labeled user category to obtain the fused classification loss information.

[0023] In some embodiments, determining the liveness detection result of the target user based on the attack probability includes: when the attack probability is greater than a preset attack probability threshold, determining the liveness detection result of the target user as an attacking user.

[0024] In some embodiments, the method further includes: when the attack probability is less than the preset attack probability threshold, determining that the liveness detection result of the target user is a normal user.

[0025] Secondly, this specification also provides a liveness detection system, comprising: at least one storage medium storing at least one instruction set for performing liveness detection; and at least one processor communicatively connected to the at least one storage medium, wherein, when the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the liveness detection method described in the first aspect of this specification according to the instructions of the at least one instruction set.

[0026] As can be seen from the above technical solutions, the liveness detection method and system provided in this specification, after obtaining the target user image, inputs the target user image into a liveness detection model to obtain the attack probability of the target user. The liveness detection model is a lightweight model obtained by decoupling and distilling easy and difficult knowledge based on a twin-teacher model group. Based on the attack probability, the liveness detection result of the target user is determined and output. Since this solution guides the student model through a twin-teacher model group, and obtains a lightweight liveness detection model on the edge by decoupling and distilling easy and difficult knowledge during the guidance process, the performance of liveness detection on the edge can be improved through the lightweight liveness detection model, thus improving the accuracy of liveness detection.

[0027] Other functions of the liveness detection methods and systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the liveness detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram illustrating an application scenario of a liveness detection system provided according to an embodiment of this specification is shown.

[0030] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0031] Figure 3 A flowchart of a liveness detection method according to embodiments of this specification is shown; and

[0032] Figure 4 A schematic diagram of the overall process of a liveness detection method in a face recognition scenario provided by an embodiment of this specification is shown. Detailed Implementation

[0033] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0034] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0035] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0036] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0037] For ease of description, the terms that will appear in the following descriptions will be explained as follows:

[0038] Knowledge distillation: refers to a model training method that uses a complex teacher model (network) to guide a lightweight student model (network), thereby improving the performance of the student model;

[0039] Liveness detection: In facial recognition systems, the method of detecting attacks using photos, screens, etc., is called liveness detection;

[0040] Easy and difficult knowledge: This refers to dividing the knowledge of the teacher network in knowledge distillation into two parts: easy knowledge and difficult knowledge.

[0041] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0042] The liveness detection method provided in this manual can be applied to any liveness detection scenario. For example, in a face payment scenario, the liveness detection method in this manual can be used to perform liveness detection on the collected face image of the user to be paid; in an identity verification scenario, the liveness detection method in this manual can be used to perform liveness detection on the collected identity image of the user; it can also be applied to any face recognition scenario, which will not be elaborated here.

[0043] Those skilled in the art should understand that the liveness detection methods and systems described in this specification are also within the scope of protection of this specification when applied to other application scenarios.

[0044] Figure 1 This diagram illustrates an application scenario of a liveness detection system 001 provided according to an embodiment of this specification. The liveness detection system 001 (hereinafter referred to as system 001) can be applied to liveness detection in any scenario, such as liveness detection in face payment scenarios, liveness detection in identity verification scenarios, liveness detection in face recognition scenarios, etc. Figure 1 As shown, system 001 may include user 100, client 200, server 300, and network 400.

[0045] User 100 can be the user who triggers the recognition of the target user's image. User 100 can perform liveness detection on client 200. User 100 can be the target user or other users.

[0046] Client 200 can be a device for performing liveness detection on a target user image in response to a liveness detection operation by user 100. In some embodiments, the liveness detection method can be executed on client 200. In this case, client 200 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, client 200 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to operate. Figure 1As shown, client 200 can communicate with server 300. In some embodiments, server 300 can communicate with multiple clients 200. In some embodiments, client 200 can interact with server 300 through network 400 to receive or send messages, such as receiving or sending images of a target user. In some embodiments, client 200 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, the mobile device may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, the smart home device may include smart TVs, desktop computers, or any combination thereof. In some embodiments, the smart mobile device may include smartphones, personal digital assistants, gaming devices, navigation devices, or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include virtual reality headsets, virtual reality glasses, virtual reality controllers, augmented reality headsets, augmented reality glasses, augmented reality controllers, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, head-mounted displays, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle TV, etc. In some embodiments, the client 200 may include an image acquisition device for acquiring at least one image of the target user, thereby obtaining an image of the target user. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera), or a combination of a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.). In some embodiments, the client 200 may be a device with positioning technology for locating the position of the client 200.

[0047] In some embodiments, the client 200 may have one or more applications (APPs) installed. The APPs provide the user 100 with the ability and interface to interact with the outside world via the network 400. The APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc. In some embodiments, the client 200 may have a target APP installed. The target APP can collect at least one user image of the target user for the client 200, thereby obtaining a target user image. In some embodiments, the user 100 can also trigger a liveness detection request through the target APP. The target APP can respond to the liveness detection request and execute the liveness detection method described in this specification. The liveness detection method will be described in detail later.

[0048] Server 300 may be a server providing various services, such as a backend server supporting liveness detection of target user images acquired on client 200. In some embodiments, the liveness detection method may be executed on server 300. In this case, server 300 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, server 300 may include hardware devices with data processing capabilities and necessary programs for driving the hardware devices. Server 300 may communicate with multiple clients 200 and receive data sent by clients 200.

[0049] Network 400 serves as a medium to provide a communication connection between client 200 and server 300. Network 400 facilitates the exchange of information or data. For example... Figure 1 As shown, client 200 and server 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth. TM Network, ZigBee TMA network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of client 200 and server 300 can connect to network 400 to exchange data or information.

[0050] It should be understood that Figure 1 The number of clients 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 200, servers 300, and networks 400.

[0051] It should be noted that the liveness detection method can be executed entirely on the client 200, entirely on the server 300, or partially on the client 200 and partially on the server 300.

[0052] Figure 2 A hardware structure diagram of a computing device 600 provided according to an embodiment of this specification is shown. The computing device 600 can execute the liveness detection method described in this specification. The liveness detection method is described in other parts of this specification. When the liveness detection method is executed on a client 200, the computing device 600 can be the client 200. When the liveness detection method is executed on a server 300, the computing device 600 can be the server 300. When the liveness detection method can be executed partly on the client 200 and partly on the server 300, the computing device 600 can be both the client 200 and the server 300.

[0053] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0054] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0055] I / O component 660 supports input / output between computing device 600 and other components.

[0056] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 400. Communication port 650 can be a wired communication port or a wireless communication port.

[0057] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the liveness detection method provided in this specification.

[0058] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. At least one processor 620 is used to execute the at least one instruction set described above. When the computing device 600 is running, at least one processor 620 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the liveness detection method provided in this specification. Processor 620 can execute all the steps included in the liveness detection method. Processor 620 can be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0059] Figure 3A flowchart of a liveness detection method P100 according to an embodiment of this specification is shown. As previously described, computing device 600 can execute the liveness detection method P100 of this specification. Specifically, processor 620 can read an instruction set stored in its local storage medium and then execute the liveness detection method P100 of this specification according to the instructions in the instruction set. Figure 3 As shown, method P100 may include:

[0060] S110: Obtain the target user image of the target user.

[0061] The target user image can be an image containing the target user's biometric features. These biometric features may include at least one of the following: face, torso, iris, sclera, fingerprint, palm print, voiceprint, or skeletal projection.

[0062] There are several ways to obtain the target user's image, including the following:

[0063] For example, the processor 620 can directly receive at least one user image of the target user uploaded by user 100 or the target user through a client, terminal or server to obtain the target user image; or, it can obtain an image containing the biometric features of the target user from a network or image database to obtain the target user image; or, it can acquire at least one user image containing biometric features of the target user through an image acquisition device to obtain the target user image; or, when there are many target user images or a large amount of memory, it can also receive a liveness detection request, which includes the storage address of the target user image of the target user, and obtain the target user image of the target user based on the storage address, and so on.

[0064] S120: Input the target user image into the liveness detection model to obtain the attack probability of the target user.

[0065] The liveness detection model is a lightweight model obtained by decoupling and distilling easy and difficult knowledge based on a twin-teacher model group. The twin-teacher model group can include two teacher models with identical network structures. The easy and difficult knowledge includes simple knowledge and hard knowledge. Simple knowledge includes knowledge corresponding to salient features, and hard knowledge includes knowledge corresponding to features other than salient features (non-salient features). The salient features can be understood as features within a preset sorting range from largest to smallest among the features of the last layer output by the liveness detection model.

[0066] The attack probability can be the probability that the target user is the attacking user, and the attacking user can be a non-live user.

[0067] The liveness detection model can include a distilled student network. The distilled student network predicts the attack probability of the target user. The distilled student network can include a feature extraction subnetwork and a liveness classification subnetwork. There are several ways to input the target user into the liveness detection model to obtain the attack probability of the target user, including the following:

[0068] For example, the processor 620 can input the target user image into the feature extraction subnetwork to obtain the user image features of the target user image, and input the user image features into the liveness classification subnetwork to obtain the attack probability of the target user. Alternatively, the processor 620 can input the target user image into the feature extraction subnetwork to obtain the user features of the target user image, and input the user features into the liveness classification subnetwork to obtain the attack probability of the target user.

[0069] The training process of the liveness detection model may include the following steps: the processor 620 can obtain a first user image sample and input the first user image sample into a twin teacher model group to obtain teacher image features and teacher predicted categories; input the first user image sample, teacher image features, and teacher predicted categories into a preset liveness detection model to obtain student image features, student predicted categories, and decoupling weights; and distill the preset liveness detection model based on teacher image features, teacher predicted categories, student image features, student predicted categories, and decoupling weights to obtain the trained liveness detection model.

[0070] Among them, the teacher image features can be the image features of the last layer output by the twin teacher model group, and the teacher prediction category can be the liveness category of the first user image sample predicted by the twin teacher model group.

[0071] The preset liveness detection model can include a student network and a meta-network corresponding to easy and difficult knowledge. Student image features can be the image features of the last layer output by the student network, and the student predicted category can be the liveness category of the first user image sample predicted by the student network. Decoupling weights can be the weights corresponding to the easy and difficult knowledge output by the meta-network, respectively. There are multiple ways to input the first user image sample into the preset liveness detection model to obtain the student image features, student predicted category, and decoupling weights corresponding to easy and difficult knowledge. For example, the processor 620 can input the first user image sample into the student network to obtain the student image features and student predicted category corresponding to the first user image sample, and input the teacher image features, teacher predicted category, student image features, and student predicted category into the meta-network to obtain the decoupling weights corresponding to easy and difficult knowledge.

[0072] After obtaining the teacher image features, teacher predicted category, student image features, student predicted category, and decoupling weights, the processor 620 can distill the preset liveness detection model based on these features to obtain a trained liveness detection model. There are several ways to distill the preset liveness detection model. For example, the processor 620 can compare the teacher's predicted category with the student's predicted category to obtain classification distillation loss information; compare the teacher's image features with the student's image features based on the decoupling weights to obtain decoupling distillation loss information; fuse the classification distillation loss information and the decoupling distillation loss information; and converge the preset liveness detection model based on the fused target distillation loss information to obtain the liveness detection model.

[0073] The classification distillation loss information can be the loss information generated by the teacher model and the student network when performing liveness classification. The constraint on the classification distillation loss information is that the difference between the prediction results of the teacher model and the student network when performing liveness classification on the same first user image sample is less than a preset difference threshold; that is, the prediction results of the teacher model and the student network when performing liveness classification should be as consistent as possible. There are several ways to compare the teacher's predicted category with the student's predicted category to obtain the classification loss information. For example, the processor 620 can use the cross-entropy loss function to compare the teacher's predicted category with the student's predicted category to obtain the classification loss information. Alternatively, it can use the contrastive loss function to compare the teacher's predicted category with the student's predicted category to obtain the classification loss information.

[0074] The decoupling distillation loss information can be understood as the distillation loss information generated when the student network learns simple and difficult knowledge under the guidance of the teacher model. There are several ways to compare teacher image features with student image features to obtain the decoupling distillation loss information. For example, the processor 620 can extract salient features from both teacher and student image features to obtain the simple knowledge distillation loss information corresponding to simple knowledge; compare the absolute values ​​of teacher image features with the absolute values ​​of student image features to obtain the difficult knowledge distillation loss information corresponding to difficult knowledge; and, based on decoupling weights, perform weighted fusion of the simple knowledge distillation loss information and the difficult knowledge distillation loss information to obtain the decoupling distillation loss information.

[0075] The simple knowledge distillation loss information refers to the distillation loss information generated when the twin-teacher model group distills simple knowledge into the student network. There are several ways to extract salient features from both teacher and student image features to obtain the simple knowledge distillation loss information. For example, the processor 620 can extract salient features from the teacher image features to obtain the teacher's simple knowledge corresponding to the twin-teacher model group, extract salient features from the student image features to obtain the student's simple knowledge corresponding to the student network, and compare the teacher's simple knowledge with the student's simple knowledge to obtain the simple knowledge distillation loss information.

[0076] The teacher's simple knowledge can be the simple knowledge obtained after decoupling the knowledge to be distilled from the knowledge in the twin teacher model group. This simple knowledge can be the knowledge corresponding to salient features. Teacher image features can include multiple image sub-features. It should be noted that the feature values ​​of these image sub-features are different, and the feature values ​​can be positive or negative. There are multiple ways to extract salient features from teacher image features. For example, the processor 620 can determine the feature value of each image sub-feature among multiple image sub-features, sort the multiple image sub-features from largest to smallest based on the feature values, and select the image sub-features corresponding to a preset sorting range from the multiple image sub-features based on the sorting results to obtain salient features, and use the salient features as teacher's simple knowledge.

[0077] Here, the simple knowledge of students can be the simple knowledge decoupled from the twin-teacher model group learned by the student network. The method of extracting salient features from student image features is similar to that of extracting salient features from teacher image features, as detailed above, and will not be repeated here.

[0078] Taking an image with N sub-features as an example, the preset sorting range can be the first N / 2. The corresponding salient features can be the first N / 2 of the image sub-features. Salient features can be understood as features with larger feature values. These features are easier for the learning network to learn or distill. Therefore, simple knowledge can be used to identify these easily learned or distilled salient features.

[0079] After obtaining the teacher's simple knowledge and the student's simple knowledge, the processor 620 can compare the teacher's simple knowledge with the student's simple knowledge to obtain simple knowledge distillation loss information. There are several ways to compare the teacher's unit with the student's simple knowledge. For example, the processor 620 can determine the difference between the teacher's simple knowledge and the student's simple knowledge, obtain the simple knowledge distillation difference, and determine the L2 norm of the simple knowledge distillation difference, thereby obtaining the simple knowledge distillation loss information. Taking the image sub-features where the salient features can be the top N / 2 as an example, the method for determining the simple knowledge distillation loss information can be as shown in formula (1):

[0080]

[0081] Among them, K easy For simple knowledge distillation, information is lost, f t Here, N represents the number of image sub-features in the teacher's image features. For teachers' simple knowledge, f s Image sub-features in student image features Provide students with simple knowledge.

[0082] The difficult knowledge distillation loss information can be the distillation loss information generated when the twin teacher model group distills difficult knowledge into the student network. The difficult knowledge can be the knowledge corresponding to features other than salient features (non-salient features). There are multiple ways to compare the absolute values ​​of the teacher image features and the absolute values ​​of the student image features to obtain the difficult knowledge distillation loss information. For example, the processor 620 can determine the absolute value of the teacher image features to obtain the teacher's difficult knowledge, and determine the absolute value of the student image features to obtain the student's difficult knowledge. It can then obtain the feature difference between the teacher's difficult knowledge and the student's difficult knowledge, and based on the feature difference, determine the difficult knowledge distillation loss information corresponding to the difficult knowledge.

[0083] Among them, the teacher's difficult knowledge can be the difficult knowledge obtained by decoupling the knowledge that needs to be distilled from the knowledge of the twin teacher model group. There are several ways to obtain the teacher's difficult knowledge by determining the absolute value of the teacher image features. For example, the processor 620 can determine the absolute value of the feature value of each image sub-feature in the teacher image features, thereby obtaining the teacher's difficult knowledge. Alternatively, it can delete salient features from the image sub-features of the teacher image features to obtain the target image sub-features, and determine the absolute value of the feature value of the target image sub-features, thereby obtaining the teacher's difficult knowledge.

[0084] Among these, student-related difficult knowledge can be defined as the difficult knowledge decoupled from the twin-teacher model group learned by students online. The method for determining student-related difficult knowledge is similar to that for determining teacher-related difficult knowledge, as detailed above, and will not be repeated here.

[0085] After determining the teacher's difficult knowledge and the student's difficult knowledge, the processor 620 can obtain the feature difference between the teacher's difficult knowledge and the student's difficult knowledge. Then, based on the feature difference, it determines the difficult knowledge distillation loss information corresponding to the difficult knowledge. There are several ways to determine the difficult knowledge distillation loss information. For example, the processor 620 can determine the L2 norm of the feature difference to obtain the difficult knowledge distillation loss information, as shown in formula (2):

[0086]

[0087] Among them, K hard For difficult knowledge distillation, information loss is achieved, f t For the image features of the teacher, abs(f) is an image sub-feature. t (For teachers' difficult knowledge, f) s For the image features of students, abs(f) is an image sub-feature. s This is for students to learn difficult knowledge.

[0088] After determining the simple knowledge distillation loss information and the hard knowledge distillation loss information, the processor 620 can perform weighted fusion of the simple knowledge distillation loss information and the hard knowledge distillation loss information based on decoupling weights to obtain the decoupled distillation loss information. The decoupling weights include simple knowledge distillation weights and hard knowledge distillation weights. There are several ways to perform weighted fusion of the simple knowledge distillation loss information and the hard knowledge distillation loss information. For example, the processor 620 can weight the simple knowledge distillation loss information based on the simple knowledge distillation weights to obtain weighted simple knowledge distillation loss information, and weight the hard knowledge distillation loss information based on the hard knowledge distillation weights to obtain weighted hard knowledge distillation loss information. The processor then fuses the weighted simple knowledge distillation loss information with the weighted hard knowledge distillation loss information to obtain the decoupled distillation loss information.

[0089] After determining the classification distillation loss information and the decoupled distillation loss information, the processor 620 can fuse the classification distillation loss information and the decoupled distillation loss information. There are various ways to fuse them. For example, the processor 620 can directly add the classification distillation loss information and the decoupled distillation loss information to obtain the fused target distillation loss information, as shown in formula (3):

[0090] Loss1 = Loss prediction +a*Keasy +v*K hard (3)

[0091] Where Loss1 is the target distillation loss information, Loss prediction To classify distillation loss information, 'a' represents the simple knowledge distillation weight, and 'K' represents the weight. easy For simple knowledge distillation, b represents the loss information; for difficult knowledge distillation, K represents the weight; and K represents the weight of the loss information. hard Distilling lost information for difficult knowledge.

[0092] After fusing the classification distillation loss information and the decoupled distillation loss information, the processor 620 can converge the preset liveness detection model based on the fused target distillation loss information, thereby obtaining a liveness detection model. There are several ways to converge the preset liveness detection model. For example, the processor 620 can use a gradient descent algorithm to update the network parameters of the preset liveness detection model based on the target distillation loss information, obtaining an updated liveness detection model. It then updates the preset liveness detection model to the updated liveness detection model and returns to the step of obtaining the first user image sample until the preset liveness detection model converges, thus obtaining the trained liveness detection model. Alternatively, it can use other network parameter update algorithms to update the network parameters of the preset liveness detection model based on the target distillation loss information, obtaining an updated liveness detection model. It then updates the preset liveness detection model to the updated liveness detection model and returns to the step of obtaining the first user image sample until the preset liveness detection model converges, thus obtaining the trained liveness detection model, and so on.

[0093] Traditional knowledge distillation treats the teacher network's output as a whole. However, each part varies in difficulty, requiring iterative learning. Therefore, this solution employs an adaptive distillation method based on knowledge decoupling of easy and difficult knowledge when distilling the pre-set liveness detection model. This method decouples the teacher network's output into easy and difficult knowledge, and uses different decoupling or distillation weights to distill the difficult and easy knowledge, thereby improving the detection performance of the trained liveness detection model and ultimately increasing its accuracy.

[0094] It should be noted that the execution entity for training the liveness detection model can be the processor 620 in the edge computing device 600, or the processor 620 in the cloud-side (server-side) computing device 600.

[0095] In some embodiments, before distilling the preset liveness detection model, the processor 620 may also train the twin-teacher model group, thereby distilling the preset liveness detection model based on the trained twin-teacher model group to obtain the trained liveness detection model. The training process of the twin-teacher model group may include the following steps: the processor 620 may obtain a second user image sample, input the second user image sample into the preset twin-teacher model group to obtain the sample image features and predicted user category output by each teacher model in the preset twin-teacher model, and converge the preset twin-teacher model group based on the sample image features and predicted user category to obtain the trained twin-teacher model group.

[0096] The pre-defined twin-teacher model group includes a first teacher model and a second teacher model with identical structures. There are several ways to converge the pre-defined twin-teacher model group. For example, the processor 620 can obtain the labeled user category of the second user image sample, compare the labeled user category with the predicted user category to obtain single classification loss information, compare the sample image features of the first teacher model with the sample image features of the second teacher model to obtain feature complementarity loss information, fuse the predicted user category of the first teacher model with the predicted user category of the second teacher model to obtain fused classification loss information, and fuse the single classification loss information, feature complementarity loss information, and fused classification loss information. Based on the fused target teacher loss information, the pre-defined twin-teacher model group is then converged to obtain the twin-teacher model group.

[0097] The single-classification loss information can be the loss information generated by the first teacher model independently performing liveness classification and the loss information generated by the second teacher model independently performing liveness classification. There are several ways to obtain the single-classification loss information by comparing the labeled user category with the predicted user category. For example, the processor 620 can extract the first predicted user category corresponding to the first teacher model and the second predicted user category corresponding to the second teacher model from the predicted user category. It can then compare the first predicted user category with the labeled user category to obtain the first initial classification loss information, compare the second predicted user category with the labeled user category to obtain the second initial classification loss information, and finally add the first and second initial classification loss information to obtain the single-classification loss information.

[0098] There are several ways to compare the first predicted user category with the labeled user category to obtain the first initial classification loss information. For example, the processor 620 can use the cross-entropy loss function to compare the first predicted user category with the labeled user category to obtain the first initial classification loss information. Alternatively, other types of loss functions can be used to compare the first predicted user category with the labeled user category to obtain the first initial classification loss information, and so on.

[0099] The processor 620 determines the first initial classification loss information in a similar way to determine the second initial classification loss information, as detailed above, and will not be repeated here.

[0100] The feature complementarity loss information can be the loss information corresponding to the difference between the features output by the first teacher model and the features output by the corresponding layer of the second teacher model. The constraint condition for the feature complementarity loss information is that the similarity between the target sample image features of the corresponding network layers of different teacher models is less than a preset similarity threshold; that is, the similarity (e.g., cosine similarity) of the features output by the corresponding layers of different teacher models is close to zero. There are multiple ways to compare the sample image features of the first teacher model with those of the second teacher model. For example, the processor 620 can extract the first target sample image feature output by each network layer of the first teacher model from the sample image features of the first teacher model, extract the second target sample image feature output by each network layer of the second teacher model from the sample image features of the second teacher model, and determine the similarity between the first and second target sample image features to obtain the feature complementarity loss information for each network layer.

[0101] The fusion classification loss information can be the loss information corresponding to the classification accuracy of the fusion decision results between different teacher models, or it can be understood as the loss information corresponding to the difference between the predicted user categories output by different teacher models after fusion and the labeled user categories. There are several ways to fuse the predicted user categories of the first teacher model and the predicted user categories of the second teacher model. For example, the processor 620 can extract the first prediction probability corresponding to each candidate user category from the predicted user categories of the first teacher model, extract the second prediction probability corresponding to each candidate user category from the predicted user categories of the second teacher model, add the first prediction probability and the second prediction probability to obtain the target prediction probability of each candidate user category, and determine the fused target user category based on the target prediction probability. The target user category is then compared with the labeled user category to obtain the fusion classification loss information.

[0102] After determining the single-classification loss information, feature complementation loss information, and fusion classification loss information, the processor 620 can fuse these three loss information. There are various fusion methods; for example, the processor 620 can add the single-classification loss information, feature complementation loss information, and fusion classification loss information to obtain the fused target teacher loss information, as shown in formula (4).

[0103] Loss2 = Loss cls +Loss plementary +Loss ensemble (4)

[0104] Where Loss2 represents the target teacher's loss information, Loss cls For single-class loss information, Loss plementary For feature complementation loss information, Loss ensemble To integrate classification loss information.

[0105] After fusing the single classification loss information, feature complementation loss information, and fusion classification loss information, the processor 620 can converge the preset twin-teacher model group based on the fused target teacher loss information, thereby obtaining the twin-teacher model. The convergence method for the preset twin-teacher model group is similar to the convergence method for the preset liveness detection model, as detailed above, and will not be repeated here.

[0106] It's important to note that traditional teacher model training methods often use training data to train a single model, resulting in limited and insufficient knowledge. This approach utilizes a mutual learning method based on two identical teacher models. This mutual learning between two teacher models improves the performance of each model, leading to a higher-performing individual teacher model. This, in turn, enhances the detection performance of the distilled liveness detection model and increases its accuracy.

[0107] In some embodiments, after the processor 620 distills the preset liveness detection model to obtain the trained liveness detection model, it can also determine the original distribution of easy and difficult knowledge in the training data. There are various ways to determine the original distribution of easy and difficult knowledge. For example, the processor 620 can obtain statistical features of students' simple knowledge in the easy knowledge category, obtain a first statistical feature, and determine a first feature distribution of students' simple knowledge based on the first statistical feature; and obtain statistical features of students' difficult knowledge in the difficult knowledge category, obtain a second statistical feature, and determine a second feature distribution of students' difficult knowledge based on the second statistical feature.

[0108] The statistical characteristics include at least one of the mean and variance, and may also include other statistical characteristics. There are various ways to obtain the statistical characteristics of students' simple knowledge; for example, the processor 620 can obtain students' simple knowledge... The mean and variance of the first statistical characteristic are obtained, or alternatively, the mean and variance of the first statistical characteristic can be obtained. Other statistical characteristics are used to obtain the first statistical characteristic.

[0109] After obtaining the first statistical feature, the processor 620 can determine the first feature distribution of students' simple knowledge based on the first statistical feature. There are several ways to determine the first feature distribution. For example, the processor 620 can generate a Gaussian distribution based on the mean and variance of students' simple knowledge and use the Gaussian distribution as the first feature distribution.

[0110] The method for obtaining the second statistical feature is similar to the method for obtaining the first statistical feature, and the method for determining the distribution of the second feature is similar to the method for determining the distribution of the first feature, as detailed above, and will not be repeated here.

[0111] It should be noted that determining the feature distribution of easy and difficult knowledge in the training data is mainly used to measure the degree of distribution change of the samples (user images) on the edge through changes in the distribution of easy and difficult knowledge, thereby determining the adaptive adaptation of the liveness detection model. Therefore, after distilling the preset liveness detection model to obtain the trained liveness detection model, the data distribution probability of the user image set accumulated by the liveness detection model can be statistically analyzed after the liveness detection model runs, thereby adaptively updating the liveness detection model. There are several ways to adaptively update the liveness detection model. For example, when the running time of the liveness detection model reaches a preset time period, the processor 620 can statistically analyze the data distribution probability of the user image set during the running time. The data distribution probability can include a first distribution probability of the user image in the user image set being in a first feature distribution and a second distribution probability of being in a second feature distribution. Based on the first distribution probability and the second distribution probability, one of the first, second, and third schemes can be selected and executed to adaptively update the liveness detection model.

[0112] There are several methods for calculating the data distribution probability of the user set during the running time. For example, the processor 620 can obtain the user image set during the running time, and determine the first initial distribution probability and the second initial distribution probability of each user image in the user image set that is in the first feature distribution and the second initial distribution probability that is in the second feature distribution. It can also determine the mean of the first initial classification probability to obtain the first distribution probability, and determine the mean of the second initial classification probability to obtain the second distribution probability. Finally, it can use the first distribution probability and the second distribution probability as the data distribution probability of the user image set.

[0113] Wherein, the first distribution probability can be the probability of simple knowledge corresponding to the user image set, and the second distribution can be the probability of difficult knowledge corresponding to the user image. Based on the first and second distribution probabilities, there are multiple ways to select one of the first, second, and third schemes for execution. For example, the first scheme can be: the processor 620 determines that the first distribution probability is less than the first preset probability threshold, and uses the liveness detection model as the updated target liveness detection model, that is, there is no need to update the liveness detection model; the second scheme can be: the processor 620 determines that the second distribution probability is less than the second preset probability threshold, and trains the liveness detection model based on the user image to obtain the updated target liveness detection model, that is, the liveness detection model is updated on the device side using user images (adaptive training); the third scheme: determines that the first distribution probability is greater than the first preset probability threshold, and the second distribution probability is greater than the second preset probability threshold, and sends the user image to the remote server so that the remote server can update the liveness detection model and receive the updated target liveness detection model returned by the remote server, that is, the liveness detection model needs to be retrained on the server side (remote server), and the training samples are still some or all of the user images accumulated on the device side.

[0114] On the device side, the processor 620 can train the liveness detection model based on user images in several ways. For example, the processor 620 can obtain the historical predicted user category corresponding to the user image and use the historical predicted category as the labeled user category (user label) of the user image to obtain historical user image samples. These historical user image samples are then used as the first user image samples, and the liveness detection model is used as the preset liveness detection model. The preset liveness detection model is then trained to obtain the updated target liveness detection model. The training process of the processor 620 for the preset liveness detection model can be found above and will not be repeated here.

[0115] In the cloud (remote server), there are multiple ways to handle the liveness detection model. For example, the processor 620 can send the liveness detection model and some or all of the user images to the remote server so that the remote server can manually annotate the received user images. Then, the liveness detection model can be updated based on the manual annotation results. The processor 620 can receive the updated target liveness detection model returned by the remote server.

[0116] It should be noted that traditional edge models will experience performance fluctuations as the distribution of edge data changes (edge ​​data can change significantly over time). When the data distribution changes little, adaptive adaptation can be completed directly on the edge. However, when the distribution changes significantly, the server needs to retrain the edge model to complete the adaptation. In this scheme, K samples are accumulated within a preset time period, and the data distribution probabilities of these K samples are determined. When the first distribution probability p1 is less than the first preset probability threshold T1, it indicates that the data distribution has not changed significantly, and the liveness detection model does not require adaptive training. When the second distribution probability p2 is less than the second preset probability threshold T2, it indicates that the data distribution has changed slightly, and the liveness detection model can be trained on the edge to obtain the updated target liveness detection model. When both the first distribution probability p1 and the second distribution probability p2 are greater than the second preset probability threshold T2, it indicates that the data distribution has changed significantly, and the server (remote server) needs to retrain the liveness detection model deployed on the edge and send the trained target liveness detection model back to the edge so that the edge can obtain the target liveness detection model. By measuring the degree of change in sample distribution through the change in the distribution of knowledge of varying difficulty, and by adaptively updating the liveness detection model based on the degree of distribution change, the liveness detection model can maintain high performance, thereby improving the accuracy of liveness detection.

[0117] S130: Based on the attack probability, determine the liveness detection result of the target user and output the liveness detection result.

[0118] The liveness detection result can include either an attacking user or a normal user. An attacking user can be a user who uses non-liveness attack methods such as photos or screen captures. A normal user can be a live user, i.e., a normally living user who has not been attacked.

[0119] There are several ways to determine the liveness detection result of a target user based on the attack probability, including the following:

[0120] For example, the processor 620 can determine that the target user's liveness detection result is an attacking user when the attack probability is greater than a preset attack probability threshold, or determine that the target user's liveness detection result is a normal user when the attack probability is less than the preset attack probability threshold.

[0121] After determining the liveness detection result of the target user, the processor 620 can output the liveness detection result. There are several ways to output the liveness detection result. For example, the processor 620 can directly send the liveness detection result to the user 100 or the client 200, terminal or server corresponding to the target user, so that the client 200, terminal or server can respond to the request of the target user or the target user based on the liveness detection result. Alternatively, the liveness detection result can be directly visualized, and so on.

[0122] There are several ways to visualize the liveness detection results. For example, the processor 620 can directly display the liveness detection result, or it can display the liveness detection result through sound and light (for example, by broadcasting the liveness detection result by voice, or by displaying different types of liveness detection results by displaying different colored lights, or by displaying the liveness detection result through sound and light linkage), or it can display the liveness detection result for specific types of liveness detection results (for example, only displaying the liveness detection result for the attacking user type, or only displaying the liveness detection result for the live user type, etc.), and so on.

[0123] In some embodiments, after determining or outputting the liveness detection result of the target user, the processor 620 may respond to the target user or the request corresponding to the target user based on the liveness detection result. There may be various ways to respond. For example, the processor 620 may directly intercept the target user or the request corresponding to the target user, or the processor 620 may directly perform secondary verification on the target user and, based on the secondary verification result, provide a final response to the target user or the request corresponding to the target user, and so on.

[0124] In the context of face recognition, this solution can perform liveness detection on the edge based on a lightweight liveness detection model that decouples knowledge of difficulty. The overall liveness detection process can be as follows: Figure 4 As shown, it can include four parts: teacher model training, decoupling and distillation of easy and difficult knowledge, liveness detection, and edge-side periodic adaptation, which can be specifically as follows:

[0125] (1) Twin teacher model group training: Based on the mutual learning method of twin model groups, the performance of each teacher model is improved by mutual learning between two teacher models with the same structure, so as to obtain a single teacher model with better performance. The specific training process can be described as described above, and will not be repeated here.

[0126] (2) Decoupling Distillation of Easy and Difficult Knowledge: Traditional knowledge distillation treats the output of the teacher model as a whole, but each part of the knowledge has varying levels of difficulty, requiring gradual learning. In this scheme, the knowledge output of the teacher model is decoupled to obtain simple and difficult knowledge. Based on the distillation weights of simple and difficult knowledge, adaptive easy and difficult knowledge distillation is performed on the simple and difficult knowledge, resulting in a lighter, higher-performance liveness detection model. The specific decoupling distillation process can be found above and will not be repeated here. After obtaining the liveness detection model through easy and difficult knowledge decoupling distillation, the original distribution of easy and difficult knowledge in the training data can be determined. This mainly involves calculating the mean and variance of the simple and difficult knowledge in the training data, and using the mean and variance to characterize the simple and difficult knowledge with an external Gaussian distribution.

[0127] (3) Liveness Detection: A lightweight liveness detection model, trained on the device side, is deployed. After the target user begins face recognition, the target user's face image is acquired and input into the liveness detection model to obtain the target user's attack probability p. If p is greater than a pre-set attack probability threshold T, the target user can be determined to be an attacker; otherwise, the user is considered a live user.

[0128] (4) Edge-based periodic adaptation: Traditional edge-based liveness detection models experience performance fluctuations as the distribution of edge data changes (edge ​​data undergoes significant distribution changes over time). When the data distribution changes are small, adaptation can be completed directly on the edge. However, when the distribution changes are large, the server (remote server) needs to retrain the edge-based liveness detection model to complete the adaptation. In this solution, the degree of distribution change of edge-based samples is measured by the distribution change of easy and difficult knowledge, thereby determining the adaptive adaptation method. The adaptive adaptation method may specifically include: after the edge-based liveness detection model has run for a preset time period (e.g., one week, one month, six months, one year, or any time period), after accumulating K samples within the preset time, the probabilities of the K samples being in the simple knowledge Gaussian distribution and the difficult knowledge Gaussian distribution are calculated respectively, and the mean of the probabilities is calculated to obtain the simple knowledge probability p1 and the difficult knowledge probability p2. When p1 is less than the threshold T1, the liveness detection model on the client side does not need to be adaptively trained (adapted); when p2 is less than the threshold T2, the liveness detection model on the client side is adaptively trained (adapted) to obtain the updated target liveness detection model; when p1 is greater than T1 and p2 is greater than T2, it is determined that retraining is required on the server side (remote server), and the retrained target liveness detection model returned by the server side is received, and the liveness detection model deployed on the client side is updated to the target liveness detection model.

[0129] The process of adaptively training the liveness detection model on the edge can include: using the labels of N samples predicted by the edge liveness detection model, and using the sample-label pairs as input for training, thereby obtaining the target liveness detection model. N is any integer in [1,K].

[0130] The process of retraining the liveness detection model on the client side can include: receiving all or part of the K samples transmitted from the local device, manually labeling the received samples, training the liveness detection model based on the manually labeled label-sample pairs as input, and sending the trained target liveness detection model to the local device so that the local device can update the current liveness detection model on the client side to the target liveness detection model.

[0131] This scheme first trains the teacher model using mutual learning to obtain a pair of teacher models. Then, the pair of teacher models guides the student model (preset liveness detection model) to perform adaptive knowledge decoupling and distillation based on difficulty, resulting in a high-performance, lightweight liveness detection model. Finally, the characteristics of difficulty knowledge are used to perform hierarchical and periodic adaptive adaptation on the edge liveness detection model and perform liveness detection, thereby improving the performance and accuracy of liveness detection.

[0132] In summary, the liveness detection method P100 and system 001 provided in this specification, after obtaining the target user image, the processor 620 inputs the target user image into the liveness detection model to obtain the attack probability of the target user. The liveness detection model is a lightweight model obtained by decoupling and distilling easy and difficult knowledge based on a twin-teacher model group. Based on the attack probability, the liveness detection result of the target user is determined and output. Since this scheme guides the student model through a twin-teacher model group, and decouples and distills the easy and difficult knowledge during the guidance process to obtain a lightweight liveness detection model on the edge, the liveness detection performance on the edge can be improved through the lightweight liveness detection model, thus improving the accuracy of liveness detection.

[0133] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing liveness detection. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the liveness detection method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of the liveness detection method P100 described herein. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 600, partially on computing device 600, as a standalone software package, partially on computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0134] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0135] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0136] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0137] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0138] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0139] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for detecting liveness, comprising: Obtain the target user image of the target user; The target user image is input into a liveness detection model to obtain the attack probability of the target user. as well as Based on the attack probability, the liveness detection result of the target user is determined and the liveness detection result is output; wherein, the training process of the liveness detection model includes: A first user image sample is obtained and input into the teacher model to obtain teacher image features and predicted teacher category. The first user image sample, the teacher image features, and the teacher predicted category are input into a preset liveness detection model to obtain decoupling weights corresponding to student image features, student predicted categories, and easy / difficult knowledge. The easy / difficult knowledge includes simple knowledge and difficult knowledge, with simple knowledge being easier for the preset liveness detection model to learn than difficult knowledge. Both the teacher image features and the student image features include salient and insignificant features. The salient features correspond to the simple knowledge in the first user image sample, and the insignificant features correspond to the difficult knowledge in the first user image sample. The feature value of the salient features is greater than the feature value of the insignificant features. Based on the teacher image features, the teacher predicted category, the student image features, the student predicted category, and the decoupling weights, the preset liveness detection model is distilled to obtain the trained liveness detection model.

2. The liveness detection method according to claim 1, wherein, The preset liveness detection model includes a student network and a meta-network corresponding to the difficulty level of the knowledge; and The step of inputting the first user image sample, the teacher image features, and the teacher predicted category into a preset liveness detection model to obtain the decoupling weights corresponding to the student image features, the student predicted category, and the difficulty knowledge includes: The first user image sample is input into the student network to obtain the student image features and the predicted student category corresponding to the first user image sample. The teacher image features, the teacher prediction category, the student image features, and the student prediction category are input into the meta-network to obtain the decoupling weights corresponding to the difficulty knowledge.

3. The liveness detection method according to claim 1, wherein, The step of distilling the preset liveness detection model to obtain the trained liveness detection model includes: The teacher's predicted category is compared with the student's predicted category to obtain classification distillation loss information; Based on the decoupling weights, the teacher image features are compared with the student image features to obtain decoupling distillation loss information; and The classified distillation loss information and the decoupled distillation loss information are fused together, and the preset liveness detection model is converged based on the fused target distillation loss information to obtain the liveness detection model.

4. The live detection method according to claim 3, wherein, The step of comparing the teacher image features with the student image features to obtain decoupling distillation loss information includes: The salient features are extracted from the teacher image features and the student image features respectively to obtain the simple knowledge distillation loss information corresponding to the simple knowledge; The absolute values ​​of the teacher's image features and the absolute values ​​of the student's image features are compared to obtain the difficult knowledge distillation loss information corresponding to the difficult knowledge; and Based on the decoupling weights, the simple knowledge distillation loss information and the difficult knowledge distillation loss information are weighted and fused respectively to obtain the decoupling distillation loss information.

5. The liveness detection method according to claim 4, wherein, The step of extracting the salient features from the teacher's image features and the student's image features respectively to obtain the simple knowledge distillation loss information corresponding to the simple knowledge includes: The salient features are extracted from the teacher image features to obtain the teacher's basic knowledge corresponding to the teacher model; By extracting the salient features from the student image features, simple student knowledge corresponding to the student network is obtained; and The teacher's simple knowledge is compared with the student's simple knowledge to obtain the distillation loss information of the simple knowledge.

6. The liveness detection method according to claim 5, wherein, The teacher image features include multiple image sub-features; as well as The step of extracting salient features from the teacher image features to obtain the simple knowledge of the teacher corresponding to the teacher model includes: Determine the feature value of each of the plurality of image sub-features. Based on the feature values, the plurality of image sub-features are sorted from largest to smallest, and Based on the ranking results, image sub-features corresponding to a preset ranking range are selected from the multiple image sub-features to obtain the salient features, and the salient features are used as the teacher's simple knowledge.

7. The liveness detection method according to claim 6, wherein, The number of the plurality of image sub-features is N, and the preset sorting range is the first N / 2 image sub-features.

8. The liveness detection method according to claim 4, wherein, The step of comparing the absolute values ​​of the teacher image features with the absolute values ​​of the student image features to obtain the difficult knowledge distillation loss information corresponding to the difficult knowledge includes: Determine the absolute value of the teacher's image features to obtain teacher difficulty knowledge, and determine the absolute value of the student's image features to obtain student difficulty knowledge; Obtain the feature difference between the teacher's difficult knowledge and the student's difficult knowledge; and Based on the feature difference, the difficult knowledge distillation loss information corresponding to the difficult knowledge is determined.

9. The liveness detection method according to claim 1, wherein, After distilling the preset liveness detection model to obtain the trained liveness detection model, the method further includes: Obtain statistical characteristics of students' simple knowledge within the simple knowledge set, derive a first statistical characteristic, and determine a first feature distribution of the students' simple knowledge based on the first statistical characteristic; and Statistical characteristics of student difficult knowledge in the difficult knowledge are obtained, second statistical characteristics are obtained, and based on the second statistical characteristics, the second characteristic distribution of student difficult knowledge is determined.

10. The liveness detection method according to claim 9, wherein, The statistical characteristics include at least one of the mean and variance.

11. The liveness detection method according to claim 9, wherein, Also includes: When the running time of the liveness detection model reaches a preset time period, the data distribution probability of the user image set during the running time is statistically analyzed. The data distribution probability includes a first distribution probability that the user image in the user image set falls within the first feature distribution and a second distribution probability that it falls within the second feature distribution; and Based on the first probability distribution and the second probability distribution, one of the first, second, and third schemes is selected for execution, wherein: The first scheme includes: determining that the first distribution probability is less than a first preset probability threshold, and using the liveness detection model as the updated target liveness detection model. The second scheme includes: determining that the second distribution probability is less than a second preset probability threshold; training the liveness detection model based on the user image to obtain an updated target liveness detection model; and The third approach includes: determining that the first distribution probability is greater than the first preset probability threshold and the second distribution probability is greater than the second preset probability threshold, sending the user image to a remote server so that the remote server updates the liveness detection model and receives the updated target liveness detection model returned by the remote server.

12. The liveness detection method according to claim 11, wherein, The statistical distribution probability of the user image set during the running time includes: Obtain the set of user images within the specified runtime, and determine the first initial distribution probability and the second initial distribution probability of each user image in the set being in the first feature distribution and the second feature distribution, respectively. Determine the mean of the first initial probability distribution to obtain the first probability distribution, and determine the mean of the second initial probability distribution to obtain the second probability distribution; and The first probability distribution and the second probability distribution are used as the data probability distribution of the user image set.

13. The liveness detection method according to claim 1, wherein, The teacher model is a twin teacher model group, and the training process of the twin teacher model group includes the following steps: Obtain a second user image sample; The second user image sample is input into a preset twin-teacher model group to obtain the sample image features and predicted user category output by each teacher model in the preset twin-teacher model group; and Based on the sample image features and the predicted user category, the preset twin teacher model group is converged to obtain the trained twin teacher model group.

14. The liveness detection method according to claim 13, wherein, The preset twin teacher model group includes a first teacher model and a second teacher model with the same structure; as well as The process of converging the preset twin-teacher model group to obtain the trained twin-teacher model group includes: Obtain the labeled user category of the second user image sample, and compare the labeled user category with the predicted user category to obtain single classification loss information. The sample image features of the first teacher model are compared with the sample image features of the second teacher model to obtain feature complementarity loss information. The predicted user categories from the first teacher model and the predicted user categories from the second teacher model are fused to obtain fused classification loss information. The single classification loss information, the feature complementation loss information, and the fusion classification loss information are fused together, and the preset twin teacher model group is converged based on the fused target teacher loss information to obtain the twin teacher model group.

15. The liveness detection method according to claim 14, wherein, The step of comparing the sample image features of the first teacher model with the sample image features of the second teacher model to obtain feature complementarity loss information includes: Extract the first target sample image features output by each network layer of the first teacher model from the sample image features of the first teacher model; Extract the second target sample image features output by each network layer of the second teacher model from the sample image features of the second teacher model; and The similarity between the first target sample image features and the corresponding second target sample image features is determined to obtain the feature complementarity loss information of each network layer. The constraint condition of the feature complementarity loss information is that the similarity between the target sample image features of the corresponding network layers of different teacher models is less than a preset similarity threshold.

16. The liveness detection method according to claim 14, wherein, The step of fusing the predicted user categories from the first teacher model and the predicted user categories from the second teacher model to obtain fused classification loss information includes: Extract the first prediction probability corresponding to each candidate user category from the predicted user categories of the first teacher model; Extract the second prediction probability corresponding to each candidate user category from the predicted user categories in the second teacher model; Add the first predicted probability to the corresponding second predicted probability to obtain the target predicted probability for each candidate user category; and Based on the target prediction probability, the fused target user category is determined, and the target user category is compared with the labeled user category to obtain the fused classification loss information.

17. The liveness detection method according to claim 1, wherein, The step of determining the liveness detection result of the target user based on the attack probability includes: When the attack probability is greater than a preset attack probability threshold, the liveness detection result of the target user is determined to be the attacking user.

18. The liveness detection method according to claim 17, wherein, Also includes: When the attack probability is less than the preset attack probability threshold, the liveness detection result of the target user is determined to be that of a normal user.

19. A liveness detection system, comprising: At least one storage medium storing at least one instruction set for performing liveness detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the liveness detection method according to any one of claims 1-18.

Images