Liveness detection method and system

Abstract

The present specification provides a kind of live detection method and system, target terminal equipment is used to the target image of target user feature extraction, and the target feature extracted is sent to server, and live detection is carried out by server, so that the overall operation of live detection is divided into two parts, the part of small operation amount is carried out on target terminal equipment, and the part of large operation amount is carried out on server, so as to improve the performance of live detection. At the same time, the feature of target image is transmitted between target terminal equipment and server instead of target image, so as to improve security.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a liveness detection method and system. Background Technology

[0002] Facial recognition is a primary means of user authentication. With the widespread application of facial recognition systems, their security has been challenged by liveness detection attacks. Therefore, it is necessary to add a liveness detection step to facial recognition systems. Currently, there are two approaches to liveness detection. One approach involves the terminal device capturing facial images and transmitting them to the cloud, then using a high-performance liveness detection model deployed in the cloud to perform liveness detection on the facial images, and finally returning the results to the terminal device. This approach requires facial image transmission between the terminal device and the cloud, posing a risk of user privacy leakage. The other approach involves the terminal device capturing facial images and using a lightweight liveness detection model deployed on the terminal device to perform liveness detection. This approach is limited by the computing power and storage capacity of the terminal device, resulting in poor liveness detection performance. Therefore, there is a need to provide a liveness detection method and system that can simultaneously balance security and liveness detection performance. Summary of the Invention

[0003] This specification provides a liveness detection method and system that can improve the performance and safety of liveness detection.

[0004] In a first aspect, this specification provides a liveness detection method applied to a server. The method includes: receiving target features sent by a target terminal device, the target features being obtained by the target terminal device through feature extraction based on a target image of a target user; performing liveness detection based on the target features to obtain a first liveness detection result; determining a target liveness detection result based at least on the first liveness detection result; and outputting the target liveness detection result.

[0005] In some embodiments, the target feature includes: a first feature, including features obtained by extracting features from the target image based on a feature extraction model, wherein the feature extraction model is deployed on the target terminal device; or a second feature, including compressed features obtained by compressing the first feature.

[0006] In some embodiments, the step of performing liveness detection based on the target features to obtain a first liveness detection result includes: determining target liveness detection features based on the target features; and performing liveness detection on the target liveness detection features based on a liveness classification model to obtain the first liveness detection result, wherein the liveness classification model is deployed on the server.

[0007] In some embodiments, determining the target liveness detection feature based on the target feature includes: using the first feature as the target liveness detection feature; or decompressing the second feature to obtain a decompressed feature, and using the decompressed feature as the target liveness detection feature.

[0008] In some embodiments, the feature extraction model and the liveness classification model are trained simultaneously, and the training objective includes the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range.

[0009] In some embodiments, the training objective further includes the difference between the predicted image and its corresponding original image within a second preset range, wherein the predicted image includes an image reconstructed based on the predicted features output by the feature extraction model.

[0010] In some embodiments, the second feature includes N compressed features obtained by compressing the first feature based on N different compression models, where N is a positive integer, and the N compression models are deployed on the target terminal device; the step of decompressing the second feature to obtain decompressed features and using the decompressed features as the target liveness detection feature includes: decompressing the N compressed features based on the N decompression models to obtain N decompressed features, and using the N decompressed features as the target liveness detection feature, wherein the N decompression models correspond to the N compressed models, and the N decompression models are deployed on the server.

[0011] In some embodiments, the N compression models and the N decompression models are trained simultaneously, and the training objective includes ensuring that the difference between the N predicted decompression features output by the N decompression models and the first feature is within a third preset range.

[0012] In some embodiments, the training objective further includes at least one of the following: the number of non-zero elements in the N predicted compression features output by the N compression models is within a fourth preset range; and the difference between the predicted image reconstructed based on the N predicted compression features and its corresponding original training image is greater than a preset first difference threshold.

[0013] In some embodiments, the second feature includes a compressed fusion feature obtained by fusing N compressed features based on a fusion model. The N compressed features are obtained by compressing the first feature based on N different compression models, where N is an integer greater than 1. The N compression models and the fusion model are deployed on the target terminal device. Decompressing the second feature to obtain a decompressed feature and using the decompressed feature as the target liveness detection feature includes: decompressing the compressed fusion feature based on the fusion decompression model and using the decompressed fusion feature as the target liveness detection feature. The fusion decompression model is deployed on the server.

[0014] In some embodiments, the N compression models, the fusion model, and the fusion decompression model are trained simultaneously, and the training objective includes ensuring that the difference between the predicted decompression fusion feature output by the fusion decompression model and the first feature is within a fifth preset range.

[0015] In some embodiments, the training objective further includes at least one of the following: the difference between the N predicted decompression features obtained by decompressing the N predicted compression features output by the N compression models and the first feature is within a sixth preset range; the number of non-zero elements in the N predicted compression features is within a seventh preset range; the number of non-zero elements in the predicted fusion features output by the fusion model is within an eighth preset range; and the difference between the predicted image reconstructed based on the predicted fusion features and its corresponding original training image is greater than a preset second difference threshold.

[0016] In some embodiments, determining the target liveness detection result based at least on the first liveness detection result includes: determining a second liveness detection result; and determining the target liveness detection result based on the first liveness detection result and the second liveness detection result.

[0017] In some embodiments, determining the second liveness detection result includes: acquiring M risk features corresponding to M risk images, wherein the M risk images include images of attack targets whose liveness detection difficulty is higher than a preset threshold, and M is an integer greater than 0; determining M similarities between the target features and the M risk features; and determining the second liveness detection result based on the M similarities.

[0018] In some embodiments, determining the second liveness detection result based on the M similarities includes: determining that at least one of the M similarities is greater than a preset similarity threshold, and determining the second liveness detection result as an attack category; or determining that all M similarities are less than the similarity threshold, and determining the second liveness detection result as a liveness category.

[0019] In some embodiments, the M risk features include features obtained by at least one terminal device through feature extraction based on the M risk images, and the at least one terminal device includes the target terminal device.

[0020] In some embodiments, the first liveness detection result includes the liveness category or the attack category, and the step of determining the target liveness detection result based on the first liveness detection result and the second liveness detection result includes: determining that at least one of the first liveness detection result and the second liveness detection result is the attack category, and determining the target liveness detection result as the attack category; or determining that both the first liveness detection result and the second liveness detection result are the liveness category, and determining the target liveness detection result as the liveness category.

[0021] Secondly, this specification also provides a liveness detection system, including a server, the server comprising: at least one storage medium storing at least one instruction set for performing liveness detection; and at least one processor communicatively connected to the at least one storage medium, wherein, when the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the method described in any of the first aspects according to the instructions of the at least one instruction set.

[0022] Thirdly, this specification also provides a liveness detection method applied to a target terminal device. The method includes: acquiring a target image of a target user; determining target features corresponding to the target image; and sending the target features to a server so that the server performs liveness detection based on the target features to obtain a target liveness detection result.

[0023] In some embodiments, determining the target features corresponding to the target image includes: extracting features from the target image using a deployed feature extraction model to obtain a first feature; and determining the target features based on the first feature; wherein the server performs liveness detection on the target features based on a deployed liveness classification model, and the feature extraction model and the liveness classification model are trained simultaneously.

[0024] In some embodiments, determining the target feature based on the first feature includes: using the first feature as the target feature; or compressing the first feature to obtain a second feature, and using the second feature as the target feature.

[0025] In some embodiments, compressing the first feature to obtain the second feature includes: compressing the first feature using N deployed compression models to obtain N compressed features; and determining the N compressed features as the second feature; wherein the server decompresses the N compressed features based on the N deployed decompression models, and performs liveness detection based on the N decompressed features, wherein the N compression models and the N decompression models are trained simultaneously.

[0026] In some embodiments, compressing the first feature to obtain the second feature includes: compressing the first feature using N deployed compression models to obtain N compressed features; fusing the N compressed features using a deployed fusion model to obtain a compressed fused feature; and determining the compressed fused feature as the second feature; wherein the server decompresses the compressed fused feature based on a deployed fusion decompression model and performs liveness detection based on the decompressed fusion decompression feature, and the N compression models, the fusion model, and the fusion decompression model are trained simultaneously.

[0027] Fourthly, this specification also provides a liveness detection system, comprising: at least one storage medium storing at least one instruction set for performing liveness detection; and at least one processor communicatively connected to the at least one storage medium, wherein, when the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the method described in any of the second aspects according to the instructions of the at least one instruction set.

[0028] As can be seen from the above technical solutions, the liveness detection method and system provided in this specification involve the target terminal device extracting features from the target user's target image, determining target features based on the extracted first features, and sending the target features to the server so that the server can perform liveness detection based on the target features. Since this solution assigns the computationally less demanding task of feature extraction to the target terminal device and the computationally more demanding task of liveness classification to the server, and since the target terminal device transmits features to the server, the difficulty of reconstructing the target image from these features is relatively high, thus improving the privacy protection capability of the target image. Furthermore, the server possesses high computing power, and assigning the liveness classification task to the server also ensures the performance of liveness detection. In summary, this solution can simultaneously balance security and liveness detection performance.

[0029] Other functions of the liveness detection methods and systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the liveness detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram illustrating an application scenario of a liveness detection system provided according to an embodiment of this specification is shown.

[0032] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0033] Figure 3 A flowchart of a liveness detection method provided according to an embodiment of this specification is shown;

[0034] Figure 4 A schematic diagram illustrating the data flow of compressing and decompressing a first feature according to an embodiment of this specification is shown; and

[0035] Figure 5 A schematic diagram illustrating another data flow for compressing and decompressing a first feature according to an embodiment of this specification is shown. Detailed Implementation

[0036] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0037] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0038] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0039] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0040] For ease of description, the terms used in this specification are explained as follows:

[0041] Liveness attack: refers to an attack method that uses screens, paper, masks, etc. to attempt to bypass facial recognition systems.

[0042] Liveness detection: also known as liveness protection, refers to the technology that uses artificial intelligence models to detect and block liveness attacks, such as those on mobile phone screens, printed paper, etc.

[0043] Privacy protection: In this solution, the liveness detection phase involves compressing and protecting the user's facial image and features containing facial information.

[0044] Heterogeneous Expert Group: In this scheme, it refers to using multiple models with different structures for feature compression (privacy protection) to make the final privacy protection capability stronger.

[0045] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0046] The liveness detection method provided in this specification can be applied to any liveness detection scenario in biometric processes. For example, in scenarios such as face payment or face recognition, the liveness detection method can be used to perform liveness detection on the original image of the biometric features of the user to be paid or identified. In identity verification scenarios, the liveness detection method can also be used to perform liveness detection on the original image of the user's biometric features. The liveness detection method can also be applied to other liveness detection scenarios, which will not be elaborated here. The biometric features may include, but are not limited to, one or more of the following: facial image, iris, sclera, fingerprint, palm print, voiceprint, and skeletal projection. For ease of description, this application will use the application of the liveness detection method in a face recognition scenario to perform liveness detection on a face as an example.

[0047] Those skilled in the art should understand that the liveness detection methods and systems described in this specification are also within the scope of protection of this specification when applied to other application scenarios.

[0048] Figure 1 This diagram illustrates an application scenario of a liveness detection system 001 provided according to an embodiment of this specification. The liveness detection system 001 (hereinafter referred to as System 001) can be applied to liveness detection in any scenario, such as liveness detection in face payment scenarios, liveness detection in identity verification scenarios, liveness detection in other face recognition scenarios, etc. Figure 1 As shown, system 001 may include target user 100, target terminal device 200, server 300 and network 400.

[0049] Target user 100 can be a user who needs to undergo biometric identification, or a user who is currently undergoing biometric identification. Target user 100 can be the object detected by system 001. Target user 100 can initiate a biometric identification process, thereby triggering liveness detection of target user 100.

[0050] The target terminal device 200 can be a device for performing liveness detection on the target user 100 in response to a liveness detection operation by the target user 100. In some embodiments, the liveness detection method can be executed on the target terminal device 200. In this case, the target terminal device 200 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute the data or instructions. In some embodiments, the target terminal device 200 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work. In some embodiments, the target terminal device 200 may include a mobile device, tablet computer, laptop computer, built-in device of a motor vehicle, or similar content, or any combination thereof. In some embodiments, the mobile device may include a smart home device, smart mobile device, virtual reality device, augmented reality device, or similar device, or any combination thereof. In some embodiments, the smart home device may include a smart TV, desktop computer, etc., or any combination thereof. In some embodiments, the smart mobile device may include a smartphone, personal digital assistant, gaming device, navigation device, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include a virtual reality headset, virtual reality glasses, virtual reality patch, augmented reality headset, augmented reality glasses, augmented reality patch, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, a head-mounted display, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle television, etc. In some embodiments, the target terminal device 200 may include an image acquisition device. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (e.g., an RGB camera), or it may be a two-dimensional image acquisition device (e.g., an RGB camera) and a depth image acquisition device (e.g., a 3D structured light camera, a laser detector, etc.). In some embodiments, the target terminal device 200 may be a device with positioning technology for locating the position of the target terminal device 200.

[0051] In some embodiments, the target terminal device 200 may have one or more applications (APPs) installed. The APPs provide the target user 100 with the ability and interface to interact with the outside world via the network 400. The APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email terminals, social media platform software, etc. In some embodiments, the target terminal device 200 may have a target APP installed. The target APP can instruct the target terminal device 200 to collect target images of the target user 100's biometric features, such as facial images. The target images can be used for liveness detection. In some embodiments, the target object 100 can also trigger a liveness detection request through the target APP. The target APP can respond to the liveness detection request by executing the liveness detection method described in this specification. The liveness detection method will be described in detail later.

[0052] like Figure 1 As shown, the target terminal device 200 can communicate with the server 300. In some embodiments, the server 300 can communicate with multiple terminal devices. The multiple terminal devices may include the target terminal device 200. In some embodiments, the multiple terminal devices 200 can interact with the server 300 through the network 400 to receive or send messages, etc. The server 300 may be a server that provides various services corresponding to the target APP, such as a backend server that supports liveness detection of target images of target users collected on the target terminal device 200. In some embodiments, the liveness detection method can be executed on the server 300. In this case, the server 300 may store data or instructions for executing the liveness detection method described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, the server 300 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work.

[0053] Network 400 serves as a medium to provide a communication connection between target terminal device 200 and server 300. Network 400 can facilitate the exchange of information or data. For example... Figure 1As shown, the target terminal device 200 and the server 300 can be connected to the network 400 and transmit information or data to each other through the network 400. In some embodiments, the network 400 can be any type of wired or wireless network, or a combination thereof. For example, the network 400 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), or a Bluetooth network. TM ZigBee TM A network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of target terminal device 200 and server 300 can connect to network 400 to exchange data or information.

[0054] It should be understood that Figure 1 The number of target terminal devices 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, there can be any number of target terminal devices 200, servers 300, and networks 400.

[0055] Figure 2 A hardware structure diagram of a computing device 600 provided according to an embodiment of this specification is shown. The computing device 600 can execute the liveness detection method described in this specification. The liveness detection method is described in other parts of this specification. When the liveness detection method is executed on a target terminal device 200, the computing device 600 can be the target terminal device 200. When the liveness detection method is executed on a server 300, the computing device 600 can be the server 300. When the liveness detection method can be executed partly on the target terminal device 200 and partly on the server 300, the computing device 600 can be both the target terminal device 200 and the server 300.

[0056] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0057] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0058] I / O component 660 supports input / output between computing device 600 and other components.

[0059] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 400. Communication port 650 can be a wired communication port or a wireless communication port.

[0060] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the liveness detection method provided in this specification.

[0061] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. At least one processor 620 is used to execute the at least one instruction set described above. When the computing device 600 is running, at least one processor 620 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the liveness detection method provided in this specification. Processor 620 can execute all the steps included in the liveness detection method. Processor 620 can be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0062] Figure 3 A flowchart of a liveness detection method P100 according to an embodiment of this specification is shown. As previously described, the computing device 600 can execute the liveness detection method P100 of this specification. Specifically, the computing device 600 can read an instruction set stored in its local storage medium and then execute the liveness detection method P100 of this specification according to the instructions in the instruction set. Figure 3 As shown, method P100 may include:

[0063] S120: The target terminal device 200 acquires the target image of the target user.

[0064] The target users are those who are about to undergo liveness detection or are currently undergoing liveness detection.

[0065] The target image includes the biometric features of the target user. Biometric features are inherent physiological characteristics of the human body, and may include at least one of the following: face, iris, sclera, fingerprint, palm print, voiceprint, and skeletal projection; or other inherent physiological characteristics of the human body capable of facial recognition. For ease of description, this specification will use the face as an example of biometric features. Those skilled in the art should understand that other biometric features are also within the scope of this specification.

[0066] The target image can be acquired by the target terminal device 200. The target terminal device 200 can be any of the aforementioned terminal devices. The target terminal device 200 can be a device for performing liveness verification or facial verification. The target user can complete the login on the target terminal device 200 by performing liveness verification or facial verification on the target terminal device 200.

[0067] The target terminal device 200 integrates an image acquisition module. When a liveness verification or face verification request from the target user is triggered, the target terminal device 200 controls the image acquisition module to acquire an original image containing the target user's biometric features, or retrieves the original image from the storage address based on a received liveness detection request carrying the storage address of the original image containing the target user's biometric features. The original image may contain the user's biometric features, i.e., the liveness detection object. The original image may be an image including all or part of the user's body parts, acquired by the image acquisition module.

[0068] After acquiring the original image, the target terminal device 200 can further preprocess the original image to obtain the target image. Preprocessing could include face detection; for example, performing face detection on the original image to obtain the face regions within it.

[0069] S140: The target terminal device 200 determines the target features corresponding to the target image.

[0070] Biometrics are crucial information used for liveness detection. Taking facial recognition as an example, it's a biometric technology that identifies individuals based on their facial features. Liveness detection primarily works by recognizing the physiological characteristics of a living person, using these characteristics as biometrics to distinguish them from fake biometrics created using non-biological materials such as photographs, silicone, or plastic. When biometric information is obtained from a legitimate user, it can be identified as originating from a legitimate user with a living body.

[0071] The target feature refers to the biometric features corresponding to the target user in the target image, which may include a first feature or a second feature. The first feature includes features extracted from the target image based on the feature extraction model deployed on the target terminal device 200. The second feature includes compressed features obtained by compressing the first feature.

[0072] When determining the target features corresponding to a target image, the target terminal device 200 can use a feature extraction model deployed on the target terminal device 200 to extract features from the target image, obtain a first feature, and determine the target features based on the first feature. The feature extraction model deployed on the target terminal device 200 is configured to extract features from the image. The target terminal device 200 inputs the target image into the feature extraction model, and the feature extraction model then extracts features from the target image to obtain the first feature.

[0073] After acquiring the first feature, the target terminal device 200 can determine the target feature based on the first feature. In some embodiments, determining the target feature based on the first feature may include the target terminal device 200 directly using the first feature as the target feature. In some embodiments, determining the target feature based on the first feature may include the target terminal device 200 compressing the first feature to obtain a compressed feature (i.e., a second feature) and using the second feature as the target feature. There are various ways for the target terminal device 200 to compress the first feature to obtain the second feature.

[0074] In some embodiments, the process of the target terminal device 200 compressing the first feature to obtain the second feature may include: the target terminal device 200 compressing the first feature using N compression models deployed on the target terminal device 200 to obtain N compressed features, and determining the N compressed features as the second feature, where N is a positive integer.

[0075] Figure 4 A schematic diagram illustrating the data flow of compressing and decompressing a first feature according to an embodiment of this specification is shown. Figure 4 As shown, the N different compression models can be denoted as compression model 1, compression model 2, ..., compression model N. Each of the N different compression models is configured to compress the first feature. The target terminal device 200 inputs the first feature into each of the N different compression models, allowing each model to compress the first feature, resulting in N compressed features.

[0076] When N is 1, N compression models form one compression model, which compresses the first feature to obtain one compressed feature, such as compressed feature 1, compressed feature 2, ..., or compressed feature N. When N is an integer greater than 1, at least some of the N compression models can have different structures. Furthermore, when all N compression models have different structures, the target terminal device 200 can use N different compression models to compress the first feature, resulting in N different compressed features, such as compressed feature 1, compressed feature 2, ..., compressed feature N.

[0077] When the target terminal device 200 compresses the first feature using N compression models, it can employ either parallel processing or non-parallel processing. Parallel processing means that the first feature is compressed using N compression models separately and simultaneously. Non-parallel processing can be as follows: for example, the target terminal device 200 processes the first feature using some of the N compression models, and then processes the first feature again using the remaining N compression models; that is, the N compression models do not compress the first feature simultaneously.

[0078] In some embodiments, the process of the target terminal device 200 compressing the first feature to obtain the second feature may further include: compressing the first feature using N compression models deployed on the target terminal device 200 to obtain N compressed features, fusing the N compressed features using a fusion model deployed on the target terminal device 200 to obtain a compressed fused feature, and determining the compressed fused feature as the second feature.

[0079] Figure 5 A schematic diagram illustrating another data flow diagram for compressing and decompressing the first feature according to an embodiment of this specification is shown. For example... Figure 5 As shown, the target terminal device 200 still uses N compression models to compress the first feature separately to obtain N compressed features. The process of the target terminal device 200 compressing the first feature using N compression models can be found in [link to documentation]. Figure 4 This is only a partial introduction; only a partial introduction is given here. Figure 5 Compared to Figure 4 The differences. Compared to Figure 4 , Figure 5 A fusion model has been added, and the fusion model is connected to N compressed models. The target terminal device 200 can input N compressed features into the fusion model, so that the fusion model can perform fusion processing on the N features and determine the compressed fusion feature obtained after fusion processing as the target feature.

[0080] Continue reading Figure 3After step S140, the method P100 may further include step S160.

[0081] S160: The target terminal device 200 sends the target features to the server 300, and the server 300 receives the target features sent by the target terminal device 200.

[0082] To improve the accuracy of liveness detection, method P100 can set the computationally intensive liveness detection calculation process on server 300, and use the high cloud computing power and storage capacity of server 300 to perform liveness detection calculation, thereby improving the accuracy and performance of liveness detection.

[0083] After determining the target features, the target terminal device 200 can send the target features to the server 300 so that the server 300 can perform liveness detection based on the target features, thereby improving the liveness detection performance.

[0084] Continue reading Figure 3 After step S160, the method P100 may further include step S180.

[0085] S180: Server 300 performs liveness detection based on target features and obtains the first liveness detection result.

[0086] After receiving the target features, server 300 can perform liveness detection based on the target features to obtain a first liveness detection result. This process can include: server 300 determining target liveness detection features based on the target features, and performing liveness detection on the target liveness detection features based on a liveness classification model deployed on server 300 to obtain the first liveness detection result.

[0087] As mentioned above, the target features may include a first feature or a second feature. Correspondingly, when the server 300 determines the target liveness detection features based on the target features, it may include the following implementation methods:

[0088] In some embodiments, when the target feature includes the first feature, the server 300 can directly use the first feature as the target liveness detection feature and use the liveness classification model deployed on the server 300 to perform liveness detection on the first feature to obtain the first liveness detection result.

[0089] As mentioned above, the target terminal device 200 can use its deployed feature extraction model to extract features from the target image to obtain a first feature, which is then sent to the server 300 as the target feature. Correspondingly, the server 300 can input the first feature into the liveness classification model to obtain a first liveness detection result. The feature extraction model and the liveness classification model can be trained simultaneously. In some embodiments, the training objective of the feature extraction model and the liveness classification model may include the difference between the predicted liveness classification result and its corresponding true liveness classification result within a first preset range. The specific training process can be as follows:

[0090] For example, server 300 acquires the original training image and its corresponding first label, and based on the original training image and its corresponding first label, with the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range as the training objective, trains the preset feature extraction model and the preset liveness classification model to obtain the feature extraction model and the liveness classification model.

[0091] The original training images include multiple training image samples, each containing the biometric features of the training user. For details on biometric features, please refer to the preceding description; they will not be repeated here. Server 300 can receive images containing user biometric features sent by target terminal device 200 as original training images, or it can obtain original training images from a public dataset. Alternatively, some training image samples can be obtained from target terminal device 200, while others can be obtained from a public dataset. This specification does not impose any limitations on this.

[0092] The first label indicates whether the liveness classification corresponding to the original training image is liveness category or attack category. The label corresponding to the original training image can be obtained by manual labeling, model annotation, etc., and this specification does not impose any restrictions on this.

[0093] After acquiring the original training images and their corresponding first labels, server 300 can train a preset feature extraction model and a preset liveness classification model based on the original training images, their corresponding first labels, and a first comprehensive loss. For example, server 300 uses the preset feature extraction model to extract features from the original training images to obtain the first predicted features; it then uses the preset liveness classification model to classify the first predicted features for liveness detection to obtain the predicted liveness classification result; and based on the difference between the predicted liveness detection result and the first label, it determines the first comprehensive loss, and then converges the preset feature extraction model and the preset liveness classification model based on the first comprehensive loss to obtain the feature extraction model and the liveness classification model.

[0094] The preset feature extraction model can be a lightweight network including multiple residual blocks, such as three residual blocks. After the server 300 inputs the original training image into the preset feature extraction model, it can obtain the first predicted feature. Then, the server 300 inputs the first predicted feature into the preset liveness classification model to obtain the predicted liveness classification result.

[0095] After obtaining the predicted liveness classification result, server 300 can determine the first comprehensive loss based on the difference between the predicted liveness classification result and the first label, and adjust the parameters of the preset feature extraction model and the preset liveness classification model based on the first comprehensive loss, so that the outputs of the preset feature extraction model and the preset liveness classification model meet the training objective. The determination of the training objective can include the following implementation methods:

[0096] In some embodiments, the training objective of the training process for the feature extraction model and the liveness classification model may further include the difference between the predicted image and its corresponding original image within a second preset range, wherein the predicted image includes an image reconstructed based on the predicted features output by the feature extraction model. The specific training process can be as follows:

[0097] For example, server 300 acquires the original training image and its corresponding first label, and based on the original training image and its corresponding first label, with the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range, and the difference between the predicted image and its corresponding original image within a second preset range as the training target, trains the preset feature extraction model and the preset liveness classification model to obtain the feature extraction model and the liveness classification model.

[0098] For a description of the original training images and the first label, please refer to the foregoing description; it will not be repeated here. The difference from the previous embodiment is that the training objective in this embodiment includes not only the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range, but also the difference between the predicted image and its corresponding original image within a second preset range. For the training objective including the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range, please refer to the foregoing description; here, we only describe the difference between the predicted image and its corresponding original image within a second preset range. For example, the preset first image reconstruction model can be a UNET network, etc., which is configured to reconstruct the image corresponding to the feature based on the feature. After obtaining the first predicted feature, the server 300 can use the preset first image reconstruction model to reconstruct the image of the first predicted feature, obtaining the predicted image corresponding to the first predicted feature. A first comprehensive loss is determined based on the weighted sum of the difference between the predicted liveness detection result and the first label, and the difference between the predicted image and the original training image. The preset feature extraction model, the preset liveness classification model, and the preset first image reconstruction model are then converged based on the first comprehensive loss. And determine the converged preset feature extraction model as the trained feature extraction model, and determine the converged preset liveness classification model as the trained liveness classification model.

[0099] The first comprehensive loss can include a liveness classification loss and an image reconstruction loss. The liveness classification loss can be determined based on the difference between the predicted liveness detection result and the first label. The image reconstruction loss can be determined based on the difference between the predicted image output by the preset first image reconstruction model and the original training image. The image reconstruction loss ensures that the biometric information of the training user in the original training image is encoded in the first predicted feature. In other words, the image reconstruction loss ensures that the training effect of the preset feature extraction model achieves richer feature information from the target image, thereby guaranteeing the liveness detection performance of the liveness classification model.

[0100] In the above embodiments, the convergence conditions may include: the first comprehensive loss being less than a first comprehensive preset range, or the number of iterations reaching a preset number, or the accuracy of the trained model reaching a preset accuracy, etc. The first comprehensive preset range can be determined based on the sum of a first preset range and a second preset range.

[0101] Upon completion of training, the trained feature extraction model and liveness classification model are obtained. Server 300 deploys the trained feature extraction model on the target terminal device 200 to extract the first feature from the target image. The liveness classification model is then deployed on server 300 to perform liveness detection based on the first feature.

[0102] In some embodiments, when the target feature includes a second feature, the server 300 can decompress the second feature to obtain a decompressed feature, and use the decompressed feature as the target liveness detection feature. The server 300's decompression of the second feature can also include the following implementation methods:

[0103] As mentioned above, in some embodiments, the second feature may include N compressed features obtained by compressing the first feature based on N different compression models. Corresponding to the N compression models, the N compression models can be data-connected with N decompression models. When the server 300 obtains the second feature, the server 300 can decompress the second feature using the N decompression models deployed on the server 300 to obtain N decompressed features, and use the N decompressed features as the target liveness detection features. (Continue reading...) Figure 4 Server 300 inputs compression feature 1, compression feature 2, ..., compression feature N into the corresponding decompression model. For example, compression feature 1 is input into decompression model 1, compression feature 2 is input into decompression model 2, ..., compression feature N is input into decompression model N, and decompression feature 1, decompression feature 2, ..., decompression feature N can be obtained. Figure 4 The N compression models and N decompression models in the model can be understood as a heterogeneous expert group model. This heterogeneous expert group model includes N encoder-decoder structures with different characteristics.

[0104] In this model, N compression models and N decompression models can be trained simultaneously. In some embodiments, the training objective of the N compression and N decompression models may include ensuring that the difference between the N predicted decompression features output by the N decompression models and their corresponding original features before compression is within a third preset range. The training objective may also include at least one of the following: the number of non-zero elements in the N predicted compression features output by the N compression models is within a fourth preset range; and the difference between the predicted image reconstructed based on the N predicted compression features and its corresponding original training image is greater than a preset first difference threshold. The specific training process can be as follows:

[0105] Continuing to refer to the training process of the feature extraction model and the liveness classification model, after the server 300 extracts the first predicted feature from the original training image using the preset feature extraction model, it can use N different preset compression models to compress the first predicted feature to obtain N predicted compressed features, and use N preset decompression models corresponding to the N preset compression models to decompress the N predicted compressed features to obtain N predicted decompressed features.

[0106] After obtaining N predicted decompression features, the server 300 can determine the second comprehensive loss based on the N predicted decompression features and their corresponding original features before compression, and adjust the parameters of the N preset compression models and N preset decompression models based on the second comprehensive loss so that the outputs of the N preset compression models and N preset decompression models meet the training objective.

[0107] The second comprehensive loss may include the first feature map reconstruction loss, Loss_feat1. The first feature map reconstruction loss, Loss_feat1, can be determined based on the differences between the N predicted decompression features output by the N decompression models and their corresponding original features before compression. For example, for each of the N predicted decompression features, the server 300 can determine the sub-feature map reconstruction loss corresponding to that predicted decompression feature based on the difference between that predicted decompression feature and its corresponding original features before compression. After performing the above steps on all N predicted decompression features, the server 300 can obtain N sub-feature map reconstruction losses. Then, the server 300 performs a weighted sum of the N sub-feature map reconstruction losses to obtain the first feature map reconstruction loss, Loss_feat1.

[0108] The second comprehensive loss may also include at least one of the first volume compression loss Loss_compression1 and the first anti-reverse engineering loss Loss_anti1. The first volume compression loss Loss_compression1 can be determined based on the number of non-zero elements in the N predicted compressed features output by the N compressed models. That is, the number of non-zero elements in each of the N predicted compressed features should be as small as possible. For example, server 300 calculates the L1 norm for each of the N predicted compressed features and determines the first volume compression loss Loss_compression1 by the weighted sum of the N L1 norms of the N predicted compressed features. Since the transmission of feature maps will occupy a large amount of bandwidth, it is not friendly in weak network environments. The first volume compression loss Loss_compression1 aims to constrain the training process of the N preset compressed models so that when the trained N compressed models compress the first feature, they can greatly reduce the volume of the first feature, improve the transmission efficiency of the target feature, and thus improve the efficiency of liveness detection. Of course, the server 300 can also determine the number of non-zero elements in the N predicted compression features output by the N compression models based on the L0 norm, L2 norm, etc., and this specification does not impose any restrictions on this.

[0109] The first anti-reverse engineering loss, Loss_anti1, can be determined based on the difference between the predicted image reconstructed from N predicted compressed features and its corresponding original image. For example, the fusion model can be connected to a preset second image reconstruction model. The preset second image reconstruction model can be a network like UNET, configured to perform feature-based image reconstruction. Server 300 inputs N predicted compressed features into the preset second image reconstruction model, which then reconstructs the image based on these N features to obtain the predicted image. Loss_anti1 aims to maximize the difference between the predicted image output by the preset second image reconstruction model and its corresponding original training image. This prevents attackers from reconstructing the target image through feature map reverse engineering and using it to attack the target user, causing losses and enhancing the defense against reverse engineering attacks. The preset second image reconstruction model can reconstruct images based on N predictive compression features. The implementation of obtaining the predicted image can be as follows: For example, the preset second image reconstruction model can reconstruct images based on N predictive compression features, obtain N sub-predicted images, and perform weighted summation on the N sub-predicted images to obtain the predicted image corresponding to the N predictive compression features.

[0110] It's important to note that when the original training images include risky images, no additional constraints are applied to these risky images in the second anti-reverse engineering loss. In other words, the second anti-reverse engineering loss, Loss_anti1, is determined by the difference between the predicted image reconstructed from the N predicted compressed features corresponding to the non-risk original training image and the original non-risk training image. This does not involve the second anti-reverse engineering loss for risky images, which enhances the detection capability against high-risk attacks.

[0111] After determining at least one of the first feature map reconstruction loss Loss_feat1, the first volume compression loss Loss_compression1, and the first reverse engineering loss Loss_anti1, server 300 can determine the second comprehensive loss based on the weighted sum of the first feature map reconstruction loss Loss_feat1 and at least one of the first volume compression loss Loss_compression1 and the first reverse engineering loss Loss_anti1. That is, the second comprehensive loss may include the first feature map reconstruction loss Loss_feat1, or it may include the first feature map reconstruction loss Loss_feat1 and the first volume compression loss Loss_compression1, or it may include the first feature map reconstruction loss Loss_feat1 and the first reverse engineering loss Loss_anti1, or it may include the first feature map reconstruction loss Loss_feat1, the first volume compression loss Loss_compression1, and the first reverse engineering loss Loss_anti1. Then, server 300 can converge the N preset compression models, N preset decompression models and preset second reconstruction models based on the second comprehensive loss, and determine the converged N preset compression models as the trained N compression models, and determine the converged N preset decompression models as the trained N decompression models.

[0112] In the above embodiments, the convergence conditions may include: the second comprehensive loss being less than a second comprehensive preset range, or the number of iterations reaching a preset number, or the accuracy of the trained model reaching a preset accuracy, etc. The second comprehensive preset range can be determined based on at least one of a third preset range, a fourth preset range, and a preset first difference threshold. For example, when the second comprehensive preset range is determined based on the third preset range, the fourth preset range, and the preset first difference threshold, the server 300 can define the interval between the preset first difference threshold and a preset value (e.g., 1) as the preset first difference range. The lower limit of the third preset range, the fourth preset range, and the preset first difference range is summed to obtain the lower limit of the second comprehensive preset range, and the upper limit of the third preset range, the fourth preset range, and the preset value are summed to obtain the upper limit of the second comprehensive preset range, thus obtaining the second comprehensive preset range.

[0113] Upon completion of training, N compressed models and N decompressed models are obtained. Server 300 deploys the N compressed models on the target terminal device 200 to compress the first feature, resulting in N compressed features. It also deploys the N decompressed models on server 300 to decompress the N compressed features, resulting in N decompressed features, such as decompressed feature 1, decompressed feature 2, ..., decompressed feature N. A liveness classification model is then used to perform liveness detection on decompressed feature 1, decompressed feature 2, ..., decompressed feature N. Here, when performing liveness detection based on decompressed feature 1, decompressed feature 2, ..., decompressed feature N, server 300 can include the following implementation methods: for example, see [continued for details]. Figure 4 Server 300 inputs decompression feature 1, decompression feature 2, ..., decompression feature N into the liveness classification model, obtaining N liveness classification results, each represented by an attack probability. Then, server 300 performs a weighted summation of the N liveness classification results to obtain a weighted attack probability, and determines the first liveness detection result based on a comparison between the weighted attack probability and an attack probability threshold. For example, if the weighted attack probability P' is greater than the set threshold T', the target image is identified as an attack category; if the weighted attack probability P' is less than the set threshold T', the target image is identified as a liveness category.

[0114] It should be noted that when the probability of a live attack, P', is equal to the set threshold, the target image can be identified as either a live target or an attack target. This manual does not impose any restrictions on this.

[0115] In some embodiments, the second feature may further include a compressed fused feature obtained by fusing N compressed features based on a fusion model. See also... Figure 5 The fusion model also corresponds to a fusion decompression model. The fusion decompression model can include a decoder. After obtaining the compressed fusion features, server 300 can decompress the compressed fusion features based on the fusion decompression model deployed on server 300 to obtain decompressed fusion features. These decompressed fusion features are then used as target liveness detection features, and liveness detection is performed based on a liveness classification model to obtain the target liveness detection result. N compression models, fusion models, and fusion decompression models can be trained simultaneously.

[0116] Continuing to refer to the training process of the feature extraction model and the liveness classification model, server 300 extracts the first predicted feature from the original training image using a preset feature extraction model; compresses the first predicted feature using N different preset compression models to obtain N predicted compressed features; decompresses the N predicted compressed features using N preset decompression models to obtain N predicted decompressed features; fuses the N predicted compressed features using a preset fusion model to obtain predicted fused features; and decompresses the predicted fused features using a preset fusion decompression model to obtain predicted fused decompressed features.

[0117] After obtaining N predicted compression features, N predicted decompression features, predicted fusion features, and predicted fusion decompression features, server 300 can determine a third comprehensive loss based on the N predicted decompression features and their corresponding original features before compression; and adjust the parameters of the N preset compression models, N preset decompression models, preset fusion models, and preset fusion decompression models based on the third comprehensive loss, so that the outputs of the N preset compression models, N preset decompression models, preset fusion models, and preset fusion decompression models meet the training objective. The training objective may include the difference between the predicted decompression fusion features output by the fusion decompression model and their corresponding original features before compression being within a fifth preset range, and may also include at least one of the following: the difference between the N predicted decompression features obtained by decompressing the N predicted compression features output by the N compression models and their corresponding original features before compression being within a sixth preset range; the number of non-zero elements in the N predicted compression features being within a seventh preset range; the number of non-zero elements in the predicted fusion features output by the fusion model being within an eighth preset range; and the difference between the predicted image reconstructed based on the predicted fusion features and its corresponding original image being greater than a preset second difference threshold.

[0118] The third comprehensive loss can include the second feature map reconstruction loss, Loss_feat2. The second feature map reconstruction loss, Loss_feat2, can be determined based on the difference between the predicted fused decompression features output by the fused decompression model and the first feature. Loss_feat2 aims to constrain the difference between the predicted fused decompression features output by the fused decompression model and the first feature to be as small as possible, so that the predicted fused decompression features output by the fused decompression model approximate the first feature.

[0119] The third comprehensive loss may also include at least one of the following: the first feature map reconstruction loss Loss_feat1, the first volume compression loss Loss_compression1, the second volume compression loss Loss_compression2, and the second reverse engineering loss Loss_anti2. The determination process for the first feature map reconstruction loss and the first volume compression loss can be found in the preceding description and will not be repeated here.

[0120] The second volume compression loss, Loss_compression2, can be determined based on the number of non-zero elements in the predicted fusion features output by the fusion model. For example, server 300 determines the L1 norm of the predicted fusion features and uses this L1 norm as the second volume compression loss. The L1 norm is the sum of the absolute values ​​of all elements in the vector corresponding to the predicted fusion features. When the L1 norm is as small as possible, such as approaching 0, it indicates that there are many zero elements in the vector corresponding to the predicted fusion features. A large number of zero elements in the vector corresponding to the predicted fusion features indicates a better compression effect of the predicted fusion features. Since the transmission of feature maps occupies a large amount of bandwidth, it is not suitable for weak network environments. The second volume compression loss, Loss_compression2, aims to constrain the training process of the preset fusion model so that after the trained fusion model fuses the first feature, it can still ensure a reduction in the volume of the first feature, improve the transmission efficiency of the target feature, and thus improve the liveness detection efficiency. Of course, server 300 can also determine the number of non-zero elements in the predicted fusion features output by the fusion model based on L0 norm, L2 norm, etc., which is not limited in this specification.

[0121] The second anti-reverse engineering loss, Loss_anti2, can be determined based on the difference between the predicted image reconstructed from the predicted fusion features and its corresponding original training image. For example, the fusion model can also be connected to a preset third image reconstruction model. The preset third image reconstruction model is configured to perform image reconstruction based on features. After obtaining the predicted fusion features, server 300 can also input the predicted fusion features into the preset third image reconstruction model, so that the preset third image reconstruction model can reconstruct the image based on the predicted fusion features to obtain the predicted image corresponding to the predicted fusion features. Loss_anti2 aims to maximize the difference between the predicted image output by the preset third image reconstruction model and its corresponding original training image. This prevents attackers from reconstructing the target image through feature map reverse engineering, thereby avoiding attacks on the target user and causing losses to the target user, thus enhancing the defense against reverse engineering attacks.

[0122] It's important to note that Loss_anti2 is determined by the difference between the predicted image reconstructed from the predicted fusion features corresponding to the non-risk image and the non-risk image. The second anti-reverse engineering loss for risky images is not involved here. In the second anti-reverse engineering loss, no additional constraints are imposed on risky images, which enhances the detection capability against high-risk attacks.

[0123] After determining the second feature map reconstruction loss Loss_feat2, and at least one of the first feature map reconstruction loss Loss_feat1, the first volume compression loss Loss_compression1, the second volume compression loss Loss_compression2, and the second anti-reverse engineering loss Loss_anti2, server 300 can determine the third comprehensive loss based on the weighted sum of the second feature map reconstruction loss Loss_feat2 and at least one of the first feature map reconstruction loss Loss_feat1, the first volume compression loss Loss_compression1, the second volume compression loss Loss_compression2, and the second anti-reverse engineering loss Loss_anti2. Based on the third comprehensive loss, server 300 performs convergence on N preset compression models, preset fusion models, N preset decompression models, preset fusion decompression models, and a preset third reconstruction model. The converged N preset compression models are then determined as the trained N compression models, the converged preset fusion models are determined as the trained fusion models, and the converged preset fusion decompression models are determined as the trained fusion decompression models.

[0124] In the above embodiments, the convergence conditions may include: the third comprehensive loss is less than a third comprehensive preset range, or the number of iterations reaches a preset number, or the accuracy of the trained model reaches a preset accuracy, etc. The process of determining the third comprehensive preset range is similar to the process of determining the second comprehensive preset range; for details, please refer to the process of determining the second comprehensive preset range, which will not be repeated here.

[0125] Upon completion of training, N trained compression models, fusion models, and fusion-decompression models are obtained. Server 300 can deploy these N compression and fusion models on the target terminal device 200. The N compression models compress the first feature to obtain N compressed features, and the fusion models fuse these N compressed features to obtain compressed fused features. The fusion-decompression model is then deployed on server 300 to decompress the compressed fused features to obtain decompressed fused features, which are then used for liveness detection.

[0126] Server 300 can perform liveness detection using a liveness classification model based on decompressed and fused features. The liveness classification model can be configured to perform liveness detection based on features extracted from the image. The liveness classification model can be a high-performance ResNet50 network. Server 300 inputs the decompressed and fused features into the liveness classification model to obtain either a liveness probability P1 or an attack probability P2. The liveness probability P1 represents the probability that the target feature is a live object. The attack probability P2 represents the probability that the target feature is an attack object. Server 300 can determine the first liveness detection result based on either the liveness probability P1 or the attack probability P2. For example, if the liveness probability P1 is greater than a set threshold T1, the target image is identified as a live object; if the liveness probability P1 is less than the set threshold T1, the target image is identified as an attack object. Similarly, if the attack probability P2 is greater than a set threshold T2, the target image is identified as an attack object; if the attack probability P2 is less than the set threshold T2, the target image is identified as a live object.

[0127] It should be noted that when the liveness probability P1 or the attack probability P is equal to the set threshold T1 or T2, the target image can be identified as either a liveness category or an attack category. This manual does not impose any restrictions on this.

[0128] Continue reading Figure 3 After step S180, the method P100 may further include step S200.

[0129] S200: Server 300 determines the target liveness detection result based at least on the first liveness detection result.

[0130] In some embodiments, once the server 300 has determined the first liveness detection result, it may designate the first liveness detection result as the target liveness detection result.

[0131] However, for liveness detection, liveness classification alone cannot comprehensively cover all risks. Therefore, in some embodiments, server 300 can also determine a second liveness detection result based on feature retrieval, and jointly determine a target liveness detection result based on the first and second liveness detection results, thereby improving the security recall capability for risks. For example, server 300 can calculate the similarity between the target liveness detection features of the target user and the risk features of pre-labeled risk images in server 300 to search for whether there are risk features similar to the target liveness detection features, and perform liveness detection based on the search results to obtain a second liveness detection result. The risk image can be a manually labeled biometric image of a risk user or an attack target, or a biometric image of an attack target whose liveness detection difficulty is higher than a preset threshold. In some embodiments, the liveness detection difficulty can be obtained based on the liveness probability P1 and threshold T1 output by the liveness classification model, or the attack probability P2 and threshold T2. For example, the liveness detection difficulty can be the reciprocal of the absolute value of the difference between the liveness probability P1 and threshold T1, or the reciprocal of the absolute value of the difference between the attack probability P2 and threshold T2. The closer the liveness probability P1 is to the threshold T1, or the attack probability P2 is to the threshold T2, the higher the difficulty of liveness detection for the current attack target. In some embodiments, the difficulty of liveness detection can be determined as follows: For example, a trained liveness classification model is used to perform liveness detection on X images of the attack target, resulting in X liveness detection results. These X liveness detection results include both attack targets and non-attack targets. Assuming the number of images detecting the attack target is Y, the liveness detection difficulty can be determined based on the ratio Y / X.

[0132] In some embodiments, the M risk features include features extracted by at least one terminal device based on M risk images, where at least one terminal device includes a target terminal device 200. For example, server 300 may pre-store multiple risk images. The risk images stored in server 300 may come from the target terminal device 200 or from other terminal devices communicatively connected to server 300. When performing liveness detection on a user's biometrics based on a liveness classification model, server 300 may mark features corresponding to attack targets with liveness detection difficulty exceeding a preset threshold as risk features, and mark the biometric images corresponding to the risk features as risk images. In some embodiments, server 300 may also mark the terminal device that collected the risk features and risk images in the risk features and risk images. In some embodiments, server 300 may also send the marked risk images and / or risk features to the corresponding terminal devices (i.e., the acquisition devices) for storage.

[0133] The risk features corresponding to the risk image can be obtained through the aforementioned feature extraction model. The risk features can be features extracted by the feature extraction model, or features obtained by compressing the features extracted by the feature extraction model, such as N compressed features obtained by compressing N compressed features by the aforementioned N compression models, or compressed fused features obtained by fusing N compressed features.

[0134] The server 300 can determine the second liveness detection result in the following ways:

[0135] For example, server 300 acquires M risk features corresponding to M risk images and determines M similarities between the target feature and the M risk features; and based on the M similarities, determines a second liveness detection result. Here, the M risk images include images of attack targets whose liveness detection difficulty exceeds a preset threshold, and M is an integer greater than 0. Similarities can include cosine similarity, Euclidean distance, etc.

[0136] The M risk images can be all or part of the risk images stored on server 300. Alternatively, the M risk images can be risk images acquired by the target terminal device 200 during historical liveness detection. The M risk features are the features corresponding to the M risk images.

[0137] After acquiring M risk features, server 300 can determine the second liveness detection result based on the M risk features and the target feature. For example, server 300 performs feature retrieval among the M risk features to determine whether there is at least one risk feature among the M risk features whose similarity to the target feature is greater than a similarity threshold. Feature retrieval may include: server 300 determining the similarity between the target feature and each of the M risk features, such as cosine similarity, to obtain M similarities, and determining the second liveness detection result based on the M similarities. The implementation of server 300 determining the second liveness detection result based on the M similarities can be as follows: for example, if server 300 determines that at least one of the M similarities is greater than a preset similarity threshold, then the second liveness detection result is determined to be an attack category; or if it determines that all M similarities are less than the similarity threshold, then the second liveness detection result is determined to be a liveness category.

[0138] Similar to the second liveness detection result, the first liveness detection result characterizes the liveness classification of the target image as either a liveness category or an attack category. When server 300 determines that at least one of the first and second liveness detection results is the attack category, it determines the target liveness detection result as the attack category; or when both the first and second liveness detection results are the liveness category, it determines the target liveness detection result as the liveness category. In other words, when server 300 determines that the first liveness detection result is an attack category, or determines that the second liveness detection result is an attack category, or determines that both the first and second liveness detection results are attack categories, it determines the attack category as the final target liveness detection result. Target images of the attack category can cause harm to users. By determining that the target liveness detection result is an attack category when at least one of the first and second liveness detection results is an attack category, server 300 can improve the accuracy of liveness detection and increase user security protection.

[0139] S220: Server 300 outputs the target liveness detection result to target terminal device 200.

[0140] There are several ways to output the target liveness detection result. For example, the server 300 can directly return the target liveness detection result to the target terminal device 200, or it can return the target liveness detection result to the verification device that needs to perform facial recognition or verification, so that the verification device can perform facial recognition based on the target liveness detection result. Alternatively, the server 300 can directly return the target liveness detection result to the target terminal device 200, and the target terminal device 200 can visualize the target liveness detection result.

[0141] The target terminal device 200 can visualize the target liveness detection results in various ways. For example, the target terminal device 200 can display the target liveness detection results through a display, or it can issue prompts about the target liveness detection results through sound and light, etc.

[0142] In summary, the liveness detection method P100 and system 001 provided in this specification extract a first feature from the target image of the target user on the target terminal device 200, generate target features based on the first feature, and then send the generated target features to the server 300. The server 300 then performs liveness detection based on the target features and determines the final target liveness detection result based at least on the first liveness detection result obtained from the liveness detection based on the target features, and outputs the target liveness detection result. Since this scheme extracts features from the target image on the target terminal device 200 side and sends the extracted features to the server 300, and the server 300 performs liveness detection based on the received features, it can simultaneously ensure both the security and privacy protection capabilities of liveness detection. Furthermore, after obtaining the first feature, the target terminal device 200 compresses the first feature and transmits the compressed N features to the server 300, which reduces the size of the transmitted data, improves data transmission efficiency, and enhances the privacy protection of the transmitted N compressed features. By fusing N compressed features using a fusion model and then transmitting the fused compressed features to server 300, the size of the transmitted data can be further reduced, improving data transmission efficiency and thus improving liveness detection efficiency.

[0143] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing liveness detection. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the liveness detection method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of the liveness detection method P100 described herein. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 600, partially on computing device 600, as a standalone software package, partially on computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0144] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0145] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0146] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0147] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0148] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0149] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the multimedia resources involved in this disclosure were obtained under full authorization.

[0150] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A liveness detection method, applied to a server, the method comprising: The system receives target features sent by a target terminal device. The target features are determined based on first features obtained by the target terminal device using a feature extraction model to extract features from the target image of the target user. The feature extraction model is deployed on the target terminal device. Based on the target features, a liveness classification model deployed on the server is used to perform liveness detection to obtain a first liveness detection result. The feature extraction model and the liveness classification model are trained simultaneously, and the training objective includes the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range. The target liveness detection result is determined at least based on the first liveness detection result; and Output the target liveness detection results.

2. The method according to claim 1, wherein, The target features include: The first feature; or The second feature includes a compressed feature obtained by compressing the first feature.

3. The method according to claim 2, wherein, The step of performing liveness detection based on the target features using a liveness classification model deployed on the server to obtain a first liveness detection result includes: Based on the target features, determine the target liveness detection features; and Based on the liveness classification model, liveness detection is performed on the target liveness detection features to obtain the first liveness detection result.

4. The method according to claim 3, wherein, The step of determining the target liveness detection features based on the target features includes: Use the first feature as the target liveness detection feature; or The second feature is decompressed to obtain the decompressed feature, which is then used as the target liveness detection feature.

5. The method according to claim 1, wherein, The training objective also includes the difference between the predicted image and its corresponding original image within a second preset range, wherein the predicted image includes an image reconstructed based on the predicted features output by the feature extraction model.

6. The method according to claim 4, wherein, The second feature includes N compressed features obtained by compressing the first feature based on N different compression models, where N is a positive integer, and the N compression models are deployed on the target terminal device; The step of decompressing the second feature to obtain a decompressed feature, and using the decompressed feature as the target liveness detection feature, includes: The N compressed features are decompressed using N decompression models to obtain N decompression features. These N decompression features are then used as the target liveness detection features. The N decompression models correspond to the N compressed models, and the N decompression models are deployed on the server.

7. The method according to claim 6, wherein, The N compression models and the N decompression models are trained simultaneously. The training objective is that the difference between the N predicted decompression features output by the N decompression models and the first feature is within a third preset range.

8. The method according to claim 7, wherein, The training objective also includes at least one of the following: The number of non-zero elements in the N predicted compression features output by the N compression models is within a fourth preset range; as well as The difference between the predicted image reconstructed based on the N predicted compression features and its corresponding original training image is greater than a preset first difference threshold.

9. The method according to claim 4, wherein, The second feature includes a compressed fused feature obtained by fusing N compressed features based on a fusion model. The N compressed features are obtained by compressing the first feature based on N different compression models, where N is an integer greater than 1. The N compression models and the fusion model are deployed on the target terminal device. The step of decompressing the second feature to obtain a decompressed feature, and using the decompressed feature as the target liveness detection feature, includes: The compressed fusion features are decompressed based on the fusion decompression model, and the decompressed fusion features are used as the target liveness detection features. The fusion decompression model is deployed on the server.

10. The method according to claim 9, wherein, The N compression models, the fusion model, and the fusion decompression model are trained simultaneously. The training objective is that the difference between the predicted decompression fusion feature output by the fusion decompression model and the first feature is within a fifth preset range.

11. The method according to claim 10, wherein, The training objective also includes at least one of the following: The differences between the N predicted decompression features obtained by decompressing based on the N predicted compression features output by the N compression models and the first feature are within a sixth preset range; The number of non-zero elements in the N predicted compression features is within a seventh preset range; The number of non-zero elements in the predicted fusion features output by the fusion model is within an eighth preset range; as well as The difference between the predicted image reconstructed based on the predicted fusion features and its corresponding original training image is greater than a preset second difference threshold.

12. The method according to claim 1, wherein, The determination of the target liveness detection result based at least on the first liveness detection result includes: Determine the second liveness detection result; and The target liveness detection result is determined based on the first liveness detection result and the second liveness detection result.

13. The method according to claim 12, wherein, The determination of the second liveness detection result includes: Obtain M risk features corresponding to M risk images, wherein the M risk images include images of attack targets whose liveness detection difficulty is higher than a preset threshold, and M is an integer greater than 0; Determine the M similarities between the target feature and the M risk features; and Based on the M similarities, the second liveness detection result is determined.

14. The method according to claim 13, wherein, The determination of the second liveness detection result based on the M similarities includes: If at least one of the M similarities is greater than a preset similarity threshold, the second liveness detection result is determined to be an attack category; or If all M similarities are less than the similarity threshold, the second liveness detection result is determined to be a liveness category.

15. The method according to claim 13, wherein, The M risk features include features extracted by at least one terminal device based on the M risk images, and the at least one terminal device includes the target terminal device.

16. The method according to claim 12, wherein, The first liveness detection result includes a liveness category or an attack category, and the determination of the target liveness detection result based on the first liveness detection result and the second liveness detection result includes: Determine that at least one of the first liveness detection result and the second liveness detection result is the attack category, and determine the target liveness detection result as the attack category; or If both the first liveness detection result and the second liveness detection result are determined to be the liveness category, then the target liveness detection result is determined to be the liveness category.

17. A liveness detection system, comprising a server, the server comprising: At least one storage medium storing at least one instruction set for performing liveness detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the method according to any one of claims 1-16 according to the instructions of the at least one instruction set.

18. A liveness detection method applied to a target terminal device, the method comprising: Obtain the target image of the target user; The target image is used to extract features using the deployed feature extraction model to obtain the first feature; Based on the first feature, the target feature corresponding to the target image is determined; as well as The target features are sent to the server so that the server can perform liveness detection based on the target features using the deployed liveness classification model to obtain the target liveness detection result. The feature extraction model and the liveness classification model are trained simultaneously. The training objective includes the difference between the predicted liveness classification result and its corresponding real liveness classification result within a first preset range.

19. The method according to claim 18, wherein, Determining the target feature corresponding to the target image based on the first feature includes: Use the first feature as the target feature; or The first feature is compressed to obtain the second feature, and the second feature is used as the target feature.

20. The method according to claim 19, wherein, The process of compressing the first feature to obtain the second feature includes: The first feature is compressed using N deployed compression models to obtain N compressed features; and The N compression features are determined as the second feature; The server decompresses the N compressed features based on the deployed N decompression models, and performs liveness detection based on the N decompressed features. The N compressed models and the N decompression models are trained simultaneously.

21. The method according to claim 19, wherein, The process of compressing the first feature to obtain the second feature includes: The first feature is compressed using N deployed compression models to obtain N compressed features; The deployed fusion model is used to fuse the N compressed features to obtain compressed fused features; and The compressed fusion feature is identified as the second feature; The server decompresses the compressed and fused features based on the deployed fusion and decompression model, and performs liveness detection based on the decompressed and decompressed features. The N compression models, the fusion model, and the fusion and decompression model are trained simultaneously.

22. A liveness detection system, applied to a target terminal device, the target terminal device comprising: At least one storage medium storing at least one instruction set for performing liveness detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the method according to any one of claims 18-21 according to the instructions of the at least one instruction set.

Images