Liveness detection model training method, liveness detection method and system

Abstract

This specification provides a liveness detection model training method, a liveness detection method, and a system. After acquiring a first training image set, a second training image set is generated based on the first training image set. The liveness resolution of the second training image set in the basic liveness detection model is lower than that of the first training image set in the basic liveness detection model. The target neural network model is trained using the comprehensive training image set to obtain a target liveness detection model with better liveness resolution performance than the basic liveness detection model. The comprehensive training image set includes the first training image set and the second training image set. This scheme can improve the acquisition efficiency of liveness detection data in different domains.

Description

Technical Field

[0001] This specification relates to the field of Internet of Things (IoT) technology, and in particular to a liveness detection model training method, a liveness detection method, and a system. Background Technology

[0002] The development of facial recognition technology has brought great convenience to people's lives. However, facial recognition systems also face the security threat of liveness detection attacks. For example, after obtaining user A's facial image, an attacker can display it to the facial recognition system via a mobile phone or printout, thereby attacking the user. A successful liveness detection attack can cause significant harm to the user; therefore, it is necessary to conduct liveness detection.

[0003] Currently, deep learning models are commonly used for liveness detection. However, deep learning models have poor cross-domain generalization ability. To address this issue, one approach is to train using multi-domain data, collecting as much training data as possible from different domains to obtain a deep learning model that performs well in various scenarios. Another approach is based on few-shot fine-tuning, which involves collecting a small number of samples from the domains where the trained model performs poorly to optimize the model and improve its performance. However, both of these methods require collecting samples from each data domain, resulting in significant time consumption and low efficiency in domain data acquisition. Summary of the Invention

[0004] This specification provides a liveness detection model training method, a liveness detection method, and a system that can improve the efficiency of domain data acquisition.

[0005] In a first aspect, this specification provides a method for training a liveness detection model, comprising: acquiring a first training image set; generating a second training image set based on the first training image set, wherein the liveness resolution of the second training image set in the basic liveness detection model is lower than that of the first training image set in the basic liveness detection model; and training a target neural network model using the comprehensive training image set to obtain a target liveness detection model with liveness resolution performance superior to the basic liveness detection model, wherein the comprehensive training image set includes the first training image set and the second training image set.

[0006] In some embodiments, the basic liveness detection model is trained using the following steps: acquiring a third training image set, and designating each training image in the third training image set as a target training image; randomly shuffling the image blocks corresponding to the target training image using a preset basic liveness detection network to obtain at least two shuffled training images corresponding to the target training image; performing feature extraction and liveness category prediction on the at least two shuffled training images to obtain feature vectors and liveness classification results corresponding to the at least two shuffled training images; and determining first loss information based on the feature vectors and liveness classification results corresponding to the at least two shuffled training images, and converging the preset basic liveness detection network based on the first loss information to obtain the basic liveness detection model.

[0007] In some embodiments, determining the first loss information based on the feature vectors corresponding to the at least two perturbed training images and the liveness classification result includes: determining feature alignment loss information based on the feature vectors corresponding to the at least two perturbed training images, wherein the constraint condition of the feature alignment loss information is to minimize the similarity between the feature vectors corresponding to the at least two perturbed training images and to maximize the similarity between the feature vectors corresponding to different target training images; determining at least two first liveness classification loss information corresponding to the at least two perturbed training images based on the liveness classification result corresponding to the at least two perturbed training images and the labeled liveness category of the target training image; and obtaining the first loss information by summing the feature alignment loss information and the at least two first liveness classification loss information.

[0008] In some embodiments, generating a second training image set based on the first training image set includes: combining image patches of multiple training images in the first training image set using a trained reinforcement learning network to obtain an intermediate training image set corresponding to the first training image set; and performing domain sample partitioning on the intermediate training image set to obtain the second training image set.

[0009] In some embodiments, the reinforcement learning network is trained using the following steps: obtaining a fourth training image set, and using a preset reinforcement learning network to combine image patches of multiple training images in the fourth training image set to obtain a new training image set corresponding to the fourth training image set; using the basic liveness detection model to predict the liveness category corresponding to the new training image set to obtain the liveness classification result corresponding to the new training image set; and determining second loss information based on the liveness classification result corresponding to the new training image set and the labeled liveness category of the fourth training image set, and converging the preset reinforcement learning network in a direction that maximizes the second loss information to obtain the trained reinforcement learning network.

[0010] In some embodiments, the step of partitioning the intermediate training image set into domain samples to obtain the second training image set includes: inputting the intermediate training image set into the basic liveness detection model to obtain the feature vector corresponding to the intermediate training image set; performing N-class clustering on the feature vector corresponding to the intermediate training image set to obtain N new domain categories corresponding to the intermediate training image set, where N is an integer greater than or equal to 1; and obtaining the second training image set based on the intermediate training image set and its corresponding N new domain categories.

[0011] In some embodiments, the N-class clustering method is the KMeans clustering method.

[0012] In some embodiments, the comprehensive training image set corresponds to a sub-training image set of multiple domain categories, the multiple domain categories including M domain categories corresponding to the first training image set and N new domain categories corresponding to the second training image set, where M is an integer greater than or equal to 1, the target neural network model includes a feature extraction network, a multi-branch classification network, and a fusion classification network, the multi-branch classification network including multiple sub-classification networks; and the step of training the target neural network model using the comprehensive training image set to obtain a target liveness detection model with liveness resolution performance superior to the basic liveness detection model includes: using the feature extraction network to extract features from the sub-training image sets of the multiple domain categories to obtain multiple first training feature sets corresponding to the multiple sub-training image sets. The process involves: using multiple sub-branch classification networks to extract features and predict liveness categories from multiple first training feature sets, resulting in multiple second training feature sets and multiple sub-liveness classification results corresponding to the multiple first training feature sets; using the fusion classification network to perform liveness category fusion prediction based on the multiple second training feature sets and multiple sub-liveness classification results, resulting in a fusion liveness classification result; and determining second loss information based on the fusion liveness classification result and the labeled liveness categories corresponding to the comprehensive training image set, the multiple sub-liveness classification results and their corresponding labeled liveness categories, and converging the fusion classification network, the multi-branch classification network, and the feature extraction network based on the second loss information to obtain the target liveness detection model.

[0013] In some embodiments, the step of using the fusion classification network to perform feature fusion on the second training features of the plurality of sub-training image sets and predict the liveness category to obtain a fused liveness classification result for the plurality of sub-training image sets includes: using the fusion classification network to adaptively adjust the weights of the plurality of second training feature sets based on the differences between the multiple sub-classification results and the labeled liveness categories to obtain the weights of the plurality of second training feature sets; performing feature fusion on the plurality of second training feature sets based on the weights of the plurality of second training feature sets to obtain fused training features of the plurality of second training feature sets; and predicting the liveness category of the plurality of sub-training image sets based on the fused training features to obtain a fused liveness classification result for the plurality of sub-training image sets.

[0014] In some embodiments, determining the second loss information based on the fused liveness classification result and the labeled liveness categories corresponding to the comprehensive training image set, the plurality of sub-liveness classification results and their corresponding labeled liveness categories, includes: obtaining a fused liveness classification loss based on the fused liveness classification result and the labeled liveness categories corresponding to the comprehensive training image set; determining a plurality of sub-liveness classification losses based on the plurality of sub-liveness classification results and their corresponding labeled liveness categories; and obtaining the second loss information based on the sum of the fused liveness classification loss and the plurality of sub-liveness classification losses.

[0015] Secondly, this specification also provides a liveness detection method, comprising: acquiring an original image of a target object; and performing liveness detection on the original image using a liveness detection model to obtain a liveness detection result, wherein the liveness detection model is trained using the training method for the liveness detection model described in the first aspect.

[0016] Thirdly, this specification also provides a liveness detection system, comprising: at least one storage medium storing at least one instruction set for performing liveness detection; and at least one processor communicatively connected to the at least one storage medium, wherein, when the liveness detection device is running, the at least one processor reads a liveness detection model and implements the liveness detection method described in the second aspect, wherein the liveness detection model is trained according to the liveness detection model training method described in the first aspect.

[0017] As can be seen from the above technical solutions, the training method, liveness detection method, and system for the liveness detection model provided in this specification, after acquiring a first training image set, generate a second training image set based on the first training image set. The liveness discrimination of the second training image set in the basic liveness detection model is lower than that of the first training image set in the basic liveness detection model. Furthermore, the target neural network model is trained using the first and second training image sets to obtain a target liveness detection model with better liveness discrimination performance than the basic liveness detection model. Since this solution generates the second training image set based on the acquired first training image set, and the liveness discrimination of the second training image set in the basic liveness detection model is lower than that of the first training image set, the second training image set is a new domain sample that is difficult for the basic liveness detection model to distinguish the liveness category. Therefore, it is possible to generate new domain samples from existing domain samples, thereby improving the efficiency of domain data acquisition.

[0018] The training methods for the liveness detection model, the liveness detection methods, and other functions of the system provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the training methods for the liveness detection model, the liveness detection methods, and the system provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram illustrating an application scenario of a liveness detection system provided according to an embodiment of this specification is shown.

[0021] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0022] Figure 3 A flowchart of a liveness detection model training method provided according to an embodiment of this specification is shown;

[0023] Figure 4 The training process of a liveness detection model provided according to an embodiment of this specification is illustrated;

[0024] Figure 5 A flowchart of a liveness detection method provided according to an embodiment of this specification is shown. Detailed Implementation

[0025] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0026] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0027] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0028] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0029] For ease of description, the terms used in this specification are explained as follows:

[0030] Liveness detection and protection: Algorithm technology used in facial recognition systems to detect and intercept liveness attacks (including attacks using mobile phone photos, paper photos, and masks).

[0031] Cross-domain liveness detection: When using the same liveness detection algorithm to perform liveness detection across different data domains.

[0032] Different data domains: Data acquired from image acquisition in different scenarios, such as human posture, the height of the image acquisition device, the operating time of the image acquisition device (some devices are used at night, and some are used during the day), and the duration of interaction, all of which result in different characteristics of the acquired images.

[0033] Patch-wise: In this scheme, it refers to combining patches (image blocks) from different domains to form images of new domains, thereby expanding the categories of the domains.

[0034] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0035] The liveness detection method provided in this manual can be applied to any scenario requiring identity verification, such as facial recognition payment services in offline retail stores, AI vending machines, access control systems, or other scenarios requiring facial recognition payment. The next step of identity verification is only performed when the liveness detection result of the detected object confirms that the user is alive. For example, in a payment scenario, the liveness detection method provided in this manual can be used to detect the user's liveness, and facial recognition is only performed on the user after confirming that the user is alive. Furthermore, facial recognition payment is only allowed after confirming that the user is a legitimate user. Similarly, in an access control scenario, the liveness detection method provided in this manual can be used to detect the user's liveness, and facial recognition is only performed on the user after confirming that the user is alive. Access is only allowed after confirming that the user is a legitimate user. Or, in an information query scenario, the liveness detection method provided in this manual can be used to detect the user's liveness, and facial recognition is only performed on the user after confirming that the user is alive. Information queries are only allowed for the user after confirming that the user is a legitimate user.

[0036] Those skilled in the art should understand that the liveness detection method described in this specification can also be applied to other use cases within the scope of protection of this specification.

[0037] Figure 1 A schematic diagram illustrating an application scenario of a liveness detection system 001 provided according to an embodiment of this specification is shown. For example... Figure 1 As shown, the liveness detection system 001 (hereinafter referred to as system 001) may include: a target user 100, a terminal device 200, a server 300, and a network 400. The terminal device 200 and the server 300 are both connected to the network 400.

[0038] In this context, the target user 100 can be the user who triggers the liveness detection of the area to be detected. The target user 100 can perform the liveness detection operation on the terminal device 200. For example, in a payment service, the target user 100 can enter the facial recognition payment stage through a series of operations; or in an access control scenario, the target user can enter the facial recognition verification stage through a series of operations; thereby triggering the liveness detection function.

[0039] Terminal device 200 can be a device for performing liveness detection on a body part to be detected in response to a liveness detection operation by target user 100. In some embodiments, a liveness detection model training method and a liveness detection method can be executed on terminal device 200. In this case, terminal device 200 may store data or instructions for executing the liveness detection model training method and the liveness detection method described herein, and may execute or be used to execute the data or instructions. In some embodiments, terminal device 200 may include hardware devices with data information processing capabilities and the necessary programs required to drive the hardware devices. Figure 1As shown, terminal device 200 can communicate with server 300. In some embodiments, server 300 can communicate with multiple terminal devices 200. In some embodiments, terminal device 200 can interact with server 300 through network 400 to receive or send messages, such as receiving or sending raw images or liveness detection results. In some embodiments, terminal device 200 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, mobile devices may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, smart home devices may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, smart mobile devices may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, virtual reality devices or augmented reality devices may include virtual reality headsets, virtual reality glasses, virtual reality patches, augmented reality headsets, augmented reality glasses, augmented reality patches, or similar content, or any combination thereof. For example, virtual reality devices or augmented reality devices may include Google Glass, head-mounted displays, VR, etc. In some embodiments, built-in devices in motor vehicles may include in-vehicle computers, in-vehicle televisions, etc. In some embodiments, the terminal device 200 may include an image acquisition device for acquiring a first training image set, a third training image set, a fourth training image set, and an original image. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera), or it may be a combination of a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.).

[0040] In some embodiments, the terminal device 200 may have one or more applications (APPs) installed. APPs provide the target user 100 with the ability and interface to interact with the outside world via the network 400. APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc. In some embodiments, the terminal device 200 may have a target APP installed. The target APP can obtain a first training image set, a third training image set, a fourth training image set, original images, or liveness detection results through the terminal device 200. In some embodiments, the target user 100 can also trigger a liveness detection request through the target APP. The target APP can respond to the liveness detection request by executing the liveness detection method described in this specification. The liveness detection method will be described in detail later. In some embodiments, the target user 100 can also trigger a liveness detection model training request through the target APP. The target APP can respond to the liveness detection model training request by executing the liveness detection model training method described in this specification. The liveness detection model training method and the liveness detection method will be described in detail later.

[0041] Server 300 may be a server providing various services, such as a backend server supporting training image sets and original images acquired on terminal device 200. In some embodiments, the liveness detection model training method and the liveness detection method can be executed on server 300. In this case, server 300 may store data or instructions for executing the liveness detection model training method and the liveness detection method described herein, and may execute or be used to execute the data or instructions. In some embodiments, server 300 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Server 300 may be communicatively connected to multiple terminal devices 200 and receive data sent by the terminal devices 200.

[0042] Network 400 serves as a medium to provide a communication connection between terminal device 200 and server 300. Network 400 facilitates the exchange of information or data. For example... Figure 1As shown, terminal device 200 and server 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable network, wired network, fiber optic network, telecommunications network, intranet, Internet, local area network (LAN), wide area network (WAN), wireless local area network (WLAN), metropolitan area network (MAN), public switched telephone network (PSTN), Bluetooth network. TM ZigBee TM A network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of terminal device 200 and server 300 can connect to network 400 to exchange data or information.

[0043] It should be understood that Figure 1 The number of terminal devices 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, any number of terminal devices 200, servers 300, and networks 400 can be included.

[0044] It should be noted that the liveness detection model training method and the liveness detection method can be executed entirely on the terminal device 200, entirely on the server 300, or partially on the terminal device 200 and partially on the server 300.

[0045] Figure 2 A hardware structure diagram of a computing device 500 provided according to an embodiment of this specification is shown. The computing device 500 can execute the liveness detection model training method and the liveness detection method described in this specification. The liveness detection model training method and the liveness detection method are described in other parts of this specification. When the liveness detection model training method and the liveness detection method are executed on a terminal device 200, the computing device 500 can be the terminal device 200. When the liveness detection model training method and the liveness detection method are executed on a server 300, the computing device 500 can be the server 300. When the liveness detection model training method and the liveness detection method can be executed partly on the terminal device 200 and partly on the server 300, the computing device 500 can be both the terminal device 200 and the server 300.

[0046] like Figure 2As shown, the computing device 500 may include at least one storage medium 530 and at least one processor 520. In some embodiments, the computing device 500 may also include a communication port 550 and an internal communication bus 510. Additionally, the computing device 500 may also include I / O components 560.

[0047] The internal communication bus 510 can connect different system components, including storage medium 530, processor 520 and communication port 550.

[0048] I / O component 560 supports input / output between computing device 500 and other components.

[0049] Communication port 550 is used for data communication between computing device 500 and the outside world. For example, communication port 550 can be used for data communication between computing device 500 and network 400. Communication port 550 can be a wired communication port or a wireless communication port.

[0050] Storage medium 530 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 532, a read-only storage medium (ROM) 534, or a random access storage medium (RAM) 536. Storage medium 530 also includes at least one set of instructions stored in the data storage device. Instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the liveness detection model training method and liveness detection method provided in this specification.

[0051] At least one processor 520 can be communicatively connected to at least one storage medium 530 and a communication port 550 via an internal communication bus 510. At least one processor 520 is used to execute at least one instruction set described above. When the computing device 500 is running, at least one processor 520 reads at least one instruction set and, according to the instructions of at least one instruction set, executes the liveness detection model training method and the liveness detection method provided in this specification. Processor 520 can execute all the steps included in the liveness detection model training method and the liveness detection method. Processor 520 can be in the form of one or more processors. In some embodiments, processor 520 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 520 is described in the computing device 500 described in this specification. However, it should be noted that the computing device 500 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 520 of the computing device 500 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 520 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0052] Figure 3 A flowchart of a liveness detection model training method P100 according to an embodiment of this specification is shown. As previously described, computing device 500 can execute the liveness detection model training method P100 of this specification. Specifically, processor 520 can read an instruction set stored in its local storage medium and then execute the liveness detection model training method P100 of this specification according to the instructions in the instruction set. Figure 3 As shown, method P100 may include:

[0053] S110: Obtain the first training image set.

[0054] The first training image set includes multiple training images, each of which is an image of the target detection region of the training object. The training object is the object in the training image to be subjected to liveness detection, such as a user. When the training object is a user, the target detection region can be a human body part that can extract biometric features, such as a face, iris, or fingerprint.

[0055] There are several ways to obtain the first training image set, as follows:

[0056] For example, the processor 520 may receive a set of training images of the user's face uploaded by the target user 100 through a terminal device, or it may obtain at least one face image from a network or public image dataset to obtain a first training image set, or it may receive a model training request and obtain the first training image set based on the storage address of the first training image set carried in the model training request.

[0057] S120: Generate a second training image set based on the first training image set.

[0058] Specifically, the liveness detection performance of the second training image set in the basic liveness detection model is lower than that of the first training image set. Liveness detection performance refers to the ability of the basic liveness detection model to distinguish between liveness categories. For example, the basic liveness detection model can distinguish the liveness category of training objects in the first training image set very well, but cannot distinguish the liveness category of training objects in the second training image set very well.

[0059] There are several ways to generate a second training image set based on a first training image set. For example, the processor 520 uses a trained reinforcement learning network to combine image patches from multiple training images in the first training image set to obtain an intermediate training image set corresponding to the first training image set. Then, it performs domain sample partitioning on the intermediate training image set to obtain the second training image set. The reinforcement learning network can use a patch-wise sample generation method to combine existing domain data to obtain new domain data. Each new domain data is composed of patches of different domain data from the existing domain data, and the domain category of the new domain data is a domain category that has not appeared in the existing domain data.

[0060] The training of reinforcement learning networks and the partitioning of domain samples both require the use of a pre-trained basic liveness detection model. Therefore, before introducing the training process of reinforcement learning networks, please allow us to first introduce the training process of the basic liveness detection model.

[0061] The basic liveness detection model can be trained as follows: The processor 520 can acquire a third training image set and designate each training image in the third training image set as a target training image. Then, the image blocks corresponding to the target training images are randomly shuffled using a preset basic liveness detection network to obtain at least two perturbed training images corresponding to the target training images. Feature extraction and liveness category prediction are performed on the at least two perturbed training images to obtain feature vectors and liveness classification results corresponding to the at least two perturbed training images. Based on the feature vectors and liveness classification results corresponding to the at least two perturbed training images, the first loss information is determined. The preset basic liveness detection network is then converged based on the first loss information to obtain the basic liveness detection model.

[0062] The training images in the third training image set may completely or partially overlap with the training images in the first training image set. For example, the first training image set may be used as the third training image set, or the first training image set may be expanded to obtain the third training image set.

[0063] The basic liveness detection model is a model that is insensitive to facial structure, obtained by training a pre-defined basic liveness detection network using a patch-wise random shuffle strategy. This allows the basic liveness detection model to avoid over-reliance on facial structure information such as the distribution of facial features and facial contours, and instead focus more on texture information in the facial image, such as whether the screen is reflective or the paper material, which can determine whether the object in the facial image is alive. Since the model is trained using scrambled training images obtained by randomly shuffling the image patches in the third training image set, it can remove the dependence of liveness detection on facial structure. This allows the model to maintain stable liveness detection performance even when facial structure changes, such as changes in facial pose or angle, in some cross-domain scenarios, and to perform liveness detection well.

[0064] The pre-defined basic liveness detection network includes a patch-wise random shuffle sample generator and a feature extraction module. The feature extraction module can be a network structure such as ResNet18, ResNet50, or Transform networks used for feature extraction. The patch-wise random shuffle sample generator randomly shuffles the image patches corresponding to the target training image to obtain at least two perturbed training images corresponding to the target training image. The feature extraction module extracts features from at least two perturbed training images corresponding to each training image in the third training image set, obtaining feature vectors of the perturbed training images corresponding to the training images in the third training image set. Based on the feature vectors of the perturbed training images corresponding to the third training image set, liveness detection is performed to obtain the final liveness classification result.

[0065] The patch-wise random shuffle sample generator is used to randomly shuffle the image patches corresponding to the target training image to obtain at least two perturbed training images. This can be done in the following way: for example, the target training image is divided into w*h regions, and then the order of these w*h regions is randomly swapped to obtain the shuffled target training image, i.e., the perturbed training image. Here, randomly shuffling the image patches once per pair of target training images yields one perturbed training image. By shuffling the image patches of the target training image using at least two different shuffling orders, at least two perturbed training images corresponding to the target training image can be obtained.

[0066] After obtaining at least two perturbation training images corresponding to each training image in the third training image set, these images can be input into the feature extraction network to obtain the feature vectors of the perturbation training images corresponding to the training images in the third training image set and the liveness classification result. This liveness classification result characterizes the liveness category of the training object, that is, whether the training object in the training image is a live object. For example, the liveness category can include live images / objects / samples and attack images / objects / samples.

[0067] After extracting features and predicting the liveness category of the perturbed training images corresponding to the training images in the third training image set, the processor 520 can determine the first loss information based on the feature vectors of the perturbed training images corresponding to the training images in the third training image set and the liveness classification results. There are several ways to determine the first loss information based on the feature vectors and liveness classification results of at least two perturbed training images. For example, the processor 520 can determine feature comparison loss information based on the feature vectors of at least two perturbed training images, and determine at least two first liveness classification loss information for at least two perturbed training images based on the liveness classification results of at least two perturbed training images and the labeled liveness category of the target training image. Finally, the processor 520 obtains the first loss information by summing the feature comparison loss information and the at least two first liveness classification loss information. The constraint condition for the feature comparison loss information is to minimize the similarity between the feature vectors corresponding to at least two perturbed training images and to maximize the similarity between the feature vectors of the perturbed training images corresponding to different target training images. The feature comparison loss information aims to ensure that the feature vectors of perturbed training images obtained from two different shuffling orders of the same training image remain consistent, and to keep the feature vectors of different perturbed training images obtained from the same shuffling order of different training images far apart. The first liveness classification loss information represents the loss information generated by the feature extraction module for the difference between the predicted liveness category and the labeled liveness category of at least two perturbed training images.

[0068] After determining the feature comparison loss information and at least two first liveness classification loss information, the processor 520 can accumulate the feature comparison loss information and the at least two first liveness classification loss information to obtain the first loss information. The first loss information can be expressed as the following formula (1):

[0069] Loss base =Loss contrastive +Loss cls (1)

[0070] In equation (1), Loss base As the first loss information, Loss contrastive For feature alignment loss information, Loss cls For at least two first-living classification loss information.

[0071] The feature comparison loss information aims to ensure that, for training images that are originally similar, such as at least two perturbed training images corresponding to the same training image, after dimensionality reduction (feature extraction), at least two perturbed training images are still similar in the feature space. On the other hand, for two training images that are originally dissimilar, such as different training images, even if the image patches are shuffled in the same order, after dimensionality reduction, the two training images are still dissimilar in the feature space.

[0072] After determining the first loss information, the processor 520 can converge the preset basic liveness detection network based on the first loss information to obtain the trained basic liveness detection model. The processor 520 can converge the preset basic liveness detection network based on the first loss information in several ways. For example, the processor 520 can use a gradient descent algorithm to update the network parameters of the preset basic liveness detection network based on the first loss information and continue training until the preset basic liveness detection network reaches the preset training termination condition, thereby obtaining the trained basic liveness detection model.

[0073] The preset training termination condition can be, for example, reaching a preset number of training iterations or minimizing the first loss information.

[0074] After the basic liveness detection model is trained, a reinforcement learning network can be trained based on the trained basic liveness detection model. The reinforcement learning network can be trained using the following steps: For example, the processor 520 acquires a fourth training image set and uses a pre-defined reinforcement learning network to combine image patches from multiple training images in the fourth training image set to obtain a new training image set corresponding to the fourth training image set. The basic liveness detection model is then used to predict the liveness category corresponding to the new training image set, obtaining the liveness classification result corresponding to the new training image set. Based on the liveness classification result corresponding to the new training image set and the labeled liveness category of the fourth training image set, a second loss information is determined. The pre-defined reinforcement learning network is then converged in the direction that maximizes the second loss information to obtain the trained reinforcement learning network.

[0075] The preset reinforcement learning network can be a DQN (Deep Q-network). During each iteration of training the preset reinforcement learning network, when combining image patches from multiple training images in the fourth training image set, at least two training images are randomly selected from the multiple training images in the fourth training image set, and a new training image is generated based on these randomly selected images. There are several ways to randomly select at least two training images from the multiple training images in the fourth training image set and generate a new training image based on these randomly selected images. For example, the processor 520 randomly selects at least two training images, divides them into image patches, and selects some image patches from these patches to combine to obtain a new training image. This new training image is then input into the basic liveness detection model to predict the liveness category of the new training image, and the second loss information is determined based on the liveness category of the new training image and the labeled liveness category.

[0076] The function for the second loss information can be the Loss function in formula (1) above. cls The second loss information represents the loss information generated by the difference between the liveness category of the newly trained image and the labeled liveness category. After determining the second loss information, the processor 520 can optimize the loss function in the direction of maximizing the second loss information. Here, maximizing the second loss information of the basic liveness detection model aims to make the liveness discrimination of the newly trained image learned by the reinforcement learning network on the basic liveness detection model low. Furthermore, during the training of the reinforcement learning network, only the loss function of the basic liveness detection model is used to update the parameters of the reinforcement learning network, without updating the parameters of the basic liveness detection model. That is, during the training of the reinforcement learning network, the performance of the basic liveness detection model remains unchanged, and only the trained basic liveness detection model is used to train the reinforcement learning network. In this way, the basic liveness detection model can recognize the newly trained images generated by the reinforcement learning network. The lower the liveness discrimination of the newly trained image learned by the reinforcement learning network on the basic liveness detection model, the higher the accuracy of the newly trained image learned by the reinforcement learning network in identifying images of new domain categories that the basic liveness detection model has not trained on.

[0077] After generating an intermediate training image set from the first training image set using a reinforcement learning network, it is necessary to determine the domain categories of the training images in the intermediate training image set for subsequent training of the target neural network model. There are several ways to determine the domain categories of the training images in the intermediate training image set, such as: For example, the processor 520 inputs the intermediate training image set into the basic liveness detection model to obtain the feature vectors corresponding to the intermediate training image set. Based on the feature vectors, it performs N-class clustering to obtain N domain categories corresponding to the intermediate training image set, where N is an integer greater than or equal to 1. Based on the intermediate training image set and its corresponding N domain categories, a second training image set is obtained. These N domain categories are those not present in the first training image set. When these N domain categories are not present in the first training image set, the basic liveness detection model cannot effectively distinguish liveness, indicating that the generated intermediate training image set has new domain categories relative to the first training image set.

[0078] In some embodiments, in order to ensure that the basic liveness detection model can better serve the training of the reinforcement learning network, the domain category corresponding to the first training image set can be approximately the same as the domain category corresponding to the third training image set. That is, the domain category of the first training image set can be the same as the domain category of the training image set used in the training process of the basic liveness detection model. In this way, it can be ensured that the generated second training image set is a training image of a new domain category that the basic liveness detection model cannot distinguish as a liveness category during the training process of the reinforcement learning network.

[0079] N-class clustering groups the feature vectors of training images belonging to the same domain category in the intermediate training image set into one class. Since there is a one-to-one correspondence between the feature vectors of training images, the clustering result of the training image's feature vectors can represent the overall clustering result of the training images; that is, the domain category corresponding to the feature vector of a training image is the domain category of that training image. There are various methods for N-class clustering, such as the KMeans clustering method and other clustering algorithms.

[0080] It should be understood that the KMeans clustering method described here is illustrative. Other clustering algorithms that perform N-class clustering on the feature vectors corresponding to the intermediate training image set are all within the scope of protection of this specification.

[0081] S130: Train the target neural network model using a comprehensive training image set to obtain a target liveness detection model with better liveness resolution performance than the basic liveness detection model.

[0082] The comprehensive training image set includes a first training image set and a second training image set. After obtaining the first and second training image sets, they can be combined and applied to the training of the target neural network model to obtain the target liveness detection model. Since the second training image set is a newly generated training image set for new domain categories, it expands the domain categories of the first training image set, ensuring a diverse range of training images for different domain categories. Therefore, the liveness detection performance of the target liveness detection model trained using training images with diverse domain categories will be better than that of the basic liveness detection model, and the liveness detection performance is stable in cross-domain scenarios.

[0083] After partitioning the intermediate training image set into domain samples, the training images in the intermediate training image set can be clustered into N sub-training image sets. Each of the N sub-training image sets corresponds to one of the N domain categories. Each training image in the first training image set corresponds to a manually labeled domain category, which can be denoted as M domain categories. Therefore, the first training image set can be considered as a sub-training image set corresponding to M domain categories, where M is an integer greater than or equal to 1. Based on the M domain categories corresponding to the first training image set and the N new domain categories corresponding to the second training image set, we can obtain sub-training image sets corresponding to multiple domain categories in the comprehensive training image set. This is equivalent to grouping training images belonging to the same domain category in the comprehensive training image set into a single sub-training image set, thus obtaining sub-training image sets with multiple domain categories.

[0084] The target neural network model includes a feature extraction network, a multi-branch classification network, and a fusion classification network. The multi-branch classification network includes multiple sub-classification networks. The feature extraction network can be a network such as a backbone for feature extraction. The multi-branch classification network includes multiple sub-classification networks with the same structure. Each sub-classification network can be a residual network structure with multiple convolutional layers and ReLU layers. The fusion classification network can be a structure with fully connected layers and SE blocks.

[0085] After obtaining the comprehensive training image set, the target neural network model can be trained based on the comprehensive training image set to obtain a target liveness detection model with improved liveness resolution performance compared to the basic liveness detection model. Specifically, the method for training the target neural network model using the comprehensive training image set to obtain a target liveness detection model with superior liveness resolution performance compared to the basic liveness detection model can be as follows:

[0086] For example, the processor 520 uses a feature extraction network to extract features from sub-training image sets of multiple domain categories, obtaining multiple first training feature sets corresponding to multiple sub-training image sets. It then uses multiple sub-branch classification networks to extract features from the multiple first training feature sets and predict the liveness category, obtaining multiple second training feature sets and multiple sub-liveness classification results corresponding to the multiple first training feature sets. Finally, it uses the fusion classification network to perform liveness category fusion prediction based on the multiple second training feature sets and multiple sub-liveness classification results, obtaining a fused liveness classification result. Based on the fused liveness classification result and the labeled liveness category corresponding to the comprehensive training image set, the multiple sub-liveness classification results and their corresponding labeled liveness categories, it determines second loss information. Based on the second loss information, it converges the fusion classification network, the multi-branch classification network and the feature extraction network to obtain the target liveness detection model.

[0087] The feature extraction network takes all training images in the comprehensive training image set as input and outputs the first feature vector of each training image in the comprehensive training image set. Then, the first feature vector of at least one training image under the same domain category is input into the corresponding sub-branch classification network to obtain the second feature vector and the liveness category of at least one training image under the same domain category. For example, the first feature vector of the training image corresponding to domain A is input into the sub-branch classification network of domain A to obtain the second feature vector and the sub-liveness classification result of the training image corresponding to domain A. The first feature vector of the training image corresponding to domain B is input into the sub-branch classification network of domain B to obtain the second feature vector and the sub-liveness classification result of the training image corresponding to domain B. The first feature vector of the training image corresponding to domain C is input into the sub-branch classification network of domain C to obtain the second feature vector and the sub-liveness classification result of the training image corresponding to domain C. And so on, thus obtaining the second feature vector of the training image corresponding to the sub-training image set of each domain category in multiple domain categories and the sub-liveness classification result corresponding to the sub-training image set of each domain category in multiple domain categories. The sub-liveness classification result characterizes whether the training objects in each sub-training image set are live.

[0088] After obtaining the second feature vectors corresponding to the training images in the sub-training image sets of each domain category and the sub-liveness classification results corresponding to the sub-training image sets of each domain category, these second feature vectors and sub-liveness classification results can be input into a fusion classification network. Adaptive feature fusion is then performed using the SE block (Sqeuuze-and-excitation blocks) to obtain the fusion classification result corresponding to the comprehensive training image set. This fusion classification result characterizes whether the training objects in the comprehensive training image set are live. There are several ways to implement the fusion classification network to perform feature fusion and predict the liveness category of the second training feature sets of multiple sub-training image sets to obtain the fused liveness classification result of multiple sub-training image sets. For example, the processor 520 uses the fusion classification network to adaptively adjust the weights of multiple second training feature sets based on the differences between the multiple sub-classification results and the labeled liveness categories to obtain the weights of multiple second training feature sets; it then performs feature fusion on the multiple second training feature sets based on the weights of the multiple second training feature sets to obtain the fused training features of the multiple second training feature sets; and finally, it predicts the liveness category of multiple sub-training image sets based on the fused training features to obtain the fused liveness classification result of multiple sub-training image sets.

[0089] Among the multiple sub-classification results, the difference between the sub-classification result and the labeled liveness category is negatively correlated with the weight of the second training feature corresponding to the sub-classification result. That is, the smaller the difference between the sub-classification result and the labeled liveness category, the greater the weight of the second training feature corresponding to the sub-classification result; conversely, the greater the difference between the sub-classification result and the labeled liveness category, the smaller the weight of the second training feature corresponding to the sub-classification result.

[0090] After obtaining the fused liveness classification result corresponding to the comprehensive training image set, a second loss information can be determined based on the fused liveness classification result, the labeled liveness categories corresponding to multiple sub-training image sets, and the sub-liveness classification results and labeled liveness categories of multiple sub-training image sets. The fused classification network, multi-branch classification network, and feature extraction network are then converged based on the second loss information to obtain the target liveness detection model. There are several ways to determine the second loss information based on the fused liveness classification result, the labeled liveness categories corresponding to multiple sub-training image sets, and the sub-liveness classification results and labeled liveness categories of multiple sub-training image sets. For example, the processor 520 can determine the fused liveness classification loss based on the fused liveness classification result and the labeled liveness categories corresponding to multiple sub-training image sets, determine multiple sub-liveness classification losses based on the sub-liveness classification results of multiple sub-training image sets and their corresponding labeled liveness categories, and obtain the second loss information based on the sum of the fused liveness classification loss and the multiple sub-liveness classification losses.

[0091] The fusion liveness classification loss represents the loss information arising from the difference between the fusion liveness classification result and the labeled liveness categories corresponding to multiple sub-training image sets. The sub-liveness classification loss represents the loss information arising from the difference between the sub-liveness classification result corresponding to each sub-training image set and the labeled liveness category corresponding to each sub-training image set. After determining the fusion liveness classification loss and the multiple sub-liveness classification losses, the fusion liveness classification loss and the multiple sub-liveness classification losses can be accumulated to obtain the second loss information.

[0092] After determining the second loss information, the processor 520 can converge the feature extraction network, multi-branch classification network, and fusion classification network based on the second loss information, thereby obtaining the trained target liveness detection model. The processor 520 can converge the feature extraction network, multi-branch classification network, and fusion classification network based on the second loss information in several ways. For example, the processor 520 can use the gradient descent algorithm to update the network parameters of the preset feature extraction network, multi-branch classification network, and fusion classification network based on the second loss information, and continue training until the feature extraction network, multi-branch classification network, and fusion classification network reach the preset training termination condition, thus obtaining the trained target liveness detection model. The preset training termination condition can be reaching a preset number of training iterations or minimizing the second loss information, etc.

[0093] After training the feature extraction network, multi-branch classification network, and fusion classification network to obtain the liveness detection model, the liveness detection model can be used for liveness detection.

[0094] To reduce the manpower and time costs of collecting domain sample data and improve the efficiency of domain sample data acquisition, this solution proposes a cross-domain generalized liveness detection method based on patch-wise domain generation. The overall flowchart of this patch-wise domain-based cross-domain generalized liveness detection method is shown below. Figure 4 As shown, it mainly includes four parts: basic liveness detection model training, patch-wise sample generation based on performance degradation, joint training based on multi-domain samples, and liveness detection, which can be described as follows:

[0095] (1) Training of basic liveness detection model: The basic liveness detection model is trained based on the patch-wise random shuffle strategy to obtain a basic liveness detection model that is not sensitive to facial structure information.

[0096] (2) Patch-wise sample generation based on performance degradation: Using reinforcement learning, new training images of multiple domains that are most difficult to distinguish for the base model are generated. Each new training image is composed of patches of different domains in the existing training images and can be regarded as a completely new domain training image.

[0097] (3) Joint training based on multi-domain samples: Joint training of multiple domains and branches is carried out based on the patch-wise training images generated above.

[0098] (4) Liveness detection: The above-mentioned multi-branch and intelligent fusion liveness detection model is obtained by joint training based on multi-domain samples and deployed on the corresponding device for liveness detection.

[0099] After training and obtaining the target liveness detection model, the liveness detection model can be used for liveness detection.

[0100] Figure 5 A flowchart of a liveness detection method P200 according to some embodiments of this specification is shown. As previously described, computing device 500 can execute the liveness detection method P200 described herein. Specifically, processor 620 can read an instruction set stored in its local storage medium and then execute the liveness detection method P200 described herein according to the instructions in the instruction set. Figure 5 As shown, method P200 may include:

[0101] S210: Obtain the original image of the object to be detected.

[0102] The object to be detected refers to the object to be subjected to liveness detection, such as a human body. The original image can be a facial image of the object to be detected, acquired through an image acquisition device.

[0103] S220: Input the original image into the liveness detection model to obtain the liveness detection result corresponding to the original image. The liveness detection model is trained according to the liveness detection model training method.

[0104] For example, processor 520 can directly input the raw image to... Figure 3 The training method is used to train a liveness detection model to perform liveness detection on the objects to be detected in the original image, thereby obtaining the liveness detection result.

[0105] It should be noted that when using a liveness detection model to perform liveness detection on the original image, the original image can be input into a feature extraction network for feature extraction, obtaining a first feature vector corresponding to the original image. This first feature vector is then simultaneously input into multiple sub-classification networks, allowing each sub-classification network to extract features based on the first feature vector, thereby obtaining multiple second feature vectors corresponding to the original image. These multiple second feature vectors can be input into a fusion classification network, where adaptive feature fusion is performed through an SE block to obtain a liveness classification probability. Based on this probability, either a first operation or a second operation is performed: the first operation is that if the liveness classification probability is greater than a threshold T, the target in the original image is determined to be an attacker. The second operation is that if the liveness classification probability is less than or equal to the threshold T, the target in the original image is determined to be a live object.

[0106] In this context, adaptive feature fusion can be understood as the fusion classification network adaptively adjusting the weights of the feature vectors of each of the multiple branches it receives, thereby weighting the feature vectors from each branch based on the adjusted weights, thus performing feature fusion, and performing liveness detection based on the fused feature vectors.

[0107] In the adaptive feature fusion process, the fusion classification network uses attention-based to fuse the feature vectors obtained from multiple branches to obtain the final liveness classification result. Attention-based is a similarity measure. Among the multiple feature vectors received by the fusion classification network, the feature vector that is more similar to the target state will have a larger weight, indicating that the current output is more dependent on the current input feature vector.

[0108] The specific features of this manual are as follows: (1) It adopts a patch-wise sample generation method, which combines existing domain data to obtain new domain data, thereby solving the problem of data diversity and improving the collection efficiency of different domain data. (2) In the training process of the basic detection model, the face structure is directly shuffled from the input by the patch-wise random shuffle sample generation and comparison learning method, thus removing the dependence of liveness detection on face structure. (3) In the training process of the target neural network model, a multi-branch network structure is adopted, so that each branch learns several training images of similar domains, and the features obtained by multiple branches are fused using attention-based to obtain the final liveness classification result; in this way, the generalization ability requirement of each branch can be reduced, and better overall performance can be obtained.

[0109] In summary, the liveness detection model training method, liveness detection method, and system provided in this specification, after acquiring a first training image set, generate a second training image set based on the first training image set. The liveness discrimination of the second training image set in the basic liveness detection model is lower than that of the first training image set in the basic liveness detection model. Furthermore, the target neural network model is trained using the first and second training image sets to obtain a target liveness detection model with better liveness discrimination performance than the basic liveness detection model. Since this scheme generates the second training image set based on the acquired first training image set, and the liveness discrimination of the second training image set in the basic liveness detection model is lower than that of the first training image set, the second training image set represents new domain samples for which the basic liveness detection model has difficulty in distinguishing liveness categories. Therefore, it is possible to generate new domain samples from existing domain samples, thereby improving the efficiency of domain data acquisition.

[0110] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for training a liveness detection model and performing liveness detection. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the liveness detection model training method P100 and the liveness detection method P200 described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 500, the program code causes the computing device 500 to perform the steps of the liveness detection model training method P100 and the liveness detection method P200 described in this specification. The program product for implementing the above methods may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing device 500. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages.The program code can be executed entirely on computing device 500, partially on computing device 500, as a standalone software package, partially on computing device 500 and partially on a remote computing device, or entirely on a remote computing device.

[0111] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0112] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0113] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0114] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0115] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0116] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for training a liveness detection model, comprising: Obtain a first training image set, which includes multiple first training images; A second training image set is generated based on the reinforcement learning network and the first training image set. The liveness detection performance of the second training image set in the basic liveness detection model is lower than that of the first training image set in the basic liveness detection model. The reinforcement learning network is trained using the following steps: A fourth training image set is obtained, and a preset reinforcement learning network is used to combine image patches of multiple fourth training images in the fourth training image set to obtain new training images corresponding to the multiple fourth training images. A basic liveness detection model is used to predict the liveness category corresponding to the new training image, thus obtaining the liveness classification result for the new training image; and Based on the liveness classification result corresponding to the new training image and the labeled liveness categories of the multiple fourth training images in the fourth training image set, a second loss information is determined, and the preset reinforcement learning network is converged in the direction that maximizes the second loss information to obtain a trained reinforcement learning network; and The target liveness detection model is trained using a comprehensive training image set to obtain the target liveness detection model. The comprehensive training image set includes the first training image set and the second training image set.

2. The method according to claim 1, wherein, The basic liveness detection model was trained using the following steps: Obtain a third training image set, and denote each training image in the third training image set as a target training image; A preset basic liveness detection network is used to randomly shuffle the image blocks corresponding to the target training image to obtain at least two shuffled training images corresponding to the target training image. Feature extraction and liveness classification prediction are performed on the at least two perturbed training images to obtain the feature vectors and liveness classification results corresponding to the at least two perturbed training images; as well as Based on the feature vectors corresponding to the at least two perturbed training images and the liveness classification results, a first loss information is determined, and the preset basic liveness detection network is trained based on the first loss information to obtain the basic liveness detection model.

3. The method according to claim 2, wherein, The determination of the first loss information based on the feature vectors corresponding to the at least two perturbed training images and the liveness classification results includes: Based on the feature vectors corresponding to the at least two perturbed training images, feature matching loss information is determined. The constraint condition of the feature matching loss information is to maximize the similarity between the feature vectors corresponding to the at least two perturbed training images and to minimize the similarity between the feature vectors of the perturbed training images corresponding to different target training images. Based on the liveness classification results corresponding to the at least two perturbed training images and the labeled liveness category of the target training image, at least two first liveness classification loss information corresponding to the at least two perturbed training images are determined; and The first loss information is obtained by summing the feature comparison loss information and the at least two first liveness classification loss information.

4. The method according to claim 1, wherein, The step of generating a second training image set based on the reinforcement learning network and the first training image set includes: The reinforcement learning network is used to combine image patches of the plurality of first training images in the first training image set to obtain an intermediate training image set corresponding to the first training image set; and The intermediate training images in the intermediate training image set are divided into domain samples to obtain the second training image set.

5. The method according to claim 4, wherein, The step of dividing multiple intermediate training images in the intermediate training image set into domain samples to obtain the second training image set includes: The multiple intermediate training images are input into the basic liveness detection model to obtain the feature vectors corresponding to the multiple intermediate training images; Perform N-class clustering on the feature vectors corresponding to the plurality of intermediate training images to obtain N new domain categories corresponding to the intermediate training image set, where N is an integer greater than or equal to 1; and The second training image set is obtained based on the intermediate training image set and its corresponding N new domain categories.

6. The method according to claim 5, wherein, The N-class clustering method is the KMeans clustering method.

7. A method for detecting liveness, comprising: Obtain the original image of the object to be detected; as well as The original image is input into the liveness detection model to obtain the liveness detection result corresponding to the original image. The liveness detection model is trained according to the liveness detection model training method as described in any one of claims 1-6.

8. A liveness detection system, comprising: At least one storage medium storing at least one instruction set for performing liveness detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the liveness detection system is running, the at least one processor reads the liveness detection model and implements the liveness detection method of claim 7, wherein the liveness detection model is trained according to the liveness detection model training method of any one of claims 1-6.

Images