Training method, detection method and system for detecting a network

Abstract

The training method, detection method and system for detecting a network provided in the specification obtain detection data of a to-be-detected image, the to-be-detected image depicting a biological feature of a target object; input the detection data into an M-layer Transformer network for detection of a live biological feature, to obtain a detection result of the to-be-detected image on the live biological feature; determine correlation enhancement data of at least two layers of output results of the M-layer Transformer network on the live biological feature; obtain a reference feature map of the to-be-detected image; obtain a classification loss value according to the detection result of the to-be-detected image data on the live biological feature, obtain a contrast feature loss value according to the reference feature map and the correlation enhancement data on the live biological feature, and obtain a joint loss value according to the classification loss value and the contrast feature loss value; and train the detection network according to the joint loss value, wherein M is a natural number greater than 1.

Description

Technical Field

[0001] This specification relates to the field of data processing, and in particular to a training method, detection method, and system for a detection network. Background Technology

[0002] Traditional facial recognition systems cannot distinguish between live and fake faces in acquired images. With the widespread application of facial recognition technology, various fake face attack methods, such as facial photos, facial videos, and 3D masks, are emerging, making liveness detection increasingly important. Liveness detection is gradually becoming an indispensable part of facial recognition systems.

[0003] However, as facial recognition systems are increasingly applied to more fields, they encounter various scenarios with vastly different characteristics. Therefore, how to make the detection network have better classification robustness in various scenarios has become an important research direction. Summary of the Invention

[0004] The training method, detection method, and system for the detection network provided in this manual can effectively reduce the influence of a certain block of image in the face image to be detected on the final classification result.

[0005] In a first aspect, this specification provides a method for training a detection network, comprising: acquiring detection data of an image to be detected, the image to be detected depicting the biometric features of a target object; inputting the detection data into an M-layer Transformer network for detection of live biometric features, obtaining detection results of the image to be detected regarding live biometric features; determining correlation enhancement data of at least two layers of the output of the M-layer Transformer network regarding live biometric features; obtaining a reference feature map of the image to be detected; obtaining a classification loss value based on the detection results of the image data regarding live biometric features, obtaining a contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding live biometric features, obtaining a joint loss value based on the classification loss value and the contrastive feature loss value; and training the detection network based on the joint loss value, wherein M is a natural number greater than 1.

[0006] In some embodiments, the detection network is a liveness attack detection network, and the detection results include liveness category and attack category.

[0007] In some embodiments, obtaining the detection data of the image to be detected includes: dividing the image to be detected into N blocks, and preprocessing the sub-data corresponding to each block into the detection data, where N is a natural number greater than 1.

[0008] In some embodiments, the step of preprocessing the sub-data corresponding to each block into the data to be detected includes: performing dimensional transformation and concatenation processing on the sub-data corresponding to each block to obtain block embedding data; performing position encoding on the sub-data corresponding to each block to obtain position embedding data; and performing linear mapping on the result of adding the block embedding data and the position embedding data to obtain the data to be detected.

[0009] In some embodiments, the preprocessing of the sub-data corresponding to each block into the data to be detected includes: performing dimensional transformation and concatenation processing on the sub-data corresponding to each block to obtain first block embedding data; obtaining type embedding data according to the detection type of the data to be detected by the detection network; concatenating the type embedding data and the first block embedding data to obtain second block embedding data; performing position encoding on the sub-data corresponding to each block to obtain position embedding data; and performing a linear mapping on the result of adding the second block embedding data and the position embedding data to obtain the data to be detected.

[0010] In some embodiments, determining the correlation enhancement data of at least two layers of the M-layer Transformer network with respect to live biological features includes: inputting the output of the M-th layer and the output of the (M-1)-th layer of the Transformer network into the (M-1)-th layer correlation enhancement module for fusion processing to obtain the first layer fusion output result; inputting the fusion output result of the i-th layer correlation enhancement module and the output result of the (M-1-i)-th layer of the Transformer network into the (i+1)-th layer correlation enhancement module for fusion processing to obtain the (i+1)-th layer fusion output result, wherein the output result of each layer correlation enhancement module enhances the correlation information between live biological features in different blocks of its input data, i = any integer between [1, M-1]; and using the fusion output result of the (M-1)-th layer correlation enhancement module as the correlation enhancement data with respect to live biological features.

[0011] In some embodiments, the step of inputting the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network into the i+1-th layer correlation enhancement module for fusion processing to obtain the i+1-th layer fusion output includes: performing 1×1 convolution, concatenation, and multiplication feature fusion processing on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the i+1-th layer fusion output.

[0012] In some embodiments, the step of performing a 1×1 convolution, concatenation, and multiplication on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the fusion output of the (i+1)-th layer includes: performing a 1×1 convolution on the fusion output of the i-th layer correlation enhancement module to obtain first convolutional data; performing a 1×1 convolution on the output of the M-1-i-th layer of the Transformer network to obtain second convolutional data; concatenating the first convolutional data and the second convolutional data to obtain first concatenated data; and then... The first concatenated data is subjected to two 1×1 convolutions to obtain the third and fourth convolutional data, respectively; the third and fourth convolutional data are concatenated to obtain the second concatenated data; the second concatenated data is subjected to a 1×1 convolution to obtain the fifth convolutional data; the fifth convolutional data is subjected to softmax processing, and the two output results of softmax are multiplied by the fusion output result of the i-th layer correlation enhancement module and the output result of the M-1-i-th layer of the Transformer network, and then added to obtain the fusion output result of the (i+1)-th layer.

[0013] In some embodiments, the step of inputting the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network into the i+1-th layer correlation enhancement module for fusion processing to obtain the i+1-th layer fusion output result includes: performing feature fusion processing based on the Inception network on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the i+1-th layer fusion output result.

[0014] In some embodiments, obtaining the contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding the live biological features includes: obtaining a contrastive feature map based on the correlation enhancement data regarding the live biological features, wherein the contrastive feature map is an image of the type corresponding to the reference feature map; and obtaining the contrastive feature loss value of the image to be detected based on the reference feature map and the contrastive feature map.

[0015] In some embodiments, the reference feature map is at least one type of feature map, such as a depth map, an IR map, or a reflection map.

[0016] Secondly, this specification also provides a detection method, comprising: acquiring an image to be detected, wherein the image to be detected depicts the biometric features of a target object; inputting the image to be detected into a detection network to detect the image; and outputting the detection result of the image to be detected regarding the biometric features of a living organism, wherein the detection network is trained according to the training method of the detection network as described in the first aspect.

[0017] Thirdly, this specification also provides a detection system, comprising: at least one storage medium storing a detection network for detection; and at least one processor communicatively connected to the at least one storage medium, wherein when the detection system is running, the at least one processor reads the detection network and implements the detection method as described in the second aspect, wherein the detection network is trained according to the training method for the detection network as described in the first aspect.

[0018] As can be seen from the above technical solutions, the training method, detection method, and system for the detection network provided in this specification involve: acquiring detection data of an image to be detected, wherein the image to be detected depicts the biological features of a target object; acquiring a reference feature map of the image to be detected; inputting the detection data into an M-layer Transformer network for detection of live biological features to obtain the detection result of the image to be detected regarding live biological features; determining the correlation enhancement data of at least two layers of the output of the M-layer Transformer network regarding live biological features; obtaining a classification loss value based on the detection result of the image data regarding live biological features; obtaining a contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding live biological features; obtaining a joint loss value based on the classification loss value and the contrastive feature loss value; and training the detection network based on the joint loss value, wherein M is a natural number greater than 1. By dividing the image to be detected into multiple blocks, robust feature extraction is performed on each block of the image to be detected using an M-layer Transformer network. Furthermore, efficient aggregation of features between different blocks is achieved by determining the correlation enhancement data of at least two layers of the M-layer Transformer network outputs regarding live biological features. A classification loss value is then obtained based on the detection results of the image data regarding live biological features. A contrastive feature loss value is obtained based on the reference feature map and the correlation enhancement data regarding live biological features. Finally, a joint loss value is obtained based on the classification loss value and the contrastive feature loss value. Joint end-to-end optimization is performed using the joint loss value, thereby improving the accuracy of live biological feature detection in the image to be detected.

[0019] The training methods, detection methods, and other functions of the detection network and system provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the training methods, detection methods, and systems of the detection network provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A hardware structure diagram of a computing device provided according to some embodiments of this specification is shown;

[0022] Figure 2 A structural diagram of a detection network provided according to some embodiments of this specification is shown;

[0023] Figure 3 A flowchart of a training method for a detection network according to some embodiments of this specification is shown;

[0024] Figure 4 This specification illustrates a method for segmenting an image to be detected according to some embodiments;

[0025] Figure 5 Two position encoding methods provided according to some embodiments of this specification are shown;

[0026] Figure 6 A method for generating block embedding data according to some embodiments of this specification is shown;

[0027] Figure 7 Another method for generating block embedding data is shown according to some embodiments of this specification;

[0028] Figure 8 A structural diagram of an encoder provided according to some embodiments of this specification is shown;

[0029] Figure 9 A structural diagram of a correlation fusion module provided according to some embodiments of this specification is shown; and

[0030] Figure 10A flowchart of a detection method provided according to some embodiments of this specification is shown. Detailed Implementation

[0031] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0032] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0033] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0034] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0035] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0036] As mentioned earlier, detection can be applied in various fields such as facial recognition, intelligent transportation, and industrial inspection. Taking facial recognition as an example, "liveness detection" is used to determine whether biometric features originate from a living individual. It can effectively block attacks from non-liveness-related samples such as photos and videos, improve verification security, ensure the pass rate of real individuals, and achieve the accuracy of facial recognition. Therefore, liveness detection is an indispensable part of facial recognition systems. However, as facial recognition systems are increasingly applied to more diverse scenarios, improving the accuracy and robustness of liveness detection methods has become an important research and application direction. For example, an end-to-end binary classification-based liveness detection method can be used. However, because it only uses a fixed network structure to train the liveness detection network through end-to-end binary classification, this network is overly dependent on the reasonable distribution of the original data. Alternatively, a liveness detection method based on 3D feature constraints can be used. This method extracts depth maps using a 3D camera and constrains the learning process of the liveness detection model by introducing 3D feature constraints into the loss function, thereby fusing 3D features into the 2D liveness detection task. However, this method requires multimodal hardware as a foundation, is costly, and cannot adapt to abnormal situations in different modalities.

[0037] To address the above requirements, this specification proposes a training method, detection method, and system for a detection network. This solution improves the accuracy and robustness of liveness attack detection by training the detection network with the image to be detected through a multi-layer Transformer network and using the outputs of at least two layers of the multi-layer Transformer network.

[0038] For ease of description, this manual explains the terms that may appear in the following descriptions:

[0039] Liveness attack detection: refers to the process of distinguishing between real people and attacks during the facial recognition process in daily life. Attack types mainly include non-real people such as photos, mobile phones, screens, and masks.

[0040] A multi-layer perceptron (MLP), also known as an artificial neural network (ANN), is used for feature fusion. Besides the input and output layers, it can have multiple hidden layers. By connecting multiple feature values ​​and combining linear and non-linear approaches, it ultimately achieves a desired outcome.

[0041] Self-attention: Neural networks receive inputs consisting of many vectors of varying sizes, with certain relationships between them. However, during actual training, these relationships are often not fully utilized, leading to poor network training results. Self-attention allows the machine to notice the correlations between different parts of the input.

[0042] Softmax, also known as the normalization exponential function, is used to represent the results of multi-class classification in the form of probabilities.

[0043] PRNet network: It transforms an input image into a location map, which is usually a three-dimensional matrix. The first two dimensions are the dimensions of the output texture map, and the last dimension represents the position information of each pixel in the texture map in 3D space.

[0044] Figure 1 A hardware structure diagram of a computing device 100 according to some embodiments of this specification is shown. The computing device 100 can execute the training method and detection method of the detection network described in this specification. The computing device executing the training method of the detection network and the computing device executing the detection method can be the same computing device 100 or two independent computing devices 100. The training method of the detection network and the detection method are described in other parts of this specification. The computing device 100 can be a client, a server, or a combination of a client and a server.

[0045] like Figure 1 As shown, the computing device 100 may include at least one storage medium 130 and at least one processor 120. In some embodiments, the computing device 100 may also include a communication port 150 and an internal communication bus 110. Additionally, the computing device 100 may include I / O components 160.

[0046] The internal communication bus 110 can connect different system components, including storage medium 130, processor 120 and communication port 150.

[0047] I / O component 160 supports input / output between computing device 100 and other components.

[0048] Communication port 150 can be either a wired or wireless communication port. Communication port 150 is used for data communication between the computing device 100 and external sources. For example, communication port 150 can be used for data communication between the computing device 100 and a communication network, which can be a wired or wireless network. The computing device 100 can receive data from a data acquisition device through this communication network. Figure 1(Data not shown in the image). The data acquisition device and computing device 100 can be installed as a whole or separately. The data acquisition device can acquire training data from multiple source domains and send the acquired training data to the computing device 100. The data acquisition device can be an image acquisition device, an audio acquisition device, a video acquisition device, etc., and correspondingly, the training data can be image data, audio data, video data, etc.

[0049] Storage medium 130 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 130 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that perform the face recognition methods provided in this specification.

[0050] At least one processor 120 can be communicatively connected to at least one storage medium 130 and a communication port 150 via an internal communication bus 110. At least one processor 120 is used to execute the at least one instruction set described above. When the computing device 100 is running, at least one processor 120 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the training method and detection method of the detection network provided in this specification. Processor 120 can execute all steps of the training method and detection method of the detection network. Processor 120 can be in the form of one or more processors. In some embodiments, processor 120 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 120 is described in this specification for computing device 100. However, it should be noted that computing device 100 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 120 of computing device 100 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 120 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0051] Figure 2 A structural diagram 200 of a detection network provided according to some embodiments of this specification is shown. The detection network 200 includes a main task section (…). Figure 2 The part above the dotted line (210) and the auxiliary task part ( Figure 2 The portion below the dashed line (220) includes the detection network used to implement the training method for the detection network described in this specification. For a trained detection network, only the main task part 210 is needed to implement the detection method. The main task part 210 may include an M-layer Transformer network (…). Figure 2Taking M=4 as an example, this M-layer Transformer network includes M encoders, which are used to extract features related to liveness detection. Correspondingly, the main task obtains the detection results of the image to be detected regarding liveness biometrics, and the classification loss value of the main task can be obtained based on these detection results. The auxiliary task may include at least one Contrastively Reinforced Attention (CRA) module, a decoder, and a reference feature map generation module. The auxiliary task can generate a reference feature map of the image to be detected through the reference feature map generation module. By inputting the encoder output during the main task execution into the at least one CRA module, a contrast feature map of the data to be detected is generated. This contrast feature map is then compared with the reference feature map of the image to be detected to obtain a contrast loss value. The joint loss value of the detection network can then be obtained from the classification loss value of the main task and the loss value of the auxiliary task. By feeding this joint loss value back to the detection network, the parameters of the detection network can be adjusted. Training the detection network can be achieved by inputting multiple data sets to be detected. For a trained detection network, inputting the data to be detected into the network will yield the detection results of the image regarding the biological features of the living organism.

[0052] Figure 3 A flowchart of a detection network training method 300 according to some embodiments of this specification is shown. As previously described, computing device 100 can be used to execute the detection network training method 300 described herein. Specifically, processor 120 can read an instruction set stored in its local storage medium and then execute the detection network training method 300 described herein according to the instructions in the instruction set.

[0053] Specifically, such as Figure 3 As shown, the method 300 may include:

[0054] S310: The computing device 100 acquires the detection data of the image to be detected, the image to be detected depicting the biological characteristics of the target object.

[0055] For example, the biometrics may be facial features, iris features, or fingerprint features, etc.

[0056] As previously described, the data acquisition device can collect multiple training data sets and send the collected training data to the computing device 100. Correspondingly, the processor 120 in the computing device 100 can acquire the multiple training data sets. The image to be detected can be any training data set collected by the data acquisition device.

[0057] For ease of understanding, the following description in this specification uses a scenario where the data to be detected is a face image and the detection network is used for liveness detection as an example.

[0058] In some embodiments, the computing device 100 may divide the image to be detected into N blocks of fixed size, and then preprocess the sub-data corresponding to each block into the data to be detected, where N is a natural number greater than 1.

[0059] For example, assuming N=9, we can use Figure 4 The method shown segments the image to be detected. The computing device 100 can divide the image to be detected into 9 blocks according to a predetermined number of rows and columns, such as dividing it into 9 fixed-size blocks Patch1 to Patch9 in a 3x3 grid.

[0060] In order to preprocess the above N blocks into the data to be detected, in some embodiments, the preprocessing of the sub-data corresponding to each block into the data to be detected includes: performing dimensional transformation and concatenation processing on the sub-data corresponding to each block to obtain block embedded data; performing position encoding on the sub-data corresponding to each block to obtain position embedded data; and performing linear mapping on the result of adding the block embedded data and the position embedded data to obtain the data to be detected.

[0061] Specifically, in order to achieve association and fusion between blocks at different locations, computing device 100 needs to perform location encoding on each block to obtain location embedding data. For example, such as... Figure 5 As shown, assuming N equals 9, computing device 100 can use a one-dimensional encoding method to encode the location of different blocks; it can also use a two-dimensional encoding method to encode the location of different blocks. Since face images are multi-dimensional matrix data, and the input to an M-layer Transformer network requires vector data, therefore, as... Figure 6 As shown, the computing device 100 also needs to convert the sub-data corresponding to different blocks Patch1 to Patch9 into corresponding vector data Patch1 ′ Up to Patch 9 ′ Then, the vector data Patch1 ′ Up to Patch 9 ′ The block embedding data is generated by concatenating the data in sequence. Then, the result of adding the block embedding data and the position embedding data is linearly projected to transform the dimensionality of the block data and obtain the detection data (Embedded Patches) of the image to be detected.

[0062] In other embodiments, in order to preprocess the sub-data corresponding to each block into the data to be detected, the computing device 100 may further: perform position encoding on the sub-data corresponding to each block to obtain position embedding data; perform dimensional transformation and connection processing on the sub-data corresponding to different blocks to obtain first block embedding data; obtain type embedding data according to the detection type of the data to be detected by the detection network; concatenate the type embedding data and the first block embedding data to obtain second block embedding data; and perform linear mapping on the result of adding the second block embedding data and the position embedding data to obtain the data to be detected (Embedded Patches).

[0063] Specifically, this is to distinguish different task categories in a multi-tasking scenario, such as face recognition tasks and face liveness detection tasks. Therefore, such as Figure 7 As shown, the computing device 100 can also generate type embedding data for each image to be detected; the computing device 100 can also perform position encoding on each block to obtain position embedding data; the computing device 100 can also convert the sub-data corresponding to each block into vector data, and then concatenate the vector data in order to generate the first block embedding data; then, the computing device 100 concatenates the type embedding data and the first block embedding data to obtain the second block embedding data, and then performs a linear mapping on the result of adding the second block embedding data and the position embedding data to obtain the detection data Embedded Patches of the image to be detected.

[0064] S320: The computing device 100 inputs the data to be detected into an M-layer Transformer network to detect the biological features of the living organism, and obtains the detection result of the image to be detected regarding the biological features of the living organism.

[0065] Specifically, the computing device 100 inputs the embedded patches into an M-layer Transformer network, where M is a natural number greater than 1. For example, this M-layer Transformer network may include M serial encoders. Each encoder is used for liveness feature extraction, and the output of each encoder is a feature map obtained by fusing the liveness features from the aforementioned blocks. Thus, for the i-th encoder (i>1), its input is the feature map F of the embedded patches after i-1 liveness feature extractions. i-1 Its output is for F i-1 Further liveness feature extraction yields the feature map F. i Each time the data passes through an encoder layer, the included biological characteristics are sorted and refined.

[0066] For example, assuming M=4, the encoder structure can be as follows: Figure 8 As shown, the embedded patches are first normalized by a Norm layer to stabilize the distribution of the layer; the output dimension of the Norm layer remains 10x768. Then, the output of the Norm layer passes through a multi-head attention layer, followed by an MLP layer. It's important to note that the output dimension of each encoder layer is the same as the input dimension; therefore, multiple encoders can be concatenated. This concatenated structure can enhance the extraction of live biometric features.

[0067] It should be noted that if the Embedded Patches include class embedding data, the last encoder layer will use the output corresponding to the class embedding data as the final output of the M-layer Transformer network's Embedded Patches, which is the detection result of the image to be detected regarding live biological features. If the Embedded Patches do not include class embedding data, the computing device 100 also needs to connect an MLP layer and a softmax function after the M-layer Transformer network to perform classification processing on the final output of the M-layer Transformer network to obtain the detection result of the image to be detected regarding live biological features.

[0068] Multi-head attention utilizes a self-attention mechanism, which allows the network to consider not just a single vector but a square or rectangular receptive field. This enables the network to focus on and fully learn important information. Therefore, dividing the image to be detected into blocks to obtain the detection data, and then inputting this data into an M-layer Transformer network for processing, allows for the association and fusion of data from different blocks (an effect that cannot be achieved by inputting the entire face image into an M-layer Transformer network). This achieves robust feature extraction and effectively reduces the impact of any single block on the final detection result.

[0069] It should be noted that the detection results of the above-mentioned data to be detected may include liveness category 1 and attack category 0.

[0070] S330: The computing device 100 determines the enhanced correlation data of at least two layers of the output of the M-layer Transformer network with respect to the biological characteristics of the living organism.

[0071] In some embodiments, the computing device 100 fuses the output results of the Mth layer and the (M-1)th layer of the Transformer network into the (M-1)th layer CRA to obtain a first-layer fused output result; it also fuses the fused output result of the i-th layer correlation enhancement module with the output result of the (M-1-i)th layer of the Transformer network into the (i+1)th layer correlation enhancement module to obtain the (i+1)th layer fused output result, wherein the output result of each layer correlation enhancement module enhances the correlation information in its input data, i = any integer between [1, M-1]; and the fused output result of the (M-1)th layer correlation enhancement module is used as the correlation enhancement data regarding the living biological features. Since the data input to each layer correlation enhancement module includes feature maps of Embedded Patches extracted from living features, containing living biological information, and the output result enhances the correlation information between different blocks in its input data, its output data naturally enhances the correlation information between different blocks regarding the living biological features. As a result, the bio-information of living organisms in the data after being fused by the correlation enhancement module of layer M-1 is enhanced step by step.

[0072] For example, such as Figure 2 As shown, assuming M=4, the 4-layer Transformer network has 4 serial encoders, with the output of the 3rd encoder being DataEncoder3 and the output of the 4th encoder being DataEncoder4. First, DataEncoder3 and DataEncoder4 are input into the first correlation enhancement module CRA1 for processing to obtain DataCRA1. Then, DataCRA1 and the outputs of the two encoders, DataEncoder2, are input into the second correlation enhancement module CRA2 for processing to obtain DataCRA2. Finally, DataCRA2 and the output of the first encoder, DataEncoder1, are input into CRA3 for processing to obtain DataCRA3. DataCRA3 is the correlation-enhanced data regarding live biological features. It is evident that DataCRA3 integrates the outputs of each encoder layer; therefore, the M-layer Transformer network can achieve efficient aggregation between features from different blocks.

[0073] In some embodiments, in order to input the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network into the i+1-th layer correlation enhancement module for fusion processing to obtain the i+1-th layer fusion output, the computing device 100 may: perform 1×1 convolution, concatenation and multiplication feature fusion processing on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the i+1-th layer fusion output.

[0074] In some embodiments, to perform 1×1 convolution, concatenation, and multiplication on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the (i+1)-th layer fusion output, the computing device 100 may: perform 1×1 convolution on the fusion output of the i-th layer correlation enhancement module to obtain first convolutional data; perform 1×1 convolution on the output of the M-1-i-th layer of the Transformer network to obtain second convolutional data; and concatenate the first convolutional data and the second convolutional data to obtain first concatenated data. The first concatenated data is subjected to two 1×1 convolutions to obtain the third and fourth convolutional data respectively; the third and fourth convolutional data are concatenated to obtain the second concatenated data; and the second concatenated data is subjected to a 1×1 convolution to obtain the fifth convolutional data. The fifth convolutional data is input into the softmax function for processing. The two outputs of the softmax function are multiplied by the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network, and then added to obtain the fusion output of the (i+1)-th layer.

[0075] For example, the structure of a CRA can be as follows: Figure 9As shown. When running CRA, the computing device 100 can input the output results of the third encoder (DataEncoder3) and the fourth encoder (DataEncoder4) into the first correlation enhancement module CRA1. DataEncoder3 is convolved with a 1×1 layer to obtain DataConv1, and DataEncoder4 is convolved with a 1×1 layer to obtain DataConv2. Then, DataConv1 and DataConv2 are concatenated to obtain DataConcat1. Next, the computing device 100 can perform two 1×1 convolutions on DataConcat1 to obtain DataConv3 and DataConv4 respectively. DataConv3 and DataConv4 are concatenated to obtain DataConcat2, and DataConcat2 is convolved with a 1×1 layer to obtain DataConv4. DataConv4 is then input into the softmax function for processing, resulting in two probability outputs, Para1 and Para2, thus obtaining DataCRA1 = Para1 × DataEncoder3 + Para2 × DataEncoder4.

[0076] It should be noted that the 1×1 convolution operation described above can more flexibly control the number of channels in the input data (i.e., realize information interaction between different channels) and introduce non-linearity, thereby generating a new feature map different from the two input feature maps. Feature concatenation and feature multiplication can gradually enhance the correlation between two features. Therefore, the CRA module can realize the live feature enhancement of feature maps output by different encoders and improve the correlation between different feature maps.

[0077] In some embodiments, to input the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network into the (i+1)-th layer correlation enhancement module for fusion processing to obtain the (i+1)-th layer fusion output, the computing device 100 may further: perform feature fusion processing based on the Inception structure on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network to obtain the (i+1)-th layer fusion output. The basic components of the Inception structure include: 1×1 convolution, 3×3 convolution, 5×5 convolution, and 3×3 max pooling. The results of these four component operations can then be combined channel-wise. By extracting and fusing information from different scales of the image using multiple convolution kernels, a better image representation can be obtained.

[0078] The following section details the method for calculating the loss value of the data to be detected.

[0079] It should be noted that the loss value is obtained by inputting the input data into the corresponding loss function. The appropriate loss function can be selected based on the specific problem being solved. For example, since the main task solves a classification problem, its loss function can be HingeLoss 0-1, and the auxiliary task solves a linear regression problem, its loss function can be the MSE squared loss function.

[0080] S340: The computing device 100 acquires a reference feature map of the image to be detected.

[0081] In one embodiment, the reference feature map may be at least one type of feature map, such as a depth map, an IR map, or a reflection map.

[0082] Specifically, the reference feature map can be generated using a reference feature map generation module. For example, the reference feature map generation module can use a PRNet network. It should be noted that since the original PRNet network can only generate depth maps, if the reference feature map is only a depth map, the image to be detected can be input into the original PRNet network to obtain the depth map of the image to be detected as the reference feature map. If the reference feature map is a combination of several types of feature maps, including depth maps, IR maps, and reflection maps, the PRNet network can be trained to generate the corresponding reference feature map based on the actual situation.

[0083] It should be noted that those skilled in the art can also generate the reference feature map using other neural network models, and this application embodiment does not limit this.

[0084] S350: The computing device 100 obtains a classification loss value based on the detection results of the image data to be detected regarding the living biological features, obtains a contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding the living biological features, and obtains a joint loss value based on the classification loss value and the contrastive feature loss value.

[0085] For example, the detection results of the image data to be detected regarding the biological features of a living organism can be input into the classification loss function HingeLoss 0-1 to obtain its corresponding classification loss value Loss_Binary.

[0086] In some embodiments, obtaining a contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding live biological features includes: obtaining a contrastive feature map based on the correlation enhancement data regarding live biological features, wherein the contrastive feature map is an image of the type corresponding to the reference feature map; and obtaining a contrastive feature loss value of the image to be detected based on the reference feature map and the contrastive feature map.

[0087] For example, if the reference feature map is a depth map, then the corresponding contrastive feature map is also a depth map. This contrastive feature map can be obtained by inputting the correlation-enhanced data about the live biological features into the decoder after processing it through relevant dimensions such as reshaping. Then, according to the Mean Square Error (MSE) loss function, the contrastive loss value Loss_MSE of the reference feature map and the contrastive feature map can be obtained.

[0088] It should be noted that because the output DataCRA3 after CRA processing enhances the liveness detection features of the data to be detected, the contrast feature map of DataCRA3 after decoding more clearly reflects whether the liveness features are sufficient than the reference feature map. Taking the reference feature map as a depth map as an example, for a liveness attack scenario, the reference depth map may still contain depth information. However, the contrast depth map output by the decoder after CRA processing mainly captures liveness detection features. For a liveness attack scenario, the contrast depth map output by the decoder may be flat or black, lacking liveness information. Therefore, the contrast depth map has the ability to distinguish liveness attacks. Thus, calculating the contrast feature loss value between the contrast depth map and the reference depth map plays an important role in training the detection network.

[0089] Then, for example, the classification loss value Loss_Binary and the contrastive feature loss value Loss_MSE can be directly added together to obtain the joint loss value Joint_Loss, i.e., Joint_Loss = Loss_Binary + Loss_MSE. Alternatively, different weight coefficients Para_Loss_Binary and Para_Loss_MSE can be configured for the classification loss value and the contrastive feature loss value. Then, the classification loss value and the contrastive feature loss value can be multiplied by their respective weight coefficients and then added together to obtain the joint loss value.

[0090] Joint_Loss=Para_Loss_Binary×Loss_Binary+Para_Loss_MSE×Loss_MSE.

[0091] S360: The computing device 100 trains the detection network based on the joint loss value.

[0092] Specifically, the joint loss value Joint_Loss is fed forward to the detection network so that the detection network can optimize the parameters of the target detection network based on JointLoss, thereby training the detection network.

[0093] It should be noted that the M-layer Transformer network, CRA module, Inception network, etc. mentioned in this specification are all network structures, which together form the overall detection network structure. These network structures can be one or more instruction sets that execute corresponding instructions stored in the storage medium 130 and executed by the processor 120 in the computing device 100. These network structures can also be a part of the circuitry, hardware devices, or modules in the computing device 100.

[0094] After training the detection network using method 300, the detection network can be used to detect target data. The detection network can be a liveness detection network, a vehicle detection network, etc.

[0095] Figure 10 A flowchart of a detection method 1000 provided according to some embodiments of this specification is shown. As previously mentioned, the computing device 100 can also be used to execute the detection method 1000 described herein. Specifically, the processor 120 can read an instruction set stored in its local storage medium and then execute the detection method 1000 described herein according to the provisions of the instruction set. Figure 10 As shown, the method 1000 may include:

[0096] S1010: The computing device 100 acquires an image to be detected, wherein the image to be detected depicts the biometric features of the target object.

[0097] S1020: The computing device 100 inputs the image to be detected into the detection network and performs detection on the image to be detected.

[0098] S1030: The computing device 100 outputs the detection results of the image to be detected regarding the biological features of the living organism.

[0099] The detection network is trained according to the detection network training method described in the detection method 300 above.

[0100] In summary, this specification provides a training method, detection method, and system for a detection network. This application does not overly rely on the reasonable distribution of multiple training data and does not require multimodal hardware devices as a foundation, thus possessing high practical value. Furthermore, it achieves efficient aggregation of features between different blocks by robustly extracting features from each block of the image to be detected based on an M-layer Transformer network structure, and by determining the correlation enhancement data of at least two layers of the M-layer Transformer network with respect to live biological features. Moreover, it obtains a classification loss value based on the detection results of the image data with respect to live biological features, a contrastive feature loss value based on the reference feature map and the correlation enhancement data with respect to live biological features, and then obtains a joint loss value based on the classification loss value and the contrastive feature loss value, performing joint end-to-end optimization using the joint loss value. This improves the accuracy and robustness of live attack detection.

[0101] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing data processing. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the training method 300 and detection method 1000 of the detection network described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 100, the program code causes the computing device 100 to perform the steps of the training method 300 and detection method 1000 of the detection network described in this specification. The program product for implementing the above methods may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the computing device 100. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 100, partially on computing device 100, as a standalone software package, partially on computing device 100 and partially on a remote computing device, or entirely on a remote computing device.

[0102] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0103] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0104] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0105] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0106] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0107] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for training a detection network, the method comprising: Acquire detection data of an image to be detected, wherein the image to be detected represents the biological characteristics of the target object; The data to be detected is input into a cascaded M-layer Transformer network to detect live biological features, and the detection results of the image to be detected regarding live biological features are obtained. Multiple cascaded correlation enhancement modules are used to obtain correlation-enhanced data on live biological features. The (i+1)th layer correlation enhancement module is used to fuse the fusion output of the i-th layer correlation enhancement module with the output of the M-1-i layer of the Transformer network to enhance the correlation information between live biological features in different blocks of the input data. Here, i = any integer between [1, M-1]. The fusion process includes: performing convolution and concatenation on the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i layer of the Transformer network to obtain fifth convolution data; performing softmax processing on the fifth convolution data; and multiplying and adding the two softmax outputs by the fusion output of the i-th layer correlation enhancement module and the output of the M-1-i layer of the Transformer network, respectively. Obtain a reference feature map of the image to be detected; A classification loss value is obtained based on the detection results of the data to be detected regarding the live biological features; a contrastive feature loss value is obtained based on the reference feature map and the correlation enhancement data regarding the live biological features; and a joint loss value is obtained based on the classification loss value and the contrastive feature loss value. The detection network is trained based on the joint loss value. Where M is a natural number greater than 1.

2. The method as described in claim 1, wherein the detection network is a liveness attack detection network, and the detection result includes liveness category and attack category.

3. The method of claim 1, wherein acquiring the detection data of the image to be detected includes: The image to be detected is divided into N blocks, and the sub-data corresponding to each block is preprocessed to become the data to be detected, where N is a natural number greater than 1.

4. The method as described in claim 3, wherein the preprocessing of the sub-data corresponding to each block into the data to be detected includes: The sub-data corresponding to each block is subjected to dimensional transformation and connection processing to obtain the block embedded data; The sub-data corresponding to each block is position-encoded to obtain position-embedded data; as well as The result of adding the block embedding data and the location embedding data is linearly mapped to obtain the data to be detected.

5. The method as described in claim 3, wherein preprocessing the sub-data corresponding to each block into the data to be detected includes: The sub-data corresponding to each block is subjected to dimensional transformation and connection processing to obtain the first block embedded data; Based on the detection type characterized by the data to be detected, type embedding data is obtained; The type-embedded data and the first block-embedded data are concatenated to obtain the second block-embedded data; The location-encoded sub-data corresponding to each block is used to obtain location-embedded data; as well as The result of adding the second block embedded data and the location embedded data is linearly mapped to obtain the data to be detected.

6. The method of claim 1, wherein the correlation enhancement module comprises an M-1 layer. The first-layer correlation enhancement module is used to fuse the output results of the Mth layer and the (M-1)th layer of the Transformer network to obtain the fused output result of the first-layer correlation enhancement module. The obtained enhanced correlation data regarding living biological characteristics includes: The fusion output of the correlation enhancement module at layer M-1 is used as the correlation enhancement data for the living biological characteristics.

7. The method as described in claim 1, wherein the step of convolving and concatenating the fusion output of the i-th layer correlation enhancement module with the output of the M-1-i-th layer of the Transformer network to obtain the fifth convolutional data includes: The fusion output of the i-th layer correlation enhancement module is subjected to a 1×1 convolution to obtain the first convolutional data; The output of the M-1-ith layer of the Transformer network is convolved with a 1×1 convolution to obtain the second convolutional data. The first convolutional data and the second convolutional data are concatenated to obtain the first concatenated data; The first concatenated data is subjected to two 1×1 convolutions to obtain the third and fourth convolutional data respectively. The third and fourth convolutional data are concatenated to obtain the second concatenated data; and The second concatenated data is convolved with a 1×1 convolution to obtain the fifth convolution data.

8. The method as described in claim 6, wherein the step of inputting the fusion output result of the i-th layer correlation enhancement module and the output result of the M-1-i-th layer of the Transformer network into the (i+1)-th layer correlation enhancement module for fusion processing to obtain the (i+1)-th layer fusion output result includes: The fusion output of the i-th layer correlation enhancement module and the output of the M-1-i-th layer of the Transformer network are subjected to feature fusion processing based on the Inception network to obtain the fusion output of the (i+1)-th layer.

9. The method of claim 1, wherein obtaining the contrastive feature loss value based on the reference feature map and the correlation enhancement data regarding the living biological features comprises: A contrast feature map is obtained based on the correlation enhancement data regarding the characteristics of living organisms, wherein the contrast feature map is an image of the same type as the reference feature map; as well as Based on the reference feature map and the contrast feature map, the contrast feature loss value of the image to be detected is obtained.

10. The method of claim 1, wherein the reference feature map is at least one type of feature map selected from depth map, IR map, or reflection map.

11. A detection method, comprising: Acquire an image to be detected, wherein the image to be detected depicts the biological characteristics of the target object; The image to be detected is input into a detection network for detection; and Output the detection results of the image to be detected regarding the biological features of the living organism. The detection network is trained according to the detection network training method as described in any one of claims 1-10.

12. A detection system, comprising: At least one storage medium storing a detection network for detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the detection system is running, the at least one processor reads the detection network and implements the detection method of claim 11.

Images