A living body detection method and device

Abstract

The application discloses a living body detection method and device, adopts a two-way 2D self-attention architecture, enhances the expression ability of the whole living body detection process from the network structure itself, obtains attention more in line with a face tampering task, and improves the accuracy of living body detection; and / or, introduces a self-supervised constraint into a living body detection task for the first time, solves the problem that attack clues cannot be constrained because of no real label, so that the attack clue map fully expresses attack data, and the accuracy of living body detection is improved.

Description

Technical Field

[0001] This application relates to but is not limited to artificial intelligence technology, and in particular to a method and device for liveness detection. Background Art

[0002] Face Anti-spoof (FAS) technology, also known as liveness detection, is used to determine whether a captured face is real or a forged face attack (such as a face image printed on color paper, a digital face image on an electronic device screen, or a mask), thereby preventing false authentication. In applications such as financial payments and access control, liveness detection is typically a module embedded in face detection and face recognition / verification to verify the user's authenticity.

[0003] The computer vision problem corresponding to liveness detection is a classification problem, which can be viewed as a binary classification problem (true or false) or a multi-classification problem (such as real person, paper attack, screen attack, mask attack, etc.).

[0004] In related technologies, the regression of clue graphs is only constrained on real-person data. The attack data has no supervision and is completely free to iterate. In other words, the clue graph's ability to express attack data is limited. Moreover, the network structure is not designed based on the characteristics of the face tampering task, which greatly affects the accuracy of liveness detection. Summary of the Invention

[0005] The present application provides a liveness detection method and device, which can improve the accuracy of liveness detection.

[0006] An embodiment of the present invention provides a liveness detection method, comprising:

[0007] Divide the decoded image into a face area and a background area;

[0008] Performing convolution processing and normalization processing on the face area to obtain first image information, and performing convolution processing on the face area and the background area respectively and then normalizing them to obtain second image information;

[0009] The first image information and the second image information are fused to obtain live attack detection information.

[0010] In an exemplary embodiment, before dividing the decoded image into the face area and the background area, the method further includes:

[0011] A cascaded encoder-decoder structure is used to encode and decode the image to be detected.

[0012] In one exemplary embodiment, the encoding process includes one or more stages, and the decoding process includes one or more stages;

[0013] When the decoding process includes multiple stages, the image decoded by each stage is input into the next stage after being processed by the living body detection method, until there is no decoder.

[0014] In an exemplary example, the convolution and normalization processing of the face region to obtain the first image information includes:

[0015] The 2D convolution is used on the face region to obtain a first vector Q, a first vector K, and a first vector V, and the vectors are rearranged into one-dimensional features;

[0016] The matrix multiplication is calculated according to the first vector Q and the first vector K to obtain a first attention map E;

[0017] The first attention map E is normalized;

[0018] The first result after the normalization is multiplied by the first vector V, and the result is rearranged into an image of the size of the face region;

[0019] The image of the size of the face region after the rearrangement is added to the image decoded by the squirrel to obtain the first image information.

[0020] In an exemplary example, the convolution and normalization processing of the face region and the background region to obtain the second image information includes:

[0021] The 2D convolution is used on the face region and the background region to obtain a second vector Q', a second vector K', and a second vector V', and the vectors are rearranged into one-dimensional features;

[0022] The matrix multiplication is calculated according to the second vector Q' and the second vector K' to obtain a second attention map E';

[0023] The second attention map E' is normalized;

[0024] The second result after the normalization is multiplied by the second vector V', and the result is rearranged into an image of the size of the face region;

[0025] The result after the normalization is added to the decoded image to obtain the second image information.

[0026] In an exemplary example, the method further includes:

[0027] Triplet metric learning is performed according to the result of the last stage of encoding and the living body attack detection information of each stage after the decoded image is processed by the living body detection method;

[0028] After the fusion, the following is further included:

[0029] The regression loss processing is performed on the living body attack detection information obtained after the fusion.

[0030] The attack clue map is obtained by superimposing the attack clue map after the regression loss processing on the to-be-detected image and performing binary classification.

[0031] In an exemplary instance, the regression loss processing further includes, before the regression loss processing:

[0032] The attack data obtained after the fusion of the first image information and the second image information is subjected to a data expansion operation.

[0033] The regression loss processing includes: performing regression loss processing on the result after the expansion operation and attack clue data to obtain an attack clue map.

[0034] The embodiments of the present application further provide a device for implementing living body detection, including a memory and a processor, wherein the memory stores instructions executable by the processor, including steps of the living body detection method.

[0035] The embodiments of the present application further provide another living body detection method, including:

[0036] The attack data is subjected to a data expansion operation.

[0037] The regression loss processing is performed on the result after the expansion operation and attack clue data to obtain an attack clue map.

[0038] In an exemplary instance, the attack data is subjected to a data expansion operation, and before the data expansion operation, the method further includes:

[0039] The to-be-detected image is subjected to encoding processing and decoding processing by using a cascaded encoder-decoder structure; the encoding processing includes one or more levels, and the decoding processing includes one or more levels.

[0040] Triplet metric learning is performed according to the result of the last level of encoding and the result of each level of decoding.

[0041] In an exemplary instance, the regression loss processing is performed on the result after the expansion operation and attack clue data to obtain an attack clue map, and the regression loss processing includes:

[0042] Supervised information of the attack clue itself is introduced by using a loss function for constraint, wherein the self-supervised clue regression loss function is as follows:

[0043]

[0044] Wherein, Lr represents the obtained attack clue map, live represents a live library, spoof represents an attack library; exp represents an expansion operation, C represents an attack clue of attack data I, and C' represents an attack clue of attack data after the data expansion operation.

[0045] In an exemplary instance, the method further comprises:

[0046] Superimposing the regression loss processed attack clue map and the original to-be-detected image and performing binary classification to obtain a final attack clue map.

[0047] In an exemplary instance, before the data expansion operation is performed on the attack data, the method further comprises:

[0048] An encoder-decoder structure is adopted to perform encoding processing and decoding processing on the to-be-detected image; the encoding processing comprises one or more levels, and the decoding processing comprises one or more levels;

[0049] The decoded image is divided into a face region and a background region, convolution processing and normalization processing are performed on the face region to obtain first image information, and convolution processing is performed on the face region and the background region respectively and then normalization processing is performed to obtain second image information;

[0050] When the decoding processing comprises multiple levels, the decoded image of each level is input into a next level decoder after being processed by the live body detection method until there is no decoder;

[0051] Triplet metric learning is performed on the result of the last level of encoding and the live body attack detection information of each level after the decoded image is processed by the live body detection method;

[0052] The first image information and the second image information are fused.

[0053] Embodiments of the present application further provide a device for implementing live body detection, comprising a memory and a processor, wherein the memory stores instructions executable by the processor, for performing steps of another live body detection method described in any of the above embodiments.

[0054] Embodiments of the present application further provide a live body detection method, comprising:

[0055] The decoded image is divided into a face region and a background region, convolution processing and normalization processing are performed on the face region to obtain first image information, and convolution processing is performed on the face region and the background region respectively and then normalization processing is performed to obtain second image information;

[0056] Data expansion operation is performed on the attack data fused according to the first image information and the second image information;

[0057] According to the result of the expansion operation and the attack clue data, regression loss processing is performed to obtain an attack clue map.

[0058] In an exemplary instance, before the decoded image is divided into a face region and a background region, the method further comprises:

[0059] The encoder-decoder structure is adopted to perform encoding processing and decoding processing on the image to be detected.

[0060] In an exemplary instance, the encoding processing comprises one or more levels, and the decoding processing comprises one or more levels.

[0061] When the decoding processing comprises multiple levels, each level of the decoded image is input into a next level of the decoder after being processed by the living body detection method, until there is no decoder.

[0062] In an exemplary instance, the convolution processing and the normalization processing on the face region to obtain the first image information comprises:

[0063] The 2D convolution is used on the face region to obtain a first vector Q, a first vector K, and a first vector V, and the first vector Q, the first vector K, and the first vector V are rearranged into one-dimensional features.

[0064] The matrix multiplication is calculated according to the first vector Q and the first vector K to obtain a first attention map E.

[0065] The first attention map E is normalized.

[0066] The first result after the normalization processing is multiplied by the first vector V, and the first result after the multiplication is rearranged into an image of the size of the face region.

[0067] The image of the size of the face region after the rearrangement is added to the image decoded by the squirrel to obtain the first image information.

[0068] In an exemplary instance, the convolution processing and the normalization processing on the face region and the background region to obtain the second image information comprises:

[0069] The 2D convolution is used on the face region and the background region to obtain a second vector Q', a second vector K', and a second vector V', and the second vector Q', the second vector K', and the second vector V' are rearranged into one-dimensional features.

[0070] The matrix multiplication is calculated according to the second vector Q' and the second vector K' to obtain a second attention map E'.

[0071] The second attention map E' is normalized.

[0072] The normalized second result is multiplied by the second vector V` and rearranged into an image of the size of the face region;

[0073] The normalized result is added to the decoded image to obtain the second image information.

[0074] In an exemplary example, before the data expansion operation is performed on the attack data fused according to the first image information and the second image information, the method further comprises:

[0075] An encoder-decoder structure is used to encode and decode the image to be detected; the encoding process includes one or more levels, and the decoding process includes one or more levels;

[0076] Triplet metric learning is performed according to the result of the last level of encoding and the result of each level of decoding;

[0077] In an exemplary example, the regression loss processing is performed on the result after the expansion operation and the attack clue data to obtain an attack clue map, comprising:

[0078] A loss function is used to introduce the supervision information of the attack clue itself for constraint, wherein the self-supervised clue regression loss function is as follows:

[0079]

[0080] Wherein, Lr represents the obtained attack clue map, live represents a live library, spoof represents an attack library; exp represents an expansion operation, C represents an attack clue of the attack data I, and C` represents an attack clue of the attack data after the data expansion operation.

[0081] In an exemplary example, the method further comprises:

[0082] The attack clue map after the regression loss processing is superimposed on the image to be detected and a binary classification is performed to obtain a final attack clue map.

[0083] Embodiments of the present application also provide a living body detection device, comprising: a division module, a first processing module, a second processing module, and a fusion module, wherein,

[0084] The division module is configured to divide the decoded image into a face region and a background region;

[0085] The first processing module is configured to perform convolution processing and normalization processing on the face region to obtain first image information;

[0086] The second processing module is configured to perform convolution processing on the face region and the background region respectively and then perform normalization processing to obtain second image information;

[0087] The fusion module is configured to fuse the first image information and the second image information to obtain living body attack detection information.

[0088] The application further provides a living body detection device, comprising an expansion module and a regression processing module.

[0089] The expansion module is configured to perform data expansion operation on the attack data.

[0090] The regression processing module is configured to perform regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0091] The application further provides a living body detection device, comprising a division module, a first processing module, a second processing module, a fusion module, an expansion module and a regression processing module.

[0092] The division module is configured to divide the decoded image into a face region and a background region.

[0093] The first processing module is configured to perform convolution processing and normalization processing on the face region to obtain first image information.

[0094] The second processing module is configured to perform convolution processing on the face region and the background region respectively and then perform normalization processing to obtain second image information.

[0095] The fusion module is configured to fuse the first image information and the second image information.

[0096] The expansion module is configured to perform data expansion operation on the attack data fused according to the first image information and the second image information.

[0097] The regression processing module is configured to perform regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0098] An embodiment of the application enhances the expression ability of the entire living body detection process from the network structure itself by adopting a dual-path 2D self-attention architecture, obtains attention more in line with the face tampering task, and improves the accuracy of living body detection.

[0099] Another embodiment of the application first introduces self-supervised constraints into the living body detection task, solves the problem that attack clues cannot be constrained because they have no real labels, so that the attack clue map fully expresses the attack data, and improves the accuracy of living body detection.

[0100] In another embodiment of the present application, on the one hand, a two-way 2D self-attention architecture is adopted, the expression capability of the whole live body detection process is enhanced from the network structure itself, more attention conforming to the face tampering task is obtained, and the accuracy of live body detection is improved; on the other hand, the self-supervised constraint is introduced into the live body detection task for the first time, the problem that the attack clues cannot be constrained because of no real label is solved, so that the attack clue map fully expresses the attack data, and the accuracy of live body detection is improved.

[0101] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0102] The accompanying drawings are included to provide a further understanding of the technical scheme of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical scheme of the present application, and do not constitute a limitation on the technical scheme of the present application.

[0103] Figure 1 It is a network architecture schematic diagram for realizing live body detection;

[0104] Figure 2 It is a flowchart schematic diagram of the first embodiment of the live body detection method of the present application;

[0105] Figure 3 It is a flowchart schematic diagram of the second embodiment of the live body detection method of the present application;

[0106] Figure 4 It is a flowchart schematic diagram of the third embodiment of the live body detection method of the present application;

[0107] Figure 5 It is a network architecture schematic diagram for realizing live body detection in the embodiments of the present application;

[0108] Figure 6 It is a component structure schematic diagram of the first embodiment of the live body detection device in the embodiments of the present application;

[0109] Figure 7 It is a component structure schematic diagram of the second embodiment of the live body detection device in the embodiments of the present application;

[0110] Figure 8 It is a component structure schematic diagram of the third embodiment of the live body detection device in the embodiments of the present application. DETAILED DESCRIPTION

[0111] In order to make the objects, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. It should be explained that the embodiments herein and the features in the embodiments can be combined with each other as long as there is no conflict.

[0112] In one typical configuration of the present application, the computing device includes one or more processors (CPU), input / output interface, network interface, and memory.

[0113] The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory, such as read only memory (ROM) or flash memory, in a computer readable medium. The memory is an example of computer readable media.

[0114] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carriers.

[0115] The steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions. Also, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that herein.

[0116] Figure 1 For a network architecture for realizing living body detection, as shown in Figure 1As shown, the input is face RGB image (normalized -1 to 1), and the output is the original image size of the attack clue map (spoof cue map). In the spoof cue generator, a U-Net architecture (a variant of the fully convolutional neural network) is adopted, and a multi-scale skip connection structure is constructed from the cascaded encoder (Encoder) to the decoder (Decoder) to generate the spoof cue. The encoder E (such as E1-E5 in Figure 1 The encoder E (such as E1-E5 in Figure 1 The decoder D (such as D1-D4 in

[0117] In the triplet loss supervision, there are m positive samples and n negative samples in a batch of data, then (a, p, n) can form n*n*m triplets, and the triplet loss is calculated, so that the class is compact and the class is dispersed, which is the typical content of metric learning. The regression loss, Figure 1 In the related art, the spoof cue only exists in the spoof samples, so the feature map corresponding to the live samples should be a zero matrix, and the normal samples need to be regressed to 0, and the abnormal samples are not concerned, that is, it is considered that the normal samples have a same intra-domain center, which is 0, and the center of the abnormal samples is not fixed, so only the loss of the normal samples is calculated when calculating the regression loss. The classification loss is an auxiliary loss, which is used to enhance the gradient effect and do positive and negative classification.

[0118] However, the present inventors have found that Figure 1The living body detection architecture shown, because the regression of the clue map only constrains the real person data, and has no supervision on the attack data, is completely free to iterate, and the model has limited clue map expression capability on the attack data. That is, Figure 1 The related art shown does not design a network structure for the characteristics of the face tampering task, thereby affecting the accuracy of the living body detection.

[0119] The living body attack contains multiple types of attacks, and the living body attack detection mainly includes: extracting moire, face region material, face texture abnormalities, and other features to construct a classification task for differentiation. In this way, the self-consistency of the face itself and the relationship between the face and the background are not considered, such as inconsistent colors of the left and right eyes, abnormal occlusion of the mouth, abnormal shape of the nose, and background distortion.

[0120] Figure 2 The flowchart of the first embodiment of the living body detection method of the present application is shown in Figure 2 At least includes:

[0121] Step 200: dividing the decoded image into a face region and a background region.

[0122] In an exemplary example, step 200 can further include, before step 200:

[0123] The cascaded encoder-decoder structure shown in Figure 1 The image to be detected can be encoded and decoded using the cascaded encoder-decoder structure.

[0124] In an exemplary example, the encoding process can be one or more levels, and the decoding process can be one or more levels. In combination with the network architecture for living body detection shown in Figure 5 When the decoding process includes multiple levels, the image decoded at each level can use the living body detection method shown in Figure 2 The decoded image is processed and then input to the next level of decoding until there is no decoder.

[0125] Step 201: performing convolution processing and normalization processing on the face region to obtain first image information, and performing convolution processing on the face region and the background region respectively and then performing normalization processing to obtain second image information.

[0126] In an exemplary example, in combination with Figure 5 The convolution processing and normalization processing on the face region in this step to obtain the first image information can include:

[0127] The face region X f A two-dimensional image (2D) convolution is used to obtain a first vector Q, a first vector K, and a first vector V, and they are rearranged into one-dimensional features;

[0128] According to the first vector Q and the first vector K, a matrix multiplication is calculated to obtain a first attention map E;

[0129] The first attention map E is normalized;

[0130] The first result after normalization is multiplied by the first vector V, and is rearranged into an image of the size of the face region;

[0131] The rearranged image of the size of the face region is added to the decoded image to obtain first image information Y.

[0132] In an embodiment, the first attention map E can be normalized using, for example, a Softmax function. In machine learning, especially deep learning, Softmax is a very commonly used and important function, especially in the context of multi-classification. Softmax maps some inputs to real numbers between 0 and 1, and normalization ensures that the sum is 1, so the sum of the probabilities of multi-classification is exactly 1.

[0133] The processing of the first image information obtained by the convolution and normalization of the face region in this step constitutes a self-consistent attention channel for the face, and realizes the extraction of the self-consistency of the face.

[0134] In an exemplary example, as shown in FIG. 2, the second image information obtained by the convolution and normalization of the face region and the background region in this step can include: Figure 5

[0135] The face region X f and the background region X b are respectively processed using 2D convolution to obtain a second vector Q', a second vector K', and a second vector V', which are rearranged into one-dimensional features;

[0136] According to the second vector Q' and the second vector K', a matrix multiplication is calculated to obtain a second attention map E';

[0137] The second attention map E' is normalized;

[0138] The second result after normalization is multiplied by the second vector V', and is rearranged into an image of the size of the face region;

[0139] The result after normalization is added to the decoded image to obtain second image information Y'.

[0140] In an embodiment, the second attention map E' can be normalized using, for example, a Softmax function.

[0141] ​The processing of the face region and the background region in the present step is normalized to obtain the second image information, which forms the attention channel of the correlation between the face and the background, and realizes the extraction of the correlation between the face and the background.

[0142] The face self-consistency attention channel and the face and background correlation attention channel in the embodiments of the present application form a two-path 2D self-attention architecture (as shown in Figure 5 The network structure itself enhances the expression ability of the entire live body detection process, and can obtain attention more suitable for the face tampering task, thereby improving the accuracy of live body detection.

[0143] Step 202: Fusion of the first image information and the second image information to obtain live body attack detection information.

[0144] In an exemplary example, the present step can be realized by operations such as concat, add, etc., where concat is an increase in the number of channels, and add is a feature map addition with unchanged channel number.

[0145] The two-path 2D self-attention architecture provided by the embodiments of the present application can be easily embedded into other systems and used in the network backbone of any live body detection task. The two-path 2D self-attention architecture customized for face live body attack detection simultaneously extracts face self-consistency and face and background correlation, and can be applied to other tasks with similar constraint relationships.

[0146] In an exemplary example, during the execution of steps 200-202, the following is further included:

[0147] Triplet metric learning is performed according to the result of the last level coding and each level of live body attack detection information processed by the live body detection method shown in steps 200-202;

[0148] After step 202, the following can be further included: regression loss processing is performed on the fused live body attack detection information;

[0149] The attack clue map after regression loss processing is superimposed on the original image to be detected and binary classification is performed to obtain the final attack clue map.

[0150] In an exemplary example, as shown in Figure 5 The auxiliary classifier such as ResNet18 can be used to realize binary classification.

[0151] The live body detection method provided by the embodiments of the present application uses a two-path 2D self-attention architecture, which enhances the expression ability of the entire live body detection process from the network structure itself, obtains attention more suitable for the face tampering task, and improves the accuracy of live body detection.

[0152] The application further provides a computer readable storage medium storing computer executable instructions for executing the live body detection method of any one of the above Figure 2 The application further provides a computer readable storage medium storing computer executable instructions for executing the live body detection method of any one of the above

[0153] The application further provides a device for implementing the live body detection method, comprising a memory and a processor, wherein the memory stores instructions executable by the processor, for executing the steps of the live body detection method of any one of the above Figure 2 The application further provides a computer readable storage medium storing computer executable instructions for executing the live body detection method of any one of the above

[0154] Figure 3 The flowchart of the second embodiment of the live body detection method of the application is shown in FIG. 3, which comprises at least the following steps: Figure 3

[0155] Step 300: performing data expansion operation on the attack data.

[0156] In an exemplary example, the step further comprises the following steps before the step:

[0157] The encoder-decoder structure shown in FIG. 4 can be used for encoding and decoding the image to be detected. Figure 1

[0158] Triplet metric learning is performed according to the result of the last level of encoding and the result of each level of decoding.

[0159] In an exemplary example, the encoding process can be one or more levels, and the decoding process can be one or more levels.

[0160] In an exemplary example, the expansion operation in step 300 can include but is not limited to data augmentation of the same attack data, such as face attack data, using one or any combination of data expansion methods such as cropping, scaling, rotation, Mixup, etc.

[0161] For example, assume that the original attack data, i.e., the attack data in step 300, is represented as I, and the data expansion operation is represented as Exp. Then, the data expansion operation on the attack data I can be represented as Exp(I). Assume that the attack clue of the attack data I is represented as C, then the attack clue of Exp(I) can be represented as C`. In this way, if the data expansion operation is known, C` = Exp(C) can be obtained, that is, the attack clue C and the clue C` of the corresponding expanded data are consistent.

[0162] Step 301: performing regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0163] In an exemplary example, the regression loss processing can be combined with​​Figure 5 As shown, the attack clue itself can be constrained by introducing supervised information using L1, L2 distance or NCE Loss, tripletLoss, etc. The self-supervised clue regression loss function is shown in formula (1):

[0164]

[0165] In the above formula, live represents the live library, spoof represents the attack library; exp represents the expansion operation, C represents the attack clue of the attack data I, and C' represents the attack clue of the attack data after the data expansion operation. Through formula (1), not only the real person clue graph is constrained, but also the similarity of the attack data and the expanded attack data clue graph is increased.

[0166] In an exemplary instance, it can also include:

[0167] The attack clue graph processed by the regression loss is superimposed on the original image to be detected and binary classification is performed to obtain the final attack clue graph.

[0168] In an exemplary instance, a secondary classifier such as ResNet18 can be used to implement binary classification.

[0169] In the embodiments of the present application, the regression of the clue graph not only constrains the real person data, but also constrains the attack data. When collecting face data, the real person has no attack clue, and the attack clues of different face live attacks are different. The self-supervised constraint is first introduced into the live detection task, solving the problem that the attack clue has no real label and cannot be constrained, so that the attack clue graph fully expresses the attack data, and the accuracy of live detection is improved.

[0170] The present application also provides a computer readable storage medium storing computer executable instructions for executing the live detection method of any one of the above Figure 3 The live detection method of any one of the above.

[0171] The present application further provides a device for implementing live detection, comprising a memory and a processor, wherein the memory stores instructions executable by the processor: for executing the steps of the live detection method of any one of the above Figure 3 The live detection method of any one of the above.

[0172] Figure 4 The flowchart of the third embodiment of the live detection method of the present application is shown in the above Figure 4 As shown, at least includes:

[0173] Step 400: dividing the decoded image into a face region and a background region, performing convolution processing and normalization processing on the face region to obtain first image information, and performing convolution processing on the face region and the background region respectively and then performing normalization processing to obtain second image information.

[0174] In an exemplary embodiment, step 400 can further include:

[0175] The encoder-decoder structure shown in FIG. 4 can be used for encoding and decoding the image to be detected. Figure 1

[0176] In an exemplary embodiment, the encoding processing can be one or more levels, and the decoding processing can be one or more levels. When the decoding processing includes multiple levels, the image decoded at each level can be processed using the living body detection method shown in FIG. 4 before being input to the next level of decoding. Figure 2

[0177] In an exemplary embodiment, the convolution processing and normalization processing on the face region in step 400 to obtain the first image information can include:

[0178] The face region X f The 2D convolution is used to obtain a first vector Q, a first vector K, and a first vector V, which are rearranged into one-dimensional features.

[0179] The first vector Q and the first vector K are used to calculate matrix multiplication to obtain a first attention map E.

[0180] The first attention map E is normalized.

[0181] The normalized first result is multiplied by the first vector V, and the result is rearranged into an image of the size of the face region.

[0182] The rearranged image of the size of the face region is added to the decoded image to obtain the first image information Y.

[0183] In an embodiment, the Softmax function can be used to normalize the first attention map E. In machine learning, especially deep learning, Softmax is a very common and important function, especially widely used in multi-classification scenarios. Softmax maps some inputs to real numbers between 0 and 1, and normalization ensures that the sum is 1, so the sum of the probabilities of multi-classification is exactly 1.

[0184] Through the convolution processing and normalization processing on the face region in step 400 to obtain the first image information, a self-consistent attention channel for the face is formed, and the self-consistent extraction of the face is achieved. ​​

[0185] In an example, the normalization processing after the convolution processing of the face region and the background region in step 400 to obtain the second image information can include:

[0186] The face region X f and the background region X b are respectively processed using 2D convolution to obtain the second vector Q', the second vector K', and the second vector V', and are rearranged into one-dimensional features;

[0187] The matrix multiplication is calculated according to the second vector Q' and the second vector K' to obtain the second attention map E';

[0188] The second attention map E' is normalized;

[0189] The normalized second result is multiplied by the second vector V', and is rearranged into an image of the size of the face region;

[0190] The normalized result is added to the decoded image to obtain the second image information Y'.

[0191] In an embodiment, the Softmax function can be used to normalize the second attention map E'.

[0192] The processing of the normalization processing after the convolution processing of the face region and the background region in step 400 to obtain the second image information constitutes an attention channel of the correlation between the face and the background, and realizes the extraction of the correlation between the face and the background.

[0193] Step 401: performing data expansion operation on the attack data fused according to the first image information and the second image information.

[0194] In an example, before this step, a ternary triplet metric learning is further performed according to the result of the last level encoding and each level of living body attack detection information processed by the living body detection method shown in steps 200-202.

[0195] The first image information and the second image information are fused to obtain the attack data.

[0196] In an example, the fusion in this step can be realized by operations such as concat, add, etc., where concat is an increase in the number of channels, add is a feature map addition, and the number of channels remains unchanged.

[0197] In an example, the expansion operation in step 401 can include but is not limited to: for the same attack data such as face attack data, using one or any combination of data expansion methods such as cropping, scaling, rotation, Mixup, etc. for data augmentation.

[0198] For example, assume that the original attack data, i.e., the attack data in step 401, is denoted as I, and the data expansion operation is denoted as Exp. Then, the data expansion operation on the attack data I can be denoted as Exp(I). Assume that the attack clue of the attack data I is denoted as C, and then the attack clue of Exp(I) can be denoted as C`. In this way, if the data expansion operation is known, C` = Exp(C) can be obtained, that is, the attack clue C and the clue C` of the corresponding expanded data are consistent.

[0199] Step 402: Perform regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0200] In an example, the loss function such as L1, L2 distance or NCE Loss, triplet Loss, etc. can be used to introduce the supervision information of the attack clue itself for constraint. The self-supervised clue regression loss function is shown in formula (1).

[0201] In an example, the following can also be included:

[0202] Superimpose the attack clue map after the regression loss processing on the original image to be detected and perform binary classification to obtain a final attack clue map.

[0203] In an example, the auxiliary classifier such as ResNet18 can be used to implement the binary classification.

[0204] The living body detection method provided by the embodiment of the application has the following advantages. On the one hand, the dual-path 2D self-attention architecture is used to enhance the expression ability of the entire living body detection process from the network structure itself, obtain attention that is more consistent with the face tampering task, and improve the accuracy of living body detection. On the other hand, the regression of the clue map not only constrains the real person data but also constrains the attack data, solves the problem that the attack clue cannot be constrained because of the lack of real labels, and thus makes the attack clue map fully express the attack data. The living body detection method provided by the embodiment of the application abstracts the living body attack detection into an attack clue regression task, solves the problem that the attack clue cannot be constrained because of the lack of real labels, and at the same time, the self-attention architecture designed for the living body attack task enhances the expression ability of the attack clue from the network structure, and improves the accuracy of living body detection.

[0205] The application further provides a computer-readable storage medium storing computer-executable instructions for executing the living body detection method of any one of the above Figure 4 embodiments.

[0206] The application further provides a device for implementing the living body detection, comprising a memory and a processor, wherein the memory stores instructions executable by the processor, the instructions comprising instructions for performing the living body detection method of any one of the above. Figure 4 The application further provides a device for implementing the living body detection method of any one of the above.

[0207] In an exemplary instance, Figure 2 In the living body detection method, the regression loss processing further comprises:

[0208] The attack data fused from the first image information and the second image information is subjected to a data expansion operation.

[0209] The regression loss processing comprises: performing regression loss processing on the result of the expansion operation and the attack clue data to obtain an attack clue map.

[0210] In an exemplary instance, Figure 3 In the living body detection method, before the attack data is subjected to the data expansion operation, the method can further comprise:

[0211] The encoder-decoder structure is adopted to perform encoding processing and decoding processing on the image to be detected; the encoding processing comprises one or more levels, and the decoding processing comprises one or more levels.

[0212] The decoded image is divided into a face region and a background region, the face region is subjected to convolution processing and normalization processing to obtain the first image information, and the face region and the background region are respectively subjected to convolution processing and then normalization processing to obtain the second image information.

[0213] When the decoding processing comprises multiple levels, the image decoded by each level is subjected to the processing of the living body detection method on the decoded image and then input into a next level decoder until there is no decoder.

[0214] Triplet metric learning is performed on the result of the last level encoding and each level living body attack detection information processed by the living body detection method shown in steps 200-202.

[0215] The first image information and the second image information are fused.

[0216] The living body detection method provided by the application can be applied to scenarios requiring face recognition, such as logging in through a financial APP, paying through a shopping APP, signing through a government affair APP, and the like, so as to better avoid possible face attacks.

[0217] Figure 6 A schematic diagram of the component structure of the first embodiment of the living body detection device in the application is shown in FIG. 1. Figure 6As shown, at least comprising: a division module, a first processing module, a second processing module, a fusion module; wherein,

[0218] The division module is configured to divide the decoded image into a face region and a background region.

[0219] The first processing module is configured to perform convolution processing and normalization processing on the face region to obtain first image information.

[0220] The second processing module is configured to perform convolution processing on the face region and the background region respectively, and then perform normalization processing to obtain second image information.

[0221] The fusion module is configured to fuse the first image information and the second image information to obtain living body attack detection information.

[0222] In an exemplary example, it can further comprise: a cascaded encoder-decoder structure configured to perform encoding processing and decoding processing on the image to be detected.

[0223] In an exemplary example, the encoding processing can be one or more levels, and the decoding processing can be one or more levels. For example, Figure 5 As shown, when the decoding processing includes multiple levels, the image decoded by each level can be processed by the living body detection device of the present application Figure 6 As shown, the living body detection device processes the decoded image and then inputs it to the next level of decoding until there is no decoder.

[0224] In an exemplary example, it can further comprise: a third processing module, a fourth processing module; wherein,

[0225] The third processing module is configured to perform triplet metric learning on the living body attack detection information of each level after processing of the living body detection device according to the result of the last level of encoding. Figure 6

[0226] The fourth processing module is configured to perform regression loss processing on the living body attack detection information obtained after fusion; superimpose the attack clue map after regression loss processing on the original image to be detected and perform binary classification to obtain the final attack clue map.

[0227] In an exemplary example, the fourth processing module is further configured to: perform data expansion operation on the attack data fused according to the first image information and the second image information; and the regression loss processing in the fourth processing module includes: performing regression loss processing on the result after the expansion operation and the attack clue data to obtain the attack clue map.

[0228] ​The living body detection device provided by the embodiment of the application adopts a two-way 2D self-attention architecture, enhances the expression capability of the whole living body detection process from the network structure itself, obtains attention more in line with the face tampering task, and improves the accuracy of living body detection.

[0229] Figure 7 As shown in a constituent structure schematic diagram of a second embodiment of the living body detection device in the embodiment of the application, Figure 7 at least includes: an expansion module, a regression processing module; wherein,

[0230] The expansion module is configured to perform data expansion operation on the attack data.

[0231] The regression processing module is configured to perform regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0232] In an exemplary instance, it can further include: a cascaded encoder-decoder structure, a fifth processing module; wherein,

[0233] The cascaded encoder-decoder structure is configured to perform encoding processing and decoding processing on the to-be-detected image.

[0234] The fifth processing module is configured to perform triplet metric learning according to the result of the last level of encoding and the result of each level of decoding.

[0235] In an exemplary instance, the encoding processing can be one level or multiple levels, and the decoding processing can be one level or multiple levels.

[0236] In an exemplary instance, the expansion module is further configured to:

[0237] divide the decoded image into a face region and a background region, perform convolution processing and normalization processing on the face region to obtain first image information, and perform convolution processing on the face region and the background region respectively and then perform normalization processing to obtain second image information;

[0238] perform triplet metric learning according to the result of the last level of encoding and the result of each level of living body attack detection information after processing of the living body detection device shown in Figure 6 .

[0239] fuse the first image information and the second image information.

[0240] In an exemplary instance, as shown in Figure 5 when the decoding processing includes multiple levels, the image decoded at each level can be processed by the living body detection device shown in the embodiment of the application and then input to the next level of decoding. Figure 6

[0241] ​In an example, the method further includes a sixth processing module configured to:

[0242] The attack clue map processed by the regression loss is superimposed on the original image to be detected and classified to obtain a final attack clue map.

[0243] The living body detection device provided by the embodiment of the application not only constrains the real person data, but also constrains the attack data. When the face data is collected, the real person does not have an attack clue, and the attack clues of different face living body attacks are different. The self-supervised constraint is introduced into the living body detection task for the first time, and the problem that the attack clue has no real label and cannot be constrained is solved, so that the attack clue map fully expresses the attack data, and the accuracy of the living body detection is improved.

[0244] Figure 8 As shown in FIG. 1, the living body detection device includes a division module, a first processing module, a second processing module, a fusion module, an expansion module, and a regression processing module. Figure 8

[0245] The division module is configured to divide the decoded image into a face region and a background region.

[0246] The first processing module is configured to perform convolution processing and normalization processing on the face region to obtain first image information.

[0247] The second processing module is configured to perform convolution processing on the face region and the background region respectively, and then perform normalization processing to obtain second image information.

[0248] The fusion module is configured to fuse the first image information and the second image information.

[0249] The expansion module is configured to perform data expansion operation on the attack data fused according to the first image information and the second image information.

[0250] The regression processing module is configured to perform regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue map.

[0251] In an example, the method further includes:

[0252] The cascaded encoder-decoder structure is configured to perform encoding processing and decoding processing on the image to be detected. The encoding processing includes one or more levels, and the decoding processing includes one or more levels.

[0253] When the decoding processing includes multiple levels, the image decoded by each level is input into the next level after being processed by the living body detection device.​

[0254] The living body detection device provided by the embodiment of the present application, on the one hand, adopts a two-way 2D self-attention architecture, enhances the expression ability of the whole living body detection process from the network structure itself, obtains attention more in line with the face tampering task, and improves the accuracy of living body detection. On the other hand, the regression of the clue graph not only constrains the real person data, but also constrains the attack data, solves the problem that the attack clue has no real label and cannot be constrained, so that the attack clue graph fully expresses the attack data. The living body detection method provided by the embodiment of the present application abstracts the living body attack detection into an attack clue regression task, solves the problem that the attack clue has no real label and cannot be constrained, and at the same time, the self-attention architecture designed for the living body attack task enhances the expression ability of the attack clue of the living body from the network structure, and improves the accuracy of the living body detection.

[0255] Although the embodiments disclosed in the present application are as described above, the content described is only the embodiments adopted for the purpose of facilitating the understanding of the present application, and is not intended to limit the present application. Any person skilled in the art of the present application can make any modification and change in the form and details without departing from the spirit and scope of the present application, but the patent protection scope of the present application shall be subject to the scope defined by the appended claims.

Claims

1. A liveness detection method, comprising: Divide the decoded image into a face area and a background area; Performing convolution processing and normalization processing on the face area to obtain first image information, and performing convolution processing on the face area and the background area respectively and then normalizing them to obtain second image information; fusing the first image information and the second image information to obtain live attack detection information; The performing convolution processing and normalization processing on the facial region to obtain first image information includes: Performing a 2D convolution on the facial region to obtain a first vector Q, a first vector K, and a first vector V, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the first vector Q and the first vector K to obtain the first attention map E; Normalize the first attention map E; Multiplying the first normalized result by the first vector V and rearranging the result into an image of the size of the face region; adding the rearranged image of the face region size to the decoded image to obtain the first image information; The face area and the background area are convolved and normalized to obtain the second image information, including: Performing 2D convolution on the face area and the background area to obtain a second vector Q', a second vector K', and a second vector V', respectively, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the second vector Q' and the second vector K' to obtain the second attention map E'; Normalize the second attention map E'; Multiplying the normalized second result by the second vector V' and rearranging the result into an image of the size of the face area; The normalized result is added to the decoded image to obtain the second image information.

2. The method for liveness detection according to claim 1, before dividing the decoded image into a face area and a background area, further comprising: A cascaded encoder-decoder structure is used to encode and decode the image to be detected.

3. The method for detecting living body according to claim 2, wherein: The encoding process includes one or more stages, and the decoding process includes one or more stages; When the decoding process includes multiple stages, the image obtained by decoding at each stage is processed by the living body detection method and then input into the decoder of the next stage until there is no decoder left.

4. The liveness detection method according to claim 3, further comprising: Performing triplet metric learning on each level of live attack detection information based on the encoding result of the last level and the live attack detection information after the decoded image is processed by the live detection method; The fusion further includes: Performing regression loss processing on the live attack detection information obtained after the fusion; The attack clue map after regression loss processing is superimposed on the image to be detected and binary classification is performed to obtain the final attack clue map.

5. The liveness detection method according to claim 1, further comprising: performing a data expansion operation on the attack data obtained by fusing the first image information and the second image information; The regression loss processing includes: performing regression loss processing according to the result after the expansion operation and the attack clue data to obtain an attack clue graph.

6. A device for implementing liveness detection, comprising a memory and a processor, wherein: The memory stores the following instructions that can be executed by the processor: used to execute the steps of the liveness detection method according to any one of claims 1 to 5.

7. A liveness detection method, comprising: Perform data expansion operations on attack data; Perform regression loss processing based on the results of the expansion operation and the attack clue data to obtain the attack clue graph; The step of performing regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue graph includes: The loss function is used to introduce the supervision information of the attack clue itself for constraint. The loss function of the self-supervised clue regression is as follows: Among them, L r represents the obtained attack clue graph, live represents the live body library, spoof represents the attack library; exp represents the expansion operation, C represents the attack clue of the attack data I, and C' represents the attack clue of the attack data after the data expansion operation.

8. The liveness detection method according to claim 7, before performing the data expansion operation on the attack data, further comprising: A cascaded encoder-decoder structure is used to encode and decode the image to be detected; The encoding process includes one or more stages, and the decoding process includes one or more stages; Triplet metric learning is performed based on the results of the last level encoding and the results of each level decoding.

9. The liveness detection method according to claim 7 or 8, further comprising: The attack clue map after regression loss processing is superimposed on the original image to be detected and binary classification is performed to obtain the final attack clue map.

10. The liveness detection method according to claim 7, before performing the data expansion operation on the attack data, further comprising: A cascaded encoder-decoder structure is used to encode and decode the image to be detected; The encoding process includes one or more stages, and the decoding process includes one or more stages; Dividing the decoded image into a face region and a background region, performing convolution processing and normalization processing on the face region to obtain first image information, and performing convolution processing and normalization processing on the face region and the background region respectively to obtain second image information; When the decoding process includes multiple stages, the image obtained by decoding at each stage is processed by the living body detection method and then input into the decoder of the next stage until there is no decoder left; Performing triplet metric learning on each level of live attack detection information based on the encoding result of the last level and the live attack detection information after the decoded image is processed by the live detection method; The first image information and the second image information are fused.

11. A device for implementing liveness detection, comprising a memory and a processor, wherein: The memory stores the following instructions that can be executed by the processor: used to execute the steps of the liveness detection method according to any one of claims 7 to 10.

12. A liveness detection method, comprising: Dividing the decoded image into a face region and a background region, performing convolution processing and normalization processing on the face region to obtain first image information, and performing convolution processing and normalization processing on the face region and the background region respectively to obtain second image information; performing a data expansion operation on the attack data obtained by fusing the first image information and the second image information; Perform regression loss processing based on the results of the expansion operation and the attack clue data to obtain the attack clue graph; The performing convolution processing and normalization processing on the facial region to obtain first image information includes: Applying 2D convolution to the facial region to obtain a first vector Q, a first vector K, and a first vector V, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the first vector Q and the first vector K to obtain the first attention map E; Normalize the first attention map E; Multiplying the first normalized result by the first vector V and rearranging the result into an image of the size of the face region; Adding the rearranged image of the face region size to the image obtained by squirrel decoding to obtain the first image information; The face area and the background area are convolved and normalized to obtain the second image information, including: Performing 2D convolution on the face area and the background area to obtain a second vector Q', a second vector K', and a second vector V', respectively, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the second vector Q' and the second vector K' to obtain the second attention map E'; Normalize the second attention map E'; Multiplying the normalized second result by the second vector V' and rearranging the result into an image of the size of the face area; Adding the normalized result to the decoded image to obtain the second image information; The step of performing regression loss processing on the result after the expansion operation and the attack clue data to obtain an attack clue graph includes: The loss function is used to introduce the supervision information of the attack clue itself for constraint. The loss function of the self-supervised clue regression is as follows: Among them, L r represents the obtained attack clue graph, live represents the live body library, spoof represents the attack library; exp represents the expansion operation, C represents the attack clue of the attack data I, and C' represents the attack clue of the attack data after the data expansion operation.

13. The method for liveness detection according to claim 12, before dividing the decoded image into a face area and a background area, further comprising: A cascaded encoder-decoder structure is used to encode and decode the image to be detected.

14. The living body detection method according to claim 13, wherein: The encoding process includes one or more stages, and the decoding process includes one or more stages; When the decoding process includes multiple stages, the image obtained by decoding at each stage is processed by the living body detection method and then input into the decoder of the next stage until there is no decoder left.

15. The liveness detection method according to claim 12, before performing the data expansion operation on the attack data obtained by fusing the first image information and the second image information, further comprising: A cascaded encoder-decoder structure is used to encode and decode the image to be detected; The encoding process includes one or more stages, and the decoding process includes one or more stages; Triplet metric learning is performed based on the results of the last level encoding and the results of each level decoding.

16. The liveness detection method according to claim 13, further comprising: The attack clue map after the regression loss processing is superimposed on the image to be detected and binary classification is performed to obtain the final attack clue map.

17. A living body detection device, comprising: Division module, first processing module, second processing module, fusion module; wherein, A segmentation module configured to divide the decoded image into a face region and a background region; a first processing module configured to perform convolution processing and normalization processing on the facial region to obtain first image information; A second processing module is configured to perform convolution processing on the face area and the background area respectively and then perform normalization processing to obtain second image information; a fusion module configured to fuse the first image information and the second image information to obtain live attack detection information; The first processing module is further configured to: Performing a 2D convolution on the facial region to obtain a first vector Q, a first vector K, and a first vector V, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the first vector Q and the first vector K to obtain the first attention map E; Normalize the first attention map E; Multiplying the first normalized result by the first vector V and rearranging the result into an image of the size of the face region; adding the rearranged image of the face region size to the decoded image to obtain the first image information; The second processing module is further configured to: Performing 2D convolution on the face area and the background area to obtain a second vector Q', a second vector K', and a second vector V', respectively, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the second vector Q' and the second vector K' to obtain the second attention map E'; Normalize the second attention map E'; Multiplying the normalized second result by the second vector V' and rearranging the result into an image of the size of the face area; The normalized result is added to the decoded image to obtain the second image information.

18. A living body detection device, comprising: Extension module, regression processing module; among them, An extension module configured to perform data extension operations on attack data; A regression processing module is configured to perform regression loss processing based on the result after the expansion operation and the attack clue data to obtain an attack clue graph; The regression processing module is further configured to: The loss function is used to introduce the supervision information of the attack clue itself for constraint. The loss function of the self-supervised clue regression is as follows: Among them, L r represents the obtained attack clue graph, live represents the live body library, spoof represents the attack library; exp represents the expansion operation, C represents the attack clue of the attack data I, and C' represents the attack clue of the attack data after the data expansion operation.

19. A living body detection device, comprising: Division module, first processing module, second processing module, fusion module, expansion module, regression processing module; wherein, A segmentation module configured to divide the decoded image into a face region and a background region; a first processing module configured to perform convolution processing and normalization processing on the facial region to obtain first image information; A second processing module is configured to perform convolution processing on the face area and the background area respectively and then perform normalization processing to obtain second image information; a fusion module configured to fuse the first image information and the second image information; An expansion module configured to perform a data expansion operation on the attack data obtained by fusing the first image information and the second image information; A regression processing module is configured to perform regression loss processing based on the result after the expansion operation and the attack clue data to obtain an attack clue graph; The first processing module is further configured to: Applying 2D convolution to the facial region to obtain a first vector Q, a first vector K, and a first vector V, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the first vector Q and the first vector K to obtain the first attention map E; Normalize the first attention map E; Multiplying the first normalized result by the first vector V and rearranging the result into an image of the size of the face region; Adding the rearranged image of the face region size to the image obtained by squirrel decoding to obtain the first image information; The second processing module is further configured to: Performing 2D convolution on the face area and the background area to obtain a second vector Q', a second vector K', and a second vector V', respectively, and rearranging them into a one-dimensional feature; Calculate the matrix multiplication based on the second vector Q' and the second vector K' to obtain the second attention map E'; Normalize the second attention map E'; Multiplying the normalized second result by the second vector V' and rearranging the result into an image of the size of the face area; Adding the normalized result to the decoded image to obtain the second image information; The regression processing module is further configured to: The loss function is used to introduce the supervision information of the attack clue itself for constraint. The loss function of the self-supervised clue regression is as follows: Among them, L r represents the obtained attack clue graph, live represents the live body library, spoof represents the attack library; exp represents the expansion operation, C represents the attack clue of the attack data I, and C' represents the attack clue of the attack data after the data expansion operation.

Images