Face living body detection method and detection device

Abstract

The invention provides a human face living body detection method, and relates to the technical field of living body detection. The method comprises the following steps: A, acquiring multi-feature source data of a human face; b, acquiring multi-channel face global feature data according to the multi-feature source data; c, acquiring multi-type face key feature data according to the face global feature data; d, fusing different types of face key feature data; and E, processing the fused data, and obtaining and outputting the probability of true and false faces. The invention further provides a human face living body detection device. According to the invention, when the face living body detection deployment is carried out on the Internet of Things equipment, the detection accuracy of true and false faces is improved.

Description

Technical Field

[0001] This application relates to the field of liveness detection technology, specifically to a method and device for detecting liveness of human faces. Background Technology

[0002] Existing technologies analyze various facial liveness attack images, including photos, 3D dummies, and videos. In 2D attacks, the key facial feature regions (facial edges, glasses, nose, mouth) and skin color in fake face images differ significantly from those in real face images after processing with various image algorithms (LBP (Local Binary Pattern) grayscale / illuminance invariance, Fourier transform, nonlinear diffusion, HOG (Histogram of Oriented Gradient), GIST (Global Features), SIFT (Scale Invariant Feature Transform)). These algorithms reveal significant differences between real and fake faces in structural texture information, reflectance spectrum information, facial surface depth information, and geometric relationships of facial features.

[0003] Based on the differences between real and fake faces, existing technologies have proposed many different methods, including: a. Traditional algorithms based on texture features: These hand-designed algorithms require the face to undergo feature extraction algorithms to obtain raw feature data, such as GIST features, HOG features, SIFT features, color features, etc. A feature is generally only sensitive to changes in some characteristics of the image, but not sensitive to changes in other characteristics. Therefore, these feature data often have small intra-class variance and large inter-class variance. This method is still at risk of being compromised when facing different face carrier materials, different lighting, and 3D model attacks.

[0004] b. Depth perception based on 3D imaging. This method requires 3D imaging, but it has high requirements for computing power and algorithms, and is not suitable for deployment in IoT devices. At the same time, it may still be compromised in the face of 3D model attacks.

[0005] c. CNN (Convolutional Neural Networks) based methods have advantages over other methods in terms of features learned through data-driven approaches. However, they require large-scale fake face image data to train the model parameters. Existing CNN liveness detection models have already outperformed previous ones, but they may still be compromised in the face of malicious attacks. At the same time, due to the limited computing power of IoT devices, the model parameter scale is relatively small, thus the classification performance of the model has been somewhat limited.

[0006] In developing this invention, the inventors discovered that existing face liveness detection models deployed on servers or dedicated edge devices can achieve high accuracy in liveness detection. However, due to limitations in computing power at the IoT device level, small-scale, lightweight liveness detection models can only be deployed. As a result, the liveness detection performance of these models cannot completely withstand various attacks, leading to a poor user experience. Summary of the Invention

[0007] In view of this, the embodiments of this specification provide a face liveness detection method and apparatus, which solves the defect of low accuracy in detecting real and fake faces when deploying face liveness detection on Internet of Things devices in the prior art.

[0008] The embodiments in this specification provide the following technical solutions: A method for face liveness detection is provided, the method comprising the following steps: A. Obtain multi-feature source data of the face; B. Based on the multi-feature source data, obtain multi-channel global facial feature data; C. Based on the global facial feature data, obtain key facial feature data of multiple types; D. Integrate key facial feature data from different types; E. Process the fused data to obtain and output the probability of real and fake faces.

[0009] In some embodiments, step A specifically involves processing the face to be detected using different feature extraction algorithms to obtain multi-feature source data of the face.

[0010] In some embodiments, the multi-feature source data of the face includes one or more of IR images, HOG, LBP, GIST and GaussianBlur feature data based on IR images.

[0011] In some embodiments, step B specifically involves performing a first convolution process on the multi-feature source data to obtain multi-channel global facial feature data.

[0012] In some embodiments, when performing the first convolutional processing on the multi-feature source data, global facial feature data is obtained through one or more convolutional layers.

[0013] In some embodiments, when performing the first convolution processing on the multi-feature source data, a combination of convolution kernels of the same or different sizes is used to acquire global facial feature data.

[0014] In some embodiments, step C specifically involves: performing a second convolution process on the multi-channel global facial feature data to obtain multi-type key facial feature data.

[0015] In some embodiments, the second convolution processing of the multi-channel global facial feature data specifically includes: C1. Based on the type of key facial features, the convolutional kernels used to extract key facial features are divided into multiple groups, and each group of convolutional kernels contains at least one convolutional kernel. C2. Extract the multi-type key facial feature data through convolutional kernel groups, with each convolutional kernel group used to extract the key facial features of the corresponding type.

[0016] In some embodiments, in step C1, when each convolutional kernel group contains multiple convolutional kernels, each group of facial key features is composed of facial key feature maps represented by different features.

[0017] In some embodiments, different sizes of convolution kernels are used for key facial features of different scales.

[0018] In some embodiments, after performing a second convolution processing on the multi-channel global facial feature data, step F is further included: performing size unification processing on the data generated after processing with convolution kernels of different sizes.

[0019] In some embodiments, the size unification process specifically includes: firstly, performing pooling downsampling operations of different sizes, and then performing zero-filling operations on small-sized intermediate feature maps or edge clipping operations on large-sized intermediate feature maps.

[0020] In some embodiments, step D specifically involves performing a third convolution process on the multi-type facial key feature data to fuse different types of facial key feature maps.

[0021] In some embodiments, the process of fusing different types of facial key feature maps specifically includes: performing a convolution operation between each convolution kernel used for calculating the combination of facial key features and facial key feature map data of one of different types of facial key features, and outputting the current combined features.

[0022] In some embodiments, between performing the convolution operation and outputting the current combined feature, step G is further included: setting a transitional combined feature layer and applying multi-channel attention weights to the transitional combined feature layer.

[0023] In some embodiments, step G specifically includes: G1. For the combined features of each channel of the output transition combined feature layer, a global pooling operation is used to obtain a 1×1×h feature vector, where the vector length h is the same as the number of input channels; G2. Perform a full connection operation on the 1×1×h feature vector and reduce the dimension of the vector length by h / r to obtain a 1×1×h / r feature vector. G3. Perform an activation operation on the 1×1×h / r feature vector to obtain an activated feature vector of 1×1×h / r. G4. Perform a second fully connected operation on the 1×1×h / r activation feature vector and restore its vector length to 1×1×h; G5. Perform an activation operation on the feature vector after the length is restored to obtain multi-channel attention weights; G6. Perform multiplication operations on the feature map data of each channel of the transition combined feature layer with the corresponding multi-channel attention weights to obtain the current combined features.

[0024] In some embodiments, step E specifically involves: outputting the probability of real or fake faces after processing through multiple deep convolutional layers and fully connected layers.

[0025] A face liveness detection device is provided, the device comprising a multi-feature source data acquisition module, a global face feature data acquisition module, a key face feature data acquisition module, a key face feature data fusion module, and a classification output module; wherein the global face feature data acquisition module is connected to the multi-feature source data acquisition module and the key face feature data acquisition module, respectively, and the key face feature data fusion module is connected to the key face feature data acquisition module and the classification output module, respectively. The multi-feature source data acquisition module is used to acquire multi-feature source data of a face; the global face feature data acquisition module is used to acquire multi-channel global face feature data based on the multi-feature source data; the key face feature data acquisition module is used to acquire multiple types of key face feature data based on the global face feature data; the key face feature data fusion module is used to fuse key face feature data of different types; and the classification output module is used to process the fused data to acquire and output the probability of real or fake faces.

[0026] In some embodiments, the facial key feature data acquisition module includes a second convolution processing submodule and a size unification processing submodule, which are connected. The second convolution processing submodule is used to perform a second convolution processing on the multi-channel global facial feature data; the size unification processing submodule is used to perform size unification processing on the data generated after processing with convolution kernels of different sizes.

[0027] In some embodiments, the facial key feature data fusion module includes a third convolution processing submodule and a multi-channel attention submodule, which are connected together. The third convolution processing submodule is used to perform third convolution processing on the multi-type facial key feature data; the multi-channel attention submodule is used to set up a transition combination feature layer and apply multi-channel attention weight processing to the transition combination feature layer.

[0028] Compared with the prior art, the beneficial effects that can be achieved by the above-mentioned at least one technical solution adopted in the embodiments of this specification include at least the following: when deploying face liveness detection on IoT devices, this application improves the accuracy of detecting real and fake faces; this application improves the liveness detection performance in the face recognition task process. Attached Figure Description

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 This is a flowchart of a face liveness detection method provided in an embodiment of this application; Figure 2 This is a flowchart of a face liveness detection method provided in an embodiment of this application; Figure 3 This is a schematic diagram of multi-feature source data of a face provided in an embodiment of this application; Figure 4 This is a schematic diagram of a face liveness detection method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the specific architecture of a multi-channel attention submodule provided in an embodiment of this application; Figure 6 This is a schematic diagram of the specific architecture of a channel weighting unit provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a face liveness detection device provided in an embodiment of this application; Detailed Implementation The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0031] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0033] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0034] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0035] This application provides a face liveness detection method, such as... Figure 1 As shown, refer to Figure 1 The method includes the following steps: Step s101: Obtain multi-feature source data of the face. In this embodiment, specifically: process the face to be detected using different feature extraction algorithms to obtain multi-feature source data of the face. The multi-feature source data of the face includes one or more of the following: IR (Infrared) image, HOG, LBP, GIST, and Gaussian Blur feature data based on the IR image.

[0036] Step s102: Obtain multi-channel global facial feature data based on the multi-feature source data. In this embodiment, specifically: perform a first convolution processing on the multi-feature source data to obtain multi-channel global facial feature data. During the first convolution processing of the multi-feature source data, global facial feature data is obtained through one or more convolutional layers using a combination of convolutional kernels of the same or different sizes.

[0037] Step s103: Based on the global facial feature data, obtain multi-type key facial feature data. In this embodiment, specifically: perform a second convolution process on the multi-channel global facial feature data to obtain multi-type key facial feature data.

[0038] Furthermore, the second convolution processing of the multi-channel global facial feature data specifically includes: Step s1031: Based on the type of facial key features, the convolutional kernels used to extract facial key features are divided into multiple groups, with each group containing at least one convolutional kernel. When each group contains multiple convolutional kernels, each group of facial key features consists of facial key feature maps represented by different features.

[0039] Step s1032: Extract the multi-type facial key feature data through convolutional kernel groups, where each convolutional kernel group is used to extract the corresponding type of facial key features.

[0040] Furthermore, different convolutional kernel sizes are used for key facial features of different scales. After performing a second convolutional process on the multi-channel global facial feature data, step s1033 is included: performing size unification processing on the data generated after processing with convolutional kernels of different sizes. The size unification processing specifically includes: firstly, using pooling downsampling operations of different sizes, and then performing zero-padding operation on small-sized intermediate feature maps or edge cropping operation on large-sized intermediate feature maps.

[0041] Step s104 involves fusing different types of facial key feature data. In this embodiment, this specifically involves performing a third convolution process on the multiple types of facial key feature data to fuse the different types of facial key feature maps. Specifically, each convolution kernel used for calculating the combination of facial key features is convolved with one of the different types of facial key feature map data, outputting the current combined feature.

[0042] Furthermore, between performing the convolution operation and outputting the current combined features, a multi-channel attention weight processing step is included. First, a transitional combined feature layer is set up, and then multi-channel attention weight processing is applied to the transitional combined feature layer. This step specifically includes: Step s1041: Apply global pooling to the combined features of each channel of the output transition combined feature layer to obtain a 1×1×h feature vector, where the vector length h is the same as the number of input channels; Step s1042: Perform a fully connected operation on the 1×1×h feature vector and reduce the dimension of the vector length by h / r to obtain a 1×1×h / r feature vector; Step s1043: Perform an activation operation on the 1×1×h / r feature vector to obtain an activated feature vector of 1×1×h / r; Step s1044: Perform a second fully connected operation on the activation feature vector of 1×1×h / r, and restore its vector length to 1×1×h; Step s1045: Perform an activation operation on the feature vector after the length is restored to obtain multi-channel attention weights; Step s1046: Perform multiplication operations on the feature map data of each channel of the transition combined feature layer with the corresponding multi-channel attention weights to obtain the current combined features.

[0043] Step s105 involves processing the fused data to obtain and output the probabilities of real and fake faces. In this embodiment, specifically, the probabilities of real and fake faces are output after processing through multiple depthwise convolutional layers and fully connected layers.

[0044] In some embodiments, the method employs a novel convolutional neural network model architecture, focusing on key facial feature regions to improve the accuracy of detecting real and fake faces. The method flow is as follows: Figure 2 As shown, the following steps are included: Step s201 involves processing the face to be detected using different feature extraction algorithms to obtain multi-feature source data of the face. In this embodiment, one type of multi-feature source data of the face is obtained as follows: Figure 3 As shown, it includes IR images and HOG, LBP, GIST, and GaussianBlur feature data based on IR images. In order to improve the robustness and generalization ability of the liveness detection algorithm, it is necessary to fuse different information.

[0045] This embodiment uses Figure 3 When dealing with multi-feature source data as shown, the specific implementation process of this method is as follows: Figure 4 As shown.

[0046] Step s202: Perform a first convolution process on the multi-feature source data to obtain multi-channel global face feature data. For example... Figure 4As shown, in this embodiment, for the sake of simplicity, a single convolutional layer is used, and conventional convolutional layer structures such as pooling layers, activation layers, normalization layers, quantization layers, and residual structures are not shown. Those skilled in the art should know and understand that the above-mentioned other structural layers can be combined and used according to actual conditions. The subsequent layers will not be described in detail unless there is a special need.

[0047] in addition, Figure 4 In the diagram: W1_i: 3×3×n, W1_i represents the convolution kernel of the i-th channel in the first layer, 3×3 is the size of the convolution kernel, and n represents the depth of each convolution kernel (which needs to correspond to the number of input feature map channels). That is, n corresponds to the input channel n of the multi-feature source data. After convolution operation through i-channel convolution kernels, the multi-feature source data is mapped to an output feature map C1 (C1_1…C1_i) with i channels, where C1_i represents the feature map data of the i-th output channel of the first layer.

[0048] In this embodiment, only one layer of convolution is used to extract global facial feature data. In actual model architecture construction, multiple convolutional layers and combinations of convolutional kernels of different sizes can be used for front-end feature extraction.

[0049] Step s203: Perform a second convolution process on the multi-channel global face feature data to obtain multi-type key face feature data. In this embodiment, after completing the acquisition of the multi-channel global face feature data, the following operations are performed on the multi-channel global face feature C1: Based on the type of facial key features, the convolutional kernels w2 used to extract facial key features are divided into multiple groups. Each group of convolutional kernels w2 is used to extract facial key features C2 of the corresponding type. Each group of convolutional kernels contains at least one convolutional kernel. When each group of convolutional kernels contains multiple convolutional kernels, each group of facial key features C2 is composed of facial key feature maps with different feature representations. The different feature representations refer to the fact that although each feature map in each group of facial key features represents key feature map data of different types of faces, the key feature map data generated is expressed differently due to the different parameters of the convolutional kernels.

[0050] like Figure 4 In the C2_1 feature map, the light sensitivity is expressed, the structure and texture sensitivity is expressed, and the color or skin color sensitivity is expressed. These expressions are very important in face liveness detection and are the main factors in distinguishing between real and fake faces. In terms of light sensitivity, the brightness distribution of real face images is significantly different from that of paper-printed faces. Histograms show that the brightness distribution of real faces is more balanced, while the brightness distribution of printed fake faces is more concentrated.

[0051] Furthermore, different sizes of convolutional kernels are used for key facial features of varying scales. For large-scale features, large-size convolutional kernels are used to increase the receptive field, allowing the kernel to focus more on large structural features, such as facial contours, where a 7×7 kernel can be used. Conversely, for small-scale structures like eyes and wrinkles, 3×3 or 1×1 kernels can be used. In convolutional neural networks, the receptive field refers to the region of the input image that a point on the feature map can see; that is, the point on the feature map is calculated from the receptive field size of the input image. The larger the receptive field value of a neuron, the larger the range of the original image it can access; the smaller the value, the more local and detailed the features it contains.

[0052] like Figure 4 The middle convolutional kernel W2 consists of 12 convolutional kernels, divided into 4 groups, each containing 3 convolutional kernels; each group of convolutional kernels is used to extract the corresponding key facial feature data (e.g., W2_1 / W2_2 / W2_3 are used to extract facial contour features, W2_4 / W2_5 / W2_6 are used to extract nose region features, W2_7 / W2_8 / W2_9 are used to extract eye region features, and W2_10 / W2_11 / W2_12 are used to extract mouth region features).

[0053] Furthermore, this embodiment generates corresponding facial key features C2 based on the multi-channel global face features C1 processed by each group of convolutional kernels W2. Before generating the facial key features C2, the intermediate feature data C2' (not shown in the figure) generated by the convolutional kernels W2 is also resized. Since the convolutional kernel group W2 uses convolutional kernels of different sizes based on different face feature scales, the size of the intermediate feature data C2' generated after convolutional kernel processing will be inconsistent, so size unification processing is required. The size unification processing method includes: (1) Use pooling downsampling operations of different sizes.

[0054] (2) Perform zero-filling operation on small-sized intermediate feature maps or edge cropping operation on large-sized intermediate feature maps (generally, small-sized intermediate feature maps use large-sized convolution kernels to obtain facial contour features, so edge filling still preserves the original contour feature information; while large-sized intermediate feature maps use small-sized convolution kernels to obtain small-structure facial key features, these small-structure facial features are mainly distributed in the internal area of ​​the face, so by performing edge cropping operation on large-sized feature maps, the small-scale structure facial key features will not be destroyed).

[0055] Step s204 involves performing a third convolution on the multi-type facial key feature data to fuse the different types of facial key feature maps. In this embodiment, after obtaining different types of facial key features C2, a combined convolution operation is used to fuse the different types of facial key feature maps. The specific implementation process is as follows: Each convolution kernel used for calculating the combination of facial key features performs a convolution operation with one of the facial key feature maps C2 from different groups or types of facial key features, outputting the current combined feature C3; where the depth of each convolution kernel W3 corresponds to the number of facial key feature types, that is, it represents performing a convolution operation with different pattern feature map data in different types / groups of facial feature data. Figure 2 The facial feature types are 4 or 4 groups, so the depth of each convolutional kernel W3 is 4. In the example, the kernel size is 3×3. In practical applications, the kernel size can be adjusted according to the specific situation. like Figure 4 In the diagram, convolution kernel W3_1 (3×3×4) is convolved with C2_1 / C2_4 / C2_7 / C2_10 to generate a combined feature C3_1; convolution kernel W3_2 (3×3×4) is convolved with C2_2 / C2_5 / C2_8 / C2_11 to generate a combined feature C3_2; convolution kernel W3_3 (3×3×4) is convolved with C2_3 / C2_6 / C2_9 / C2_12 to generate a combined feature C3_3; convolution kernel W4_2 (3×3×4) is convolved with C2_1 / C2_4 / C2_7 / C2_10 to generate a combined feature C3_1. C2_11 performs a convolution operation to generate a set of combined features C3_4; and so on. Based on the principle of permutation and combination, the number of convolution kernels W3 and the input feature data C2 linked to each convolution kernel W3 can be determined. In the example figure, there are four types of facial key features C1_1 / C1_2 / C1_3, C1_4 / C1_5 / C1_6, C1_7 / C1_8 / C1_9, C1_10 / C11 / C1_12. Each group has three feature data, so C3 is needed. 1 ×C3 1 ×C3 1 ×C3 1 =81 convolutional kernels can extract 81 combined features. Each combined feature represents the fusion of feature map information of different modes of key facial feature maps of different types.

[0056] Step s205: After processing through multiple deep convolutional layers and fully connected layers, the probability of real or fake faces is output. In this embodiment, there is no significant difference from the traditional classification output layer; the probability of real or fake faces is output after processing through multiple deep convolutional layers and fully connected layers.

[0057] Furthermore, in step s204, before generating the facial key combination feature C3, a multi-channel attention weight processing step is included. First, a transitional combination feature layer is set, and then multi-channel attention weight processing is applied to the transitional combination feature layer. This step is implemented by the multi-channel attention submodule, i.e., in... Figure 4 A multi-channel attention submodule has been added to the existing architecture. A schematic diagram of the specific architecture of the multi-channel attention submodule is shown below. Figure 5 As shown, the multi-channel attention submodule includes a channel weighting unit, and the specific architecture of the channel weighting unit is illustrated in the diagram below. Figure 6 As shown.

[0058] Reference Figure 5 and Figure 6 Different types of facial key features, and facial features under different patterns, after feature combination, have varying degrees of influence on the final output probability of the model for each output channel. That is, some combined features have a significant impact on the final output result. Therefore, neural networks should focus on these combined features of channels. Figure 4 A multi-channel attention submodule was added to the basic structure. By constructing channel weights, the neural network can adaptively focus on combined feature channels with larger weights after training. Based on this, an intermediate combined feature layer F3 is set before outputting the combined feature C3. After processing the F3 layer with multi-channel attention weights S3, the final combined feature C3 is output. This process specifically includes: a. For the combined features of each channel of the output F3 layer, global pooling is performed to obtain a 1×1×h feature vector, where the vector length h is the same as the number of input channels (h=81 in this embodiment).

[0059] b. Perform a fully connected operation on the feature vector and reduce the dimension of the vector by h / r to obtain a 1×1×h / r feature vector.

[0060] c. Perform activation operation on the 1×1×h / r feature vector (RELU is used in this embodiment, but it can be adjusted according to the actual situation) to obtain the 1×1×h / r activated feature vector.

[0061] d. Perform a second fully connected operation on the above activated feature vectors and restore their vector length to 1×1×h.

[0062] e. Perform activation operations on the feature vector after restoring the length (Sigmoid is used in this embodiment, but it can be adjusted according to the actual situation) to obtain the multi-channel attention weight S3: 1×1×h.

[0063] f. Perform multiplication operations on the feature map data of each channel of the transition combined feature F3 layer with the corresponding multi-channel attention weight S3 to obtain the current combined feature C3.

[0064] This application provides a face liveness detection device, such as... Figure 7 As shown, the device includes a multi-feature source data acquisition module 71, a global face feature data acquisition module 72, a key face feature data acquisition module 73, a key face feature data fusion module 74, and a classification output module 75; wherein the global face feature data acquisition module 72 is connected to the multi-feature source data acquisition module 71 and the key face feature data acquisition module 73, and the key face feature data fusion module 74 is connected to the key face feature data acquisition module 73 and the classification output module 75.

[0065] The multi-feature source data acquisition module 71 is used to acquire multi-feature source data of the face; the global face feature data acquisition module 72 is used to acquire multi-channel global face feature data based on the multi-feature source data; the key face feature data acquisition module 73 is used to acquire multiple types of key face feature data based on the global face feature data; the key face feature data fusion module 74 is used to fuse different types of key face feature data; and the classification output module 75 is used to process the fused data, acquire and output the probability of real and fake faces.

[0066] Furthermore, the facial key feature data acquisition module 73 includes a second convolution processing submodule 731 and a size unification processing submodule 732, which are connected.

[0067] The second convolution processing submodule 731 is used to perform a second convolution processing on the multi-channel global face feature data; the size unification processing submodule 732 is used to perform size unification processing on the data generated after processing with convolution kernels of different sizes.

[0068] Furthermore, the facial key feature data fusion module 74 includes a third convolution processing submodule 741 and a multi-channel attention submodule 742, which are connected. The third convolution processing submodule 741 is used to perform third convolution processing on the multi-type facial key feature data; the multi-channel attention submodule 742 is used to set a transition combination feature layer and apply multi-channel attention weight processing to the transition combination feature layer.

[0069] This application improves the accuracy of detecting real and fake faces when deploying face liveness detection on IoT devices; this application also improves the liveness detection performance during face recognition tasks. In this specification, similar or identical parts between the various embodiments can be referred to interchangeably, and each embodiment focuses on describing the differences from other embodiments. Furthermore, this specification uses specific terms to describe the embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0070] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented by hardware devices, they can also be implemented solely by software solutions, such as installing the described system on existing processing devices or mobile devices.

[0071] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

Claims

1. A method for detecting human face liveness, characterized in that, The method includes the following steps: A. Obtain multi-feature source data of the face; B. Based on the multi-feature source data, obtain multi-channel global facial feature data; C. Based on the global facial feature data, obtain key facial feature data of multiple types; D. Integrate key facial feature data from different types; E. Process the fused data to obtain and output the probability of fusing different types of facial key feature data to produce real and fake faces.

2. The face liveness detection method according to claim 1, characterized in that, Step A specifically involves processing the face to be detected using different feature extraction algorithms to obtain multi-feature source data of the face.

3. The face liveness detection method according to claim 1, characterized in that, The multi-feature source data of the face includes one or more of the following: IR image, HOG, LBP, GIST and GaussianBlur feature data based on the IR image.

4. The face liveness detection method according to claim 1, characterized in that, Step B specifically involves performing a first convolution process on the multi-feature source data to obtain multi-channel global facial feature data.

5. The face liveness detection method according to claim 4, characterized in that, When performing the first convolutional processing on the multi-feature source data, global facial feature data is obtained through one or more convolutional layers.

6. The face liveness detection method according to claim 5, characterized in that, When performing the first convolution processing on the multi-feature source data, the global facial feature data is obtained by using a combination of convolution kernels of the same or different sizes.

7. The face liveness detection method according to claim 1, characterized in that, Step C specifically involves performing a second convolution process on the multi-channel global facial feature data to obtain multi-type key facial feature data.

8. The face liveness detection method according to claim 7, characterized in that, The second convolution processing of the multi-channel global facial feature data specifically includes: C1. Based on the type of key facial features, the convolutional kernels used to extract key facial features are divided into multiple groups, and each group of convolutional kernels contains at least one convolutional kernel. C2. Extract the multi-type key facial feature data through convolutional kernel groups, with each convolutional kernel group used to extract the key facial features of the corresponding type.

9. The face liveness detection method according to claim 8, characterized in that, In step C1, when each convolutional kernel group contains multiple convolutional kernels, each group of facial key features is composed of facial key feature maps represented by different features.

10. The face liveness detection method according to claim 8, characterized in that, Different sizes of convolution kernels are used for key facial features of different sizes.

11. The face liveness detection method according to claim 10, characterized in that, After performing a second convolution on the multi-channel global facial feature data, step F is also included: performing size unification processing on the data generated after processing with convolution kernels of different sizes.

12. The face liveness detection method according to claim 11, characterized in that, The size unification process specifically includes: firstly, using pooling downsampling operations of different sizes, and then performing zero-filling operations on small-sized intermediate feature maps or edge clipping operations on large-sized intermediate feature maps.

13. The face liveness detection method according to claim 1, characterized in that, Step D specifically involves performing a third convolution process on the multi-type facial key feature data to fuse the different types of facial key feature maps.

14. The face liveness detection method according to claim 13, characterized in that, The process of fusing different types of facial key feature maps specifically includes: performing convolution operations on each convolution kernel used for facial key feature combination calculation with facial key feature map data of one of different types of facial key features, and outputting the current combined feature.

15. The face liveness detection method according to claim 14, characterized in that, Between performing the convolution operation and outputting the current combined feature, step G is also included: setting a transitional combined feature layer and applying multi-channel attention weights to the transitional combined feature layer.

16. The face liveness detection method according to claim 15, characterized in that, Step G specifically includes: G1. For the combined features of each channel of the output transition combined feature layer, a global pooling operation is used to obtain a 1×1×h feature vector, where the vector length h is the same as the number of input channels; G2. Perform a full connection operation on the 1×1×h feature vector and reduce the dimension of the vector length by h / r to obtain a 1×1×h / r feature vector. G3. Perform an activation operation on the 1×1×h / r feature vector to obtain an activated feature vector of 1×1×h / r. G4. Perform a second fully connected operation on the 1×1×h / r activation feature vector and restore its vector length to 1×1×h; G5. Perform an activation operation on the feature vector after the length is restored to obtain multi-channel attention weights; G6. Perform multiplication operations on the feature map data of each channel of the transition combined feature layer with the corresponding multi-channel attention weights to obtain the current combined features.

17. The face liveness detection method according to any one of claims 1 to 16, characterized in that, Step E specifically involves: outputting the probability of real or fake faces after processing through multiple deep convolutional layers and fully connected layers.

18. A face liveness detection device, characterized in that, The device includes a multi-feature source data acquisition module, a global face feature data acquisition module, a key face feature data acquisition module, a key face feature data fusion module, and a classification output module; wherein the global face feature data acquisition module is connected to the multi-feature source data acquisition module and the key face feature data acquisition module, respectively, and the key face feature data fusion module is connected to the key face feature data acquisition module and the classification output module, respectively. The multi-feature source data acquisition module is used to acquire multi-feature source data of a face; the global face feature data acquisition module is used to acquire multi-channel global face feature data based on the multi-feature source data; the key face feature data acquisition module is used to acquire multiple types of key face feature data based on the global face feature data; the key face feature data fusion module is used to fuse key face feature data of different types; and the classification output module is used to process the fused data to acquire and output the probability of real or fake faces.

19. The face liveness detection device according to claim 18, characterized in that, The facial key feature data acquisition module includes a second convolution processing submodule and a size unification processing submodule, which are connected together. The second convolution processing submodule is used to perform a second convolution processing on the multi-channel global facial feature data; the size unification processing submodule is used to perform size unification processing on the data generated after processing with convolution kernels of different sizes.

20. The face liveness detection device according to claim 18 or 19, characterized in that, The facial key feature data fusion module includes a third convolution processing submodule and a multi-channel attention submodule, which are connected. The third convolution processing submodule is used to perform third convolution processing on the multi-type facial key feature data; the multi-channel attention submodule is used to set up a transition combination feature layer and apply multi-channel attention weight processing to the transition combination feature layer.

Images