Face living body detection method, device and equipment based on artificial intelligence, and medium

Abstract

The application provides a face living body detection method and device based on artificial intelligence, an electronic device and a storage medium. The face living body detection method based on artificial intelligence comprises the following steps: adding multiple filter colors to a collected face image to obtain a face test image set; performing feature extraction on images in the face test image set to obtain face test features; constructing a living body detection loss function based on the face test image set and the face test features; training a neural network based on the face test image set and the living body detection loss function to obtain a face living body detection model; and detecting a face image to be detected based on the face living body detection model to obtain a face living body detection result. The application trains the neural network in all directions by constructing multiple loss functions, so that the defense capability of the obtained face living body detection model against filter color attacks is improved, and the face fraud risk is reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device and storage medium for human face liveness detection based on artificial intelligence. Background Technology

[0002] Facial recognition fraud prevention methods, also known as liveness detection, primarily determine whether a face is alive. Common liveness detection methods include action-based liveness detection and silent liveness detection. Liveness verification technology is a crucial prerequisite for the implementation of unsupervised applications, especially in scenarios such as identity verification and payment transactions. If this technology is cracked or can be attacked at a low cost, it will pose serious security risks and harm users' interests. Therefore, the reliability of liveness verification technology is of paramount importance.

[0003] However, with the widespread application of facial recognition technology, the methods of deceiving and attacking facial recognition systems have also increased rapidly. Existing liveness detection models are often deep neural network models based on common feature extractors. Although these models have achieved high detection accuracy on different datasets, they are easily misled by images with different colored filters, giving criminals an opportunity to exploit vulnerabilities and posing a significant security risk. Summary of the Invention

[0004] In view of the above, it is necessary to propose an artificial intelligence-based face liveness detection method, device, electronic device and storage medium to solve the technical problem of how to improve the defense against filter color attacks, thereby reducing the risk of face fraud.

[0005] This application provides an artificial intelligence-based face liveness detection method, the method comprising:

[0006] Multiple filter colors were added to the collected face images to obtain a set of face test images;

[0007] Feature extraction is performed on the images in the face test image set to obtain face test features;

[0008] A liveness detection loss function is constructed based on the face test image set and the face test features;

[0009] A face liveness detection model is obtained by training a neural network based on the aforementioned face test image set and liveness detection loss function;

[0010] The face liveness detection model is used to detect the face image to be detected and obtain the face liveness detection result.

[0011] In some embodiments, adding multiple filter colors to the acquired face images to obtain a face test image set includes:

[0012] Different facial targets are acquired, and multiple facial images are taken for each facial target to obtain a facial target image set;

[0013] A first filter image set and a second filter image set are obtained by adding multiple different filter colors to the images in the face target image set. Each image in the first filter image set has the same face target and different filter colors, and each image in the second filter image set has the same filter color and different face targets.

[0014] The first filter image set and the second filter image set are used as the face test image set.

[0015] In some embodiments, extracting features from the images in the face test image set to obtain face test features includes:

[0016] Based on a preset feature extractor, multi-layer convolution operations are performed on the images in the face test image set to obtain shallow features at different levels of the images;

[0017] Based on a preset color encoder, shallow features at different levels of the image are fused to obtain deep color fusion features;

[0018] Perform a convolution operation on the deep color fusion features to obtain deep color convolution features;

[0019] After pooling the deep color convolutional features, the deep color features are output through a fully connected layer.

[0020] The shallow features and deep color features of different levels of the image are used as the face test features of the image.

[0021] In some embodiments, the step of superimposing the shallow color features to obtain deep color features includes:

[0022] Deep color fusion features are obtained by fusing shallow color features at different levels;

[0023] Perform a convolution operation on the deep color fusion features to obtain deep color convolution features;

[0024] After pooling the deep color convolutional features, the deep color features are output through a fully connected layer.

[0025] In some embodiments, the liveness detection loss function includes a classification loss function, a color loss function, and a content loss function, wherein the classification loss function satisfies the following relationship:

[0026]

[0027] Among them, L classify x1 i Let x1 represent the classification loss corresponding to the i-th image in the first filter image set, and y1 represent the label image set corresponding to x1. i For the i-th image x1 in the first filter image set i The labeled image, where N1 represents the number of images in the first filter image set, N c CE(x1) represents the number of filter colors included in the first filter image set. i y1 i ) represents the cross-entropy between the i-th image in the first filter image set and its corresponding labeled image, and color k G(x1) represents the color of the k-th filter. i color k ) represents image x1 i Having the k-th filter color, CE(G(x1) i color k ), y1 i () represents the image x1 with the k-th filter color. i With the corresponding label image y1 i Cross-entropy between them.

[0028] In some embodiments, the color loss function satisfies the following relationship:

[0029]

[0030] Among them, L color x2 i Let x1 represent the color loss corresponding to the i-th image in the second filter image set, x2 represent the second filter image set, and γ represent the color loss. k，l =e -dis(k，l) sim(i,j,k,l) ​​is used to measure the similarity between the k-th filter color and the l-th filter color. sim(i,j,k,l) ​​represents the similarity between image i with the k-th filter color and image j with the l-th filter color. N² is the total number of images in the second filter image set, α is a hyperparameter, and N... c This represents the total number of filter colors included in the first filter image set, where sim(i, j, k, l) satisfies the following relationship:

[0031]

[0032] Among them, z colorTo represent deep color features, when k = l, sim(i, j, k, l) is used to represent the similarity between image i and image j belonging to the same second filter image set. When k ≠ l, sim(i, j, k, l) is used to represent the similarity between image i and image j belonging to different second filter image sets.

[0033] In some embodiments, the content loss function satisfies the following relationship:

[0034]

[0035] Among them, L content x i x represents the content loss corresponding to the i-th shallow feature image extracted from the face test image set. i f represents the i-th shallow feature image arbitrarily selected from the face test image set. conten t(x i f represents the face test feature corresponding to the i-th shallow feature image extracted from the face test image set. content (G(x i color k )) represents the face test feature corresponding to the i-th shallow feature image after adding the k-th filter color from the face test image set, and N represents the total number of images in the first filter image set and the second filter image set. c This represents the total number of filter colors included in the first filter image set.

[0036] This application also provides an artificial intelligence-based face liveness detection device, which includes a data acquisition module, an extraction module, a construction module, a training module, and a detection module.

[0037] The acquisition module is used to add various filter colors to the acquired face images to obtain a set of face test images;

[0038] The extraction module is used to extract features from the images in the face test image set to obtain face test features;

[0039] The construction module is used to construct a liveness detection loss function based on the face test image set and the face test features;

[0040] The training module is used to train a neural network based on the face test image set and the liveness detection loss function to obtain a face liveness detection model;

[0041] The detection module is used to detect the face image to be detected based on the face liveness detection model to obtain the face liveness detection result.

[0042] This application embodiment also provides an electronic device, the electronic device comprising:

[0043] Memory, storing at least one instruction;

[0044] The processor executes the instructions stored in the memory to implement the artificial intelligence-based face liveness detection method.

[0045] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the artificial intelligence-based face liveness detection method.

[0046] This application constructs multiple loss functions by applying different filter colors to the acquired face images, thereby training the neural network in all aspects and improving the ability of the obtained face liveness detection model to defend against filter color attacks, effectively reducing the risk of face fraud. Attached Figure Description

[0047] Figure 1 This is a flowchart of a preferred embodiment of the artificial intelligence-based face liveness detection method involved in this application.

[0048] Figure 2 This is a functional block diagram of a preferred embodiment of the AI-based face liveness detection device involved in this application.

[0049] Figure 3 This is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the artificial intelligence-based face liveness detection method involved in this application.

[0050] Figure 4 This is a schematic diagram of the feature extractor and color encoder involved in this application.

[0051] Figure 5 This is a schematic diagram of the neural network involved in this application. Detailed Implementation

[0052] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0053] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0055] This application provides an artificial intelligence-based face liveness detection method, which can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0056] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0057] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0058] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0059] like Figure 1The diagram shown is a flowchart of a preferred embodiment of the AI-based face liveness detection method of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0060] S10: Add various filter colors to the collected face images to obtain a face test image set.

[0061] In an optional embodiment, the step of adding multiple filter colors to the acquired face images to obtain a face test image set includes:

[0062] Different facial targets are acquired, and multiple facial images are taken for each facial target to obtain a facial target image set;

[0063] A first filter image set and a second filter image set are obtained by adding multiple different filter colors to the images in the face target image set. Each image in the first filter image set has the same face target and different filter colors, and each image in the second filter image set has the same filter color and different face targets.

[0064] The first filter image set and the second filter image set are used as the face test image set.

[0065] In this optional embodiment, multiple facial images can be captured by an RGB camera for different facial targets, such as capturing 200 images for each facial target. In this scheme, all the captured facial images are used as a facial target image set. The facial targets can be real human faces, or objects that simulate real human faces, such as face masks and face molds.

[0066] In this optional embodiment, to obtain more accurate liveness detection results and reduce detection errors caused by image color, multiple color filters can be added to the acquired face image using image processing tools such as Photoshop. Since the image consists of RGB color channels, and the value range of each color channel is [0~255], for example, each color channel can select a value from 85 and 170 as its corresponding color value. Because each of the R, G, and B color channels can be chosen from 85 and 170, a total of 2×2×2=8 color combinations are generated. Each color combination serves as a filter color, resulting in a total of 8 filter colors. In other embodiments, each color channel can also select one value from at least two other values ​​as its corresponding color value, thereby forming at least another 8 filter colors.

[0067] In this optional embodiment, the above-mentioned eight color filters can be added to the images in the face target image set. Since multiple identical face images are captured for each face target, eight color filters can be added to face images with the same face target. Each image corresponds to one filter color, and these images with the same face target but different filter colors are used as the first filter image set. That is, all images in the first filter image set are the same face target, and each face target has a first filter image set. At the same time, since the face image corresponding to each face target includes eight color filters, there will be face images with the same filter color but different face targets. In this solution, these images with the same filter color and different face targets are used as the second filter image set, and each filter color has a second filter image set. The multiple first filter image sets and the multiple second filter image sets are used together as the face test image set.

[0068] For example, consider five facial targets, A, B, C, D, and E. 200 images of each facial target were captured, totaling 1000 images. Each set of 200 images for one facial target has eight color filters applied, with filter colors 1, 2, 3, 4, 5, 6, 7, and 8. That is, each set of 200 images for one facial target includes 25 images with filter colors 1, 2, 3, 4, 5, 6, 7, and 8. Specifically, eight images with different filter colors are selected from the 200 images corresponding to any facial target, such as A, to form the first filter image set. Five images from A, B, C, D, and E that share the same filter color, such as 2, are selected to form the second filter image set. Therefore, the images in the first filter image set have the same facial target but different filter colors, while the images in the second filter image set have the same filter color but different facial targets.

[0069] In this way, by adding different filters to the collected face images, face images with multiple filter colors can be obtained, thus providing data support with multiple color dimensions for subsequent processes.

[0070] S11, perform feature extraction on the images in the face test image set to obtain face test features.

[0071] In an optional embodiment, the step of extracting features from the images in the face test image set to obtain face test features includes:

[0072] Based on a preset feature extractor, multi-layer convolution operations are performed on the images in the face test image set to obtain shallow features at different levels of the images;

[0073] Based on a preset color encoder, shallow features at different levels of the image are fused to obtain deep color fusion features;

[0074] Perform a convolution operation on the deep color fusion features to obtain deep color convolution features;

[0075] After pooling the deep color convolutional features, the deep color features are output through a fully connected layer.

[0076] The shallow features and deep color features of different levels of the image are used as the face test features of the image.

[0077] In this optional embodiment, such as Figure 4 The diagram shows the structure of the convolutional neural network containing the feature extractor and the color encoder. In a preferred embodiment, the feature extractor consists of at least four convolutional layers with a kernel size of 3×3, and the number of output channels of each convolutional layer is 16, 24, 40 and 160, respectively. The color encoder consists of at least three convolutional layers with a kernel size of 3×3, a pooling layer and a fully connected layer.

[0078] In this optional embodiment, all images in the face test image set can first be converted into images with a uniform size, such as adjusting the resolution of the images in the face test image set to obtain face images with a resolution of 224×224, where the number of channels in each face image is 3. Therefore, after adjusting the resolution of each image in the face test image set, an image with a resolution of 1×3×224×224 can be obtained.

[0079] In this optional embodiment, such as Figure 4 As shown, the feature image with a resolution of 1×3×224×224 is first input into the first convolutional layer with 16 channels in the feature extractor for convolution processing, thereby outputting a 1×3×112×112 image. This convolution process adjusts the resolution and number of channels of the input image. Next, convolution processing is performed in the second layer with 24 channels and the third layer with 40 channels, outputting images with resolutions of 1×24×56×56 and 1×40×28×28 respectively. Finally, a fourth convolutional layer with 160 channels yields an image with a resolution of 1×160×7×7. In this scheme, the images output from each convolutional layer are used as the shallow features of the corresponding layer of the image.

[0080] In this optional embodiment, such as Figure 4As shown, the shallow feature image with a resolution of 1×24×56×56 output from the second convolutional layer of the feature extractor can be simultaneously input into the first convolutional layer of the color encoder with 40 channels for convolution processing, thereby extracting the shallow color features of the image. Since the number of channels in the shallow color feature image output by the first layer of the color encoder and the number of channels in the 1×40×28×28 image output by the third convolutional layer of the feature extractor are both 40, the shallow color feature can be fused with the 1×40×28×28 shallow feature image output by the third convolutional layer. Then, the fused feature image is further convolved by the second convolutional layer with 160 channels in the color encoder to obtain the second-level shallow color feature of the image. The second-level shallow color feature image with 160 channels output by the color encoder is then fused with the 1×160×7×7 shallow feature image output by the fourth convolutional layer of the feature extractor to obtain the deep color fusion feature of the image. The obtained deep color fusion feature image is then input into the third convolutional layer with 256 channels of the color encoder for convolution operation, thereby extracting the deep color convolutional feature with a resolution of 1×256×4×4.

[0081] In this optional embodiment, such as Figure 4 As shown, the obtained deep color convolutional feature image can be pooled using an average pooling layer, thereby reducing the deep color convolutional feature image with a resolution of 1×256×4×4 (i.e., a feature dimension of 1×256×4×4) to a feature dimension of 1×256×1×1. This reduces the computational load in subsequent processes and also reduces the influence of noise in the feature image. Finally, the feature image with a feature dimension of 1×256×1×1 is processed by fully connected layers (fc) to reduce the number of channels, and the final deep color feature is output.

[0082] In this optional embodiment, since the obtained deep color feature image undergoes progressive feature extraction and feature fusion of the input face image through multiple convolutional layers in the feature extractor and color encoder, the deep color feature image contains the real deep color features of the corresponding face image, which facilitates more accurate training of the neural network based on the deep color features in subsequent processes.

[0083] It is worth noting that, since all the first filter image sets and second filter image sets obtained in step S10 are used as face test image sets in this scheme, the deep color features of each image in the face test image set can be obtained in the end.

[0084] In this way, by constructing multiple convolutional layers to extract deep color features from face images, the neural network in subsequent processes can be trained more accurately and effectively, thereby improving the training effect.

[0085] S12, Construct a liveness detection loss function based on the face test image set and the face test features.

[0086] In an optional embodiment, for images in the first filter image set, although they have different filter colors, they have the same facial target. Therefore, the output images after liveness detection should have the same result. Thus, in order to prevent the liveness detection process from being affected by filter color, a classification loss function L can be constructed. classify The classification loss function satisfies the following relationship:

[0087]

[0088] Among them, L classify x1 i Let x1 represent the classification loss corresponding to the i-th image in the first filter image set, and y1 represent the label image set corresponding to x1. i For the i-th image x1 in the first filter image set i The labeled image, where N1 represents the number of images in the first filter image set, N c CE(x1) represents the number of filter colors included in the first filter image set. i y1 i ) represents the cross-entropy between the i-th image in the first filter image set and its corresponding labeled image, and color k G(x1) represents the color of the k-th filter. i color k ) represents image x1 i Having the k-th filter color, CE(G(x1) i color k ), y1 i () represents the image x1 with the k-th filter color. i With the corresponding label image y1 i Cross-entropy between them.

[0089] In an optional embodiment, for images in the second filter image set, although they have different facial targets, they share the same filter color. Therefore, the color features of each image should be as similar as possible, thereby enabling the construction of a color loss function L. color The color loss function satisfies the following relationship:

[0090]

[0091] Among them, L color x2 i Let x1 represent the color loss corresponding to the i-th image in the second filter image set, x2 represent the second filter image set, and γ represent the color loss. k，l =e -dis(k，l) This is used to measure the similarity between the k-th filter color and the l-th filter color. sim(i, j, k, l) represents the similarity between image i with the k-th filter color and image j with the l-th filter color, where k and l can be the same. N² is the total number of images in the second filter image set, and α is a hyperparameter, which is taken as 1 in this scheme. c This represents the total number of filter colors included in the first filter image set. Since each second filter image set corresponds to only one filter color, this solution adds N to the above formula. c The average loss of a single image under all filter colors is calculated as the color loss to obtain a more accurate color loss, where sim(i,j,k,l) ​​satisfies the following relationship:

[0092]

[0093] Among them, z color The deep color features are represented by sim(i,j,k,l) ​​when k=l, which represents the similarity between image i and image j belonging to the same second filter image set. When k≠l, sim(i,j,k,l) ​​represents the similarity between image i and image j belonging to different second filter image sets. Thus, the color loss function can control the similarity of color features according to the similarity between filter colors.

[0094] In an optional embodiment, for the images in the face test image set, the image with added color filters should retain the same content information as the original image. Therefore, the colors in the extracted shallow features can be separated using the split() function in OpenCV, ensuring that the content loss between the color-separated shallow feature image and the original shallow feature image is minimized. Thus, a content loss function L can be constructed. content The content loss function satisfies the following relationship:

[0095]

[0096] Among them, L content x i This represents the content loss corresponding to the i-th shallow feature image extracted from the face test image set. Since the face test image set consists of the first filter image set and the second filter image set, therefore xi f represents the i-th shallow feature image arbitrarily selected from the first filter image set and the second filter image set. content (x i f represents the face test feature corresponding to the i-th shallow feature image extracted from the face test image set. content (G(x i color k )) represents the face test feature corresponding to the i-th shallow feature image after adding the k-th filter color from the face test image set, and N represents the total number of images in the first filter image set and the second filter image set. c This represents the total number of filter colors included in the first filter image set.

[0097] In this optional embodiment, the liveness detection loss function L can be constructed by linearly summing the classification loss function, color loss function, and content loss function. total， The liveness detection loss function satisfies the following relationship:

[0098] L total =L classify +α1L content +α2L color

[0099] α1 and α2 are weight hyperparameters, which are optional. α1 can be 5 and α2 can be 10.

[0100] In this optional embodiment, the liveness detection loss function can be used to train the neural network in a comprehensive manner in subsequent processes, thereby obtaining a well-trained face liveness detection model and reducing the risk of face fraud.

[0101] In this way, by training the neural network in a comprehensive manner in the subsequent process, the trained face liveness detection model is greatly improved, thereby reducing the risk of face fraud.

[0102] S13, a face liveness detection model is obtained by training a neural network based on the face test image set and the liveness detection loss function.

[0103] In an optional embodiment, the neural network structure to be trained is as follows: Figure 5 As shown, the neural network structure mainly consists of a feature extractor and a color encoder based on a convolutional neural network, and the neural network is trained by the constructed classification loss function, color loss function and content loss function.

[0104] In this optional embodiment, the images in the face test image set are first input into the neural network. A feature extractor performs a convolution operation on the input images to extract features, thereby obtaining shallow features at different levels. These shallow features are processed by a color encoder to obtain deep color features, and the color loss of the images in the second filter image set is calculated using these deep color features. Simultaneously, color separation is performed on the shallow features to obtain color-separated shallow features, and the content loss between the color-separated shallow features in the first and second filter image sets and the unseparated shallow features is calculated.

[0105] In this optional embodiment, images in the first filter image set can be manually labeled with different tags based on their filter colors. These tags can be letters, numbers, symbols, etc., and no specific limitations are imposed in this solution. Preferably, labels can be assigned to images of different filter colors in ascending order of natural numbers to obtain label images. The classification loss between the output image and the corresponding label images in the first filter image set can then be calculated using a classification loss function.

[0106] In this optional embodiment, a large number of face target images can be pre-collected to ensure that the face test image set can effectively train the neural network. By adjusting the parameters and activation functions of the neural network structure during training, the liveness detection loss can be continuously reduced. Finally, training stops when the liveness detection loss function decreases to a preset threshold or becomes 0, and the neural network trained at this point is used as the face liveness detection model. During training, the calculation of the classification loss function, color loss function, and content loss function are performed simultaneously. This is because the liveness detection loss function is composed of the classification loss function, color loss function, and content loss function; adjusting the liveness detection loss calculated by the liveness detection loss function is equivalent to simultaneously adjusting the classification loss, color loss, and content loss.

[0107] In this optional embodiment, since there are multiple sets of first filter images and multiple sets of second filter images, each time an image is randomly selected from the face test image set and input into the neural network during training, it means selecting an image from all the first filter image sets and all the second filter image sets and inputting it into the neural network. Therefore, the images in each set of first filter images and each set of second filter images can be assigned a classification code in advance by human intervention. The code can be letters, numbers, symbols, etc., and this scheme does not impose specific limitations on this.

[0108] In this optional embodiment, when any image is randomly selected from all the first filter image sets and all the second filter image sets and input into the neural network, the image set corresponding to the current input image can be determined first according to the attribution code of the current input image, and the corresponding loss function can be selected and calculated accordingly to obtain the liveness detection loss corresponding to the current input image, and finally the neural network can be continuously trained according to the liveness detection loss.

[0109] For example, suppose there are a total of 5 first filter image sets, where A m Let m represent the m-th first filter image set. Then, the attribute code for all images in the m-th first filter image set is A. m There are a total of 8 second filter image sets, then the nth second filter image set B n All images are assigned a B code. n If the attribution code of the image currently input to the neural network is A2, then it can be determined that the current input image belongs to the second first filter image set. In this case, it is only necessary to calculate the classification loss and content loss corresponding to the current input image and sum them to obtain the liveness detection loss corresponding to the current input image. If the attribution code of the image currently input to the neural network is B3, then it can be determined that the current input image belongs to the third second filter image set. In this case, it is only necessary to calculate the color loss and content loss corresponding to the current input image and sum them to obtain the liveness detection loss corresponding to the current input image.

[0110] In this way, the neural network can be effectively trained using the face test image set and the liveness detection loss function, thereby obtaining a face liveness detection model with more accurate detection results.

[0111] S14, based on the face liveness detection model, detect the face image to be detected to obtain the face liveness detection result.

[0112] In an optional embodiment, the obtained face liveness detection model can be used to quickly detect the face image to be detected to obtain the face liveness detection result, thereby effectively identifying whether the face image to be detected under different filter colors is a real person, improving the security of face recognition in fields such as finance and security.

[0113] In this optional embodiment, the face liveness detection model can output the face liveness probability value corresponding to the final image by using the softmax activation function, and obtain the face liveness detection result by comparing the liveness probability value with the preset liveness probability threshold. The face liveness detection result includes pass and fail.

[0114] In this optional embodiment, the liveness probability threshold can be 0.75. When the liveness probability value is greater than the liveness probability threshold, the face liveness detection result is passed. When the liveness probability value is not greater than the liveness probability threshold, the face liveness detection result is failed. At this time, staff can be notified to intervene and perform secondary verification of the face image to be detected through manual video verification.

[0115] Thus, the face liveness detection model can quickly detect the face image to be detected, thereby improving the efficiency of face liveness detection.

[0116] Please see Figure 2 , Figure 2 This is a functional block diagram of a preferred embodiment of the AI-based face liveness detection device of this application. The AI-based face liveness detection device 11 includes a data acquisition module 110, an extraction module 111, a construction module 112, a training module 113, and a detection module 114. The unit / module referred to in this application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each unit / module will be described in detail in subsequent embodiments.

[0117] In an optional embodiment, the acquisition module 110 is used to add various filter colors to the acquired face images to obtain a face test image set.

[0118] In an optional embodiment, the step of adding multiple filter colors to the acquired face images to obtain a face test image set includes:

[0119] Different facial targets are acquired, and multiple facial images are taken for each facial target to obtain a facial target image set;

[0120] A first filter image set and a second filter image set are obtained by adding multiple different filter colors to the images in the face target image set. Each image in the first filter image set has the same face target and different filter colors, and each image in the second filter image set has the same filter color and different face targets.

[0121] The first filter image set and the second filter image set are used as the face test image set.

[0122] In an optional embodiment, the extraction module 111 is used to extract features from the images in the face test image set to obtain face test features.

[0123] In an optional embodiment, the step of extracting features from the images in the face test image set to obtain face test features includes:

[0124] Based on a preset feature extractor, multi-layer convolution operations are performed on the images in the face test image set to obtain shallow features at different levels of the images;

[0125] Based on a preset color encoder, shallow features at different levels of the image are fused to obtain deep color fusion features;

[0126] Perform a convolution operation on the deep color fusion features to obtain deep color convolution features;

[0127] After pooling the deep color convolutional features, the deep color features are output through a fully connected layer.

[0128] The shallow features and deep color features of different levels of the image are used as the face test features of the image.

[0129] In an optional embodiment, the construction module 112 is used to construct a liveness detection loss function based on the face test image set and the face test features.

[0130] In an optional embodiment, for images in the first filter image set, although they have different filter colors, they have the same facial target. Therefore, the output images after liveness detection should have the same result. Thus, in order to prevent the liveness detection process from being affected by filter color, a classification loss function L can be constructed. classify The classification loss function satisfies the following relationship:

[0131]

[0132] Among them, L classify x1 i Let x1 represent the classification loss corresponding to the i-th image in the first filter image set, and y1 represent the label image set corresponding to x1. i For the i-th image x1 in the first filter image set i The labeled image, where N1 represents the number of images in the first filter image set, N c CE(x1) represents the number of filter colors included in the first filter image set. i y1 i ) represents the cross-entropy between the i-th image in the first filter image set and its corresponding labeled image, and color k G(x1) represents the color of the k-th filter. i color k ) represents image x1 i Having the k-th filter color, CE(G(x1) i colork ), y1 i () represents the image x1 with the k-th filter color. i With the corresponding label image y1 i Cross-entropy between them.

[0133] In an optional embodiment, for images in the second filter image set, although they have different facial targets, they share the same filter color. Therefore, the color features of each image should be as similar as possible, thereby enabling the construction of a color loss function L. color The color loss function satisfies the following relationship:

[0134]

[0135] Among them, L color x2 i Let x1 represent the color loss corresponding to the i-th image in the second filter image set, x2 represent the second filter image set, and γ represent the color loss. k，l =e -dis(k，l) This is used to measure the similarity between the k-th filter color and the l-th filter color. sim(i, j, k, l) represents the similarity between image i with the k-th filter color and image j with the l-th filter color, where k and l can be the same. N² is the total number of images in the second filter image set, and α is a hyperparameter, which is taken as 1 in this scheme. c This represents the total number of filter colors included in the first filter image set. Since each second filter image set corresponds to only one filter color, this solution adds N to the above formula. c The average loss of a single image under all filter colors is calculated as the color loss to obtain a more accurate color loss, where sim(i,j,k,l) ​​satisfies the following relationship:

[0136]

[0137] Among them, z color The deep color features are represented by sim(i,j,k,l) ​​when k=l, which represents the similarity between image i and image j belonging to the same second filter image set. When k≠l, sim(i,j,k,l) ​​represents the similarity between image i and image j belonging to different second filter image sets. Thus, the color loss function can control the similarity of color features according to the similarity between filter colors.

[0138] In an optional embodiment, the training module 113 is used to train a neural network based on the face test image set and the liveness detection loss function to obtain a face liveness detection model.

[0139] In an optional embodiment, the detection module 114 is used to detect the face image to be detected based on the face liveness detection model to obtain the face liveness detection result.

[0140] As can be seen from the above technical solutions, this application can construct multiple loss functions by applying different filter colors to the collected face images, thereby training the neural network in all aspects, thereby improving the ability of the obtained face liveness detection model to defend against filter color attacks and effectively reducing the risk of face fraud.

[0141] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 executes the computer-readable instructions stored in the memory to implement the artificial intelligence-based face liveness detection method described in any of the above embodiments.

[0142] In an optional embodiment, the electronic device 1 further includes a bus and a computer program stored in the memory 12 and executable on the processor 13, such as an AI-based face liveness detection program.

[0143] Figure 3 Only electronic device 1 with memory 12 and processor 13 is shown. It will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0144] Combination Figure 1 The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement an artificial intelligence-based face liveness detection method, and the processor 13 can execute the plurality of instructions to achieve the following:

[0145] Multiple filter colors were added to the collected face images to obtain a set of face test images;

[0146] Feature extraction is performed on the images in the face test image set to obtain face test features;

[0147] A liveness detection loss function is constructed based on the face test image set and the face test features;

[0148] A face liveness detection model is obtained by training a neural network based on the aforementioned face test image set and liveness detection loss function;

[0149] The face liveness detection model is used to detect the face image to be detected and obtain the face liveness detection result.

[0150] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0151] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. Electronic device 1 can be a bus-type structure or a star-type structure. Electronic device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, electronic device 1 may also include input / output devices, network access devices, etc.

[0152] It should be noted that electronic device 1 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0153] The memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a portable hard drive of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 1. The memory 12 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an AI-based face liveness detection program, but also to temporarily store data that has been output or will be output.

[0154] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the electronic device 1, connecting various components of the electronic device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing an AI-based face liveness detection program) and calls data stored in the memory 12 to perform various functions and process data in the electronic device 1.

[0155] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the various embodiments of the artificial intelligence-based face liveness detection method described above, for example... Figure 1 The steps are shown.

[0156] For example, the computer program may be divided into one or more units / modules, which are stored in the memory 12 and executed by the processor 13 to complete this application. The one or more units / modules may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a data acquisition module 110, an extraction module 111, a construction module 112, a training module 113, and a detection module 114.

[0157] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the artificial intelligence-based face liveness detection method described in the various embodiments of this application.

[0158] If the unit / module integrated in electronic device 1 is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0159] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0160] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0161] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0162] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The symbol is represented by only one arrow, but this does not indicate that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0163] This application also provides a computer-readable storage medium (not shown), which stores computer-readable instructions. These computer-readable instructions are executed by a processor in an electronic device to implement the artificial intelligence-based face liveness detection method described in any of the above embodiments.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0165] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0167] Furthermore, the word "comprising" clearly does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or devices described in the specification can also be implemented by a single module or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A face liveness detection method based on artificial intelligence, characterized in that, The method includes: A face test image set is obtained by adding multiple filter colors to the acquired face images, including: acquiring different face targets and taking multiple face images for each face target to obtain a face target image set; adding multiple different filter colors to the images in the face target image set to obtain a first filter image set and a second filter image set, wherein each image in the first filter image set has the same face target and different filter colors, and each image in the second filter image set has the same filter color and different face targets; and using the first filter image set and the second filter image set as the face test image set. Feature extraction is performed on the images in the face test image set to obtain face test features; A liveness detection loss function is constructed based on the face test image set and the face test features; the liveness detection loss function includes a classification loss function, which satisfies the following relationship: ;in, This represents the classification loss corresponding to the i-th image in the first filter image set. This represents the first filter image set. Indicates the The corresponding set of labeled images, The i-th image in the first filter image set Label image after annotation This indicates the number of images in the first filter image set. The number of filter colors included in the first filter image set. Let be the cross-entropy between the i-th image in the first filter image set and its corresponding labeled image. This represents the color of the k-th filter. Representing an image Having the k-th filter color, This represents an image with the k-th filter color. With the corresponding label image Cross-entropy between; A face liveness detection model is obtained by training a neural network based on the aforementioned face test image set and liveness detection loss function; The face liveness detection model is used to detect the face image to be detected and obtain the face liveness detection result.

2. The face liveness detection method based on artificial intelligence as described in claim 1, characterized in that, The step of extracting features from the images in the face test image set to obtain face test features includes: Based on a preset feature extractor, multi-layer convolution operations are performed on the images in the face test image set to obtain shallow features at different levels of the images; Based on a preset color encoder, shallow features at different levels of the image are fused to obtain deep color fusion features; Perform a convolution operation on the deep color fusion features to obtain deep color convolution features; After pooling the deep color convolutional features, the deep color features are output through a fully connected layer. The shallow features and deep color features of different levels of the image are used as the face test features of the image.

3. The face liveness detection method based on artificial intelligence as described in claim 2, characterized in that, The shallow features of the image at different levels include first shallow features, second shallow features, third shallow features, and fourth shallow features. The process of fusing the shallow features of the image at different levels according to a preset color encoder to obtain deep color fusion features includes: The first shallow color feature is obtained by performing a first convolution operation on the second shallow feature according to a preset color encoder; The first shallow color feature and the third shallow feature are fused to obtain a shallow color fusion feature; The second shallow color feature is obtained by performing a second convolution operation on the shallow color fusion feature according to the preset color encoder. The deep color fusion feature is obtained by fusing the second shallow color feature and the fourth shallow color feature.

4. The face liveness detection method based on artificial intelligence as described in claim 1, characterized in that, The liveness detection loss function also includes a color loss function, which satisfies the following relationship: in, This represents the color loss corresponding to the i-th image in the second filter image set. This represents the second filter image set. Used to measure the color of the k-th filter relative to the color of the first filter. The similarity between the colors of the filters This represents the image i with the k-th filter color and the image i with the k-th filter color. Similarity between images j with different filter colors This represents the total number of images in the second filter image set. For hyperparameters, This represents the total number of filter colors included in the first filter image set, where, Satisfying the relation: in, Represents deep color features, when k= hour, Used to represent the similarity between image i and image j belonging to the same second filter image set, when hour, This is used to represent the similarity between image i and image j, which belong to different sets of images with the second filter.

5. The artificial intelligence-based face liveness detection method as described in claim 1, characterized in that, The liveness detection loss function also includes a content loss function, which satisfies the following relationship: in, This represents the content loss corresponding to the i-th shallow feature image extracted from the face test image set. This represents the i-th shallow feature image arbitrarily selected from the face test image set. This represents the face test feature corresponding to the i-th shallow feature image extracted from the face test image set. This represents the face test feature corresponding to the i-th shallow feature image after adding the k-th filter color, extracted from the face test image set. N represents the total number of images in the first filter image set and the second filter image set. This represents the total number of filter colors included in the first filter image set.

6. A face liveness detection device based on artificial intelligence, characterized in that, The device includes an acquisition module, an extraction module, a construction module, a training module, and a detection module. The acquisition module is used to add multiple filter colors to the acquired face images to obtain a face test image set, including: acquiring different face targets and taking multiple face images for each face target to obtain a face target image set; adding multiple different filter colors to the images in the face target image set to obtain a first filter image set and a second filter image set, wherein each image in the first filter image set has the same face target and different filter colors, and each image in the second filter image set has the same filter color and different face targets; and using the first filter image set and the second filter image set as the face test image set. The extraction module is used to extract features from the images in the face test image set to obtain face test features; The construction module is used to construct a liveness detection loss function based on the face test image set and the face test features; the liveness detection loss function includes a classification loss function, which satisfies the following relationship: ;in, This represents the classification loss corresponding to the i-th image in the first filter image set. This represents the first filter image set. Indicates the The corresponding set of labeled images, The i-th image in the first filter image set Label image after annotation This indicates the number of images in the first filter image set. The number of filter colors included in the first filter image set. Let be the cross-entropy between the i-th image in the first filter image set and its corresponding labeled image. This represents the color of the k-th filter. Representing an image Having the k-th filter color, This represents an image with the k-th filter color. With the corresponding label image Cross-entropy between; The training module is used to train a neural network based on the face test image set and the liveness detection loss function to obtain a face liveness detection model; The detection module is used to detect the face image to be detected based on the face liveness detection model to obtain the face liveness detection result.

7. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the AI-based face liveness detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the artificial intelligence-based face liveness detection method as described in any one of claims 1 to 5.

Images