Face living body detection method, device and equipment in motion state scene and medium

Abstract

The application relates to the field of intelligent decision-making, and discloses a face living body detection method, device and equipment in a motion state scene and a medium. The method comprises the following steps: collecting a face image in a motion state scene, performing face detection on the face image to obtain a detected face, performing format standardization on the detected face to obtain a standardized face; performing texture coding on the standardized face to obtain coded texture of the standardized face, extracting texture information, calculating pixel change information, and constructing three-dimensional structure information of the standardized face; respectively performing feature extraction on the texture information, the pixel change information and the three-dimensional structure information to obtain texture features, pixel change features and three-dimensional structure features; performing feature fusion on the texture features, the pixel change features and the three-dimensional structure features to obtain fused features; and calculating a living body detection score of the standardized face to determine a face living body detection result of the face image. The application can improve the comprehensiveness of face living body detection in a motion state scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making, and in particular to a method, apparatus, device, and medium for detecting human face liveness in motion scenarios. Background Technology

[0002] Face liveness detection in motion scenarios refers to the process of identifying whether the currently acquired face image comes from a live face or a fake face.

[0003] Currently, attention-based fusion methods have been proposed for face liveness detection, fusing RGB and MSR features. However, in deep neural networks, feature maps from deeper layers express higher semantic-level information, which can cause problems when deception cues are located in lower-level image pixels. Furthermore, these methods have limitations in handling a limited number of feature classes. Concatenating or parallelizing multiple feature vectors treats each vector independently and equally, failing to utilize the interrelationships between features. Therefore, the neglect of low-level information and interrelationships in face images leads to insufficient comprehensiveness in face liveness detection in motion-based scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method, apparatus, device, and medium for face liveness detection in motion scenarios. This method enhances the focus on low-level information and the relationships between features in face images, thereby improving the comprehensiveness of face liveness detection in motion scenarios.

[0005] In a first aspect, the present invention provides a method for face liveness detection in a moving scene, comprising:

[0006] Acquire face images in a moving scene, perform face detection on the face images to obtain detected faces, and standardize the format of the detected faces to obtain standardized faces;

[0007] The standardized face is texture encoded to obtain the encoded texture of the standardized face. Based on the encoded texture, the texture information in the standardized face is extracted, the pixel change information in the standardized face is calculated, and the three-dimensional structural information of the standardized face is constructed.

[0008] Feature extraction is performed on the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features. Feature fusion is then performed on the texture features, pixel change features, and three-dimensional structure features to obtain fused features.

[0009] Based on the fusion features, the liveness detection score of the standardized face is calculated, and the liveness detection result of the face image is determined using the liveness detection score.

[0010] In one possible implementation of the first aspect, the standardization of the detected face to obtain a standardized face includes:

[0011] Obtain the facial key points corresponding to the detected face;

[0012] Calculate the similarity transformation matrix between the facial key points and the preset standard facial key points;

[0013] The detected face is transformed using the similarity transformation matrix to obtain the standardized face.

[0014] In one possible implementation of the first aspect, the step of texture encoding the standardized face to obtain the encoded texture of the standardized face includes:

[0015] The standardized face is divided into regions to obtain the divided face regions;

[0016] The pixel texture values ​​in the segmented face region are calculated using the following formula:

[0017]

[0018]

[0019] Where, L(x) c ,y c ) represents the pixel texture value, (x c ,y c ) represents the coordinates of the pixels in the defined face region, and p represents (x c ,y c The number of neighboring pixels of a pixel, p = 8, i c It means (x) c ,y c The grayscale value of a pixel, i n s(x) represents the gray value of a neighboring pixel, and s(x) represents the sign function;

[0020] The pixel texture values ​​are statistically analyzed to obtain the statistical frequency of the pixel texture values.

[0021] The statistical frequencies are normalized to obtain normalized frequencies;

[0022] The normalized frequency is vector-encoded to obtain the encoded texture.

[0023] In one possible implementation of the first aspect, calculating the pixel change information in the standardized face includes:

[0024] The pixel intensity difference value in the standardized face is calculated using the following formula:

[0025]

[0026] Wherein, ε(y) c ) represents the pixel intensity difference value, q represents the total number of neighborhood points of any randomly selected pixel in the standardized face, p = 8, y c y represents the grayscale value of any pixel selected from the standardized face. j Indicates y c The gray values ​​of the neighboring pixels, where j represents the index of the neighboring pixels;

[0027] Calculate the pixel gradient direction values ​​in the standardized face;

[0028] Perform histogram transformation on the pixel intensity difference value and the pixel gradient direction value to obtain an intensity difference image and a gradient direction image;

[0029] The intensity difference image and the gradient direction image are merged to obtain the pixel change information.

[0030] In one possible implementation of the first aspect, constructing the three-dimensional structural information of the standardized face includes:

[0031] The three-dimensional structural values ​​of the standardized face are calculated using the following formula:

[0032] I(x,y)=ρ(x,y)n T (x,y)s

[0033] Where I(x,y) represents the three-dimensional structure value of the standardized face, ρ represents the albedo of the face image corresponding to the standardized face, and n T (x, y) represents the surface normal of the standardized face, which is represented by a 3D shape. s represents the light source illuminating the face image. Since the screens for photos and replaying videos are two-dimensional planar structures, n... T (x,y) is a constant;

[0034] Construct the three-dimensional structure vector of the stated three-dimensional structure value;

[0035] The three-dimensional structure vector is converted into an image format to obtain the three-dimensional structure information.

[0036] In one possible implementation of the first aspect, the feature fusion of the texture features, the pixel variation features, and the three-dimensional structural features to obtain fused features includes:

[0037] The importance of the texture features, the importance of the pixel variation features, and the importance of the three-dimensional structural features are calculated using the following formulas:

[0038] d i =q T f i

[0039] Wherein, d1 represents the importance of the texture feature, d2 represents the importance of the variation feature, d3 represents the importance of the three-dimensional feature, and d i The importance is indicated by i = 1, 2, 3, q ​​represents the query vector, f1 represents the texture feature, f2 represents the pixel change feature, f3 represents the three-dimensional structure feature, and T represents the transpose symbol.

[0040] Based on the importance of the texture features, the importance of the variation features, and the importance of the three-dimensional features, the texture feature weights, variation feature weights, and three-dimensional feature weights of the texture features, the pixel variation features, and the three-dimensional structural features are calculated using the following formulas:

[0041]

[0042] Wherein, ω1 represents the texture feature weight, ω2 represents the variation feature weight, ω3 represents the three-dimensional feature weight, d1 represents the texture feature importance, d2 represents the variation feature importance, d3 represents the three-dimensional feature importance, and d i Indicates importance, i = 1, 2, 3;

[0043] The fused feature is calculated using the following formula based on the texture feature weights, the variation feature weights, and the 3D feature weights:

[0044]

[0045] Where v represents the fusion feature, i = 1, 2, 3, f1 represents the texture feature, f2 represents the pixel variation feature, f3 represents the three-dimensional structure feature, ω1 represents the texture feature weight, ω2 represents the variation feature weight, and ω3 represents the three-dimensional feature weight.

[0046] In one possible implementation of the first aspect, calculating the liveness detection score of the standardized face based on the fused features includes:

[0047] The fused features are linearly fitted using the following formula to obtain the fitted features:

[0048] Z = b + uv

[0049] Where Z represents the fitting feature, v represents the fusion feature, b represents the bias parameter of the model performing liveness detection, and u represents the weight parameter of the model performing liveness detection.

[0050] Based on the fitted features, the liveness detection score is calculated using the following formula:

[0051]

[0052] Among them, y ′ Z represents the liveness detection score, and Z represents the fitted feature.

[0053] Secondly, the present invention provides a face liveness detection device in a motion scenario, the device comprising:

[0054] The face standardization module is used to acquire face images in motion scenarios, perform face detection on the face images to obtain detected faces, and perform format standardization on the detected faces to obtain standardized faces.

[0055] A 3D construction module is used to perform texture encoding on the standardized face to obtain the encoded texture of the standardized face, extract texture information from the standardized face based on the encoded texture, calculate pixel change information in the standardized face, and construct the 3D structural information of the standardized face.

[0056] The feature fusion module is used to extract features from the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features, and to fuse the texture features, pixel change features, and three-dimensional structure features to obtain fused features;

[0057] The result determination module is used to calculate the liveness detection score of the standardized face based on the fusion features, and use the liveness detection score to determine the face liveness detection result of the face image.

[0058] Thirdly, the present invention provides an electronic device, comprising:

[0059] At least one processor; and a memory communicatively connected to said at least one processor;

[0060] The memory stores a computer program that can be executed by the at least one processor, enabling the at least one processor to execute the face liveness detection method in motion scenarios as described in any of the first aspects above.

[0061] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the face liveness detection method in a motion state scenario as described in any one of the first aspects above.

[0062] Compared with existing technologies, the technical principles and beneficial effects of this solution are as follows:

[0063] This invention first acquires facial images in a moving scene for liveness detection. Further, it performs face detection on the facial images to extract the facial portion while ignoring irrelevant information. Further, it standardizes the detected faces to extract accurate facial images while retaining some background information, improving the performance of subsequent feature extraction and fixing the detected faces to a standard size. Secondly, it performs texture encoding on the standardized faces to ensure the stability of facial information. Since non-living facial information may exhibit localized highlights and blurring due to poor exposure and noise interference, encoding texture details ensures information stability in non-living facial information, facilitating subsequent feature extraction. Further, it extracts texture information from the standardized faces based on the encoded texture for feature extraction. Further, it calculates pixel change information in the standardized faces to compensate for the inadequacy of texture information in reflecting facial features. To address the shortcomings of localized grayscale variations within a window and the difficulty in reflecting the inherent variations in texture, this invention enhances the relationship between texture and pixel variation features. Furthermore, this embodiment constructs the three-dimensional structural information of the standardized face to highlight the three-dimensional structural information of a living face, based on the characteristics of non-living faces primarily existing in two-dimensional planar media such as paper, video device screens, and photographs. Further, this embodiment extracts features from the texture information, pixel variation information, and three-dimensional structural information respectively, leveraging the higher output accuracy of residual networks to improve the accuracy of feature extraction from low-level information in face images, ensuring that subsequent feature fusion results are not affected by this accuracy. Further, this embodiment fuses the texture features, pixel variation features, and three-dimensional structural features to correlate the relationships between different features in the face image, improving the comprehensiveness of face liveness detection. Finally, this embodiment calculates the liveness detection score of the standardized face based on the fused features, aggregating the recognition results of features in the face image into a score value, facilitating the judgment of liveness detection results based on the numerical value. Therefore, the face liveness detection method, apparatus, device, and medium proposed in this embodiment of the invention can enhance the focus on low-level information and the interrelationships between features in face images, thereby improving the comprehensiveness of face liveness detection in motion scenarios. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart illustrating a face liveness detection method in a motion scenario according to an embodiment of the present invention.

[0067] Figure 2 As shown in one embodiment of the present invention Figure 1 A flowchart illustrating one step of a face liveness detection method in a motion scenario;

[0068] Figure 3 As shown in one embodiment of the present invention Figure 1 A flowchart illustrating another step in a face liveness detection method for motion scenarios;

[0069] Figure 4 This is a schematic diagram of a face liveness detection device in a motion scenario according to an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of the internal structure of an electronic device that implements a face liveness detection method in a motion state scenario, according to an embodiment of the present invention. Detailed Implementation

[0071] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0072] This invention provides a method for face liveness detection in motion scenarios. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this invention: a server, a terminal, etc. In other words, the face liveness detection method in motion scenarios can be executed by software or hardware installed on a terminal device or a server device. The software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0073] See Figure 1 The diagram shown is a flowchart illustrating a face liveness detection method in a motion scenario according to an embodiment of the present invention. Figure 1 The face liveness detection methods described in the article for motion scenarios include:

[0074] S1. Acquire face images in a moving scene, perform face detection on the face images to obtain detected faces, and standardize the format of the detected faces to obtain standardized faces.

[0075] This invention collects facial images in motion scenarios to perform liveness detection on facial information in motion scenarios.

[0076] Furthermore, in this embodiment of the invention, face detection is performed on the face image to extract the face portion of the image and ignore information unrelated to the face portion.

[0077] In one embodiment of the present invention, the step of performing face detection on the face image to obtain a detected face includes: identifying face candidate windows in the face image; performing window correction on the face candidate windows to obtain a corrected candidate window; determining the facial key points of the face image based on the corrected candidate window, and taking the facial key parts corresponding to the facial key points as the detected face.

[0078] For example, the process of identifying candidate face windows in the face image is implemented using the P-Net network structure of a multi-task cascaded CNN face detection deep learning model. The P-Net network structure includes three convolutions plus Max-Pooling operations. The process of window correction of the candidate face windows to obtain corrected candidate windows is implemented using the R-Net network structure of a multi-task cascaded CNN face detection deep learning model. The R-Net network structure includes three convolutions plus Max-Pooling operations, and a fully connected layer is connected after the feature map of the last layer. Fully connected operations are also used when connecting the three different tasks. The process of determining the facial key points of the face image and using the facial key parts corresponding to the facial key points as the detected face is implemented using the O-Net network structure of a multi-task cascaded CNN face detection deep learning model. The O-Net network structure is similar to the R-Net network structure described above and will not be further described here.

[0079] Furthermore, in this embodiment of the invention, the detected face is format-standardized to extract accurate face images while retaining some face background images, thereby improving the detection performance of subsequent feature extraction and fixing the detected face to a standard format size.

[0080] In one embodiment of the present invention, see reference Figure 2 As shown, the process of standardizing the detected face to obtain a standardized face includes:

[0081] S201. Obtain the facial key points corresponding to the detected face;

[0082] S202. Calculate the similarity transformation matrix between the facial key points and the preset standard facial key points;

[0083] S203. Perform a standard space transformation on the detected face using the similarity transformation matrix to obtain the standardized face.

[0084] For example, the least squares method is used to solve the spatial transformation matrix between the key points of the current face image and the predefined standard face key points. Then, the spatial transformation corresponding to the spatial transformation matrix is ​​performed on the face image to obtain the transformed standard-sized face image.

[0085] S2. Perform texture encoding on the standardized face to obtain the encoded texture of the standardized face. Based on the encoded texture, extract the texture information in the standardized face, calculate the pixel change information in the standardized face, and construct the three-dimensional structure information of the standardized face.

[0086] This invention embodiment uses texture encoding on the standardized face to ensure the stability of the face information. Since non-living face information may exhibit local highlights and image blurring due to poor exposure and noise interference, encoding the texture details can ensure that the information remains stable in non-living face information, making it easier for subsequent feature extraction.

[0087] In one embodiment of the present invention, the step of performing texture encoding on the standardized face to obtain the encoded texture of the standardized face includes: dividing the standardized face into regions to obtain divided face regions; and calculating the pixel texture values ​​in the divided face regions using the following formula:

[0088]

[0089]

[0090] Where, L(x) c ,y c ) represents the pixel texture value, (x c ,y c ) represents the coordinates of the pixels in the defined face region, and p represents (x c ,y c The number of neighboring pixels of a pixel, p = 8, i c It means (x) c ,y c The grayscale value of a pixel, i n s(x) represents the gray value of a neighboring pixel, and s(x) represents the sign function;

[0091] The pixel texture values ​​are statistically analyzed to obtain the statistical frequency of the pixel texture values; the statistical frequency is normalized to obtain the normalized frequency; the normalized frequency is vector-encoded to obtain the encoded texture.

[0092] Optionally, the process of performing frequency statistics on the pixel texture values ​​to obtain the statistical frequency of the pixel texture values ​​refers to calculating the frequency of the pixel texture value of each pixel in each segmented face region in its respective segmented face region; the process of performing vector encoding on the normalized frequency to obtain the encoded texture refers to concatenating the frequencies of pixels in each segmented face region into a vector.

[0093] Furthermore, in this embodiment of the invention, texture information in the standardized face is extracted based on the encoded texture for feature information extraction of the standardized face.

[0094] Optionally, the process of extracting texture information from the standardized face based on the encoded texture is achieved by using a convolutional neural network to extract features from the encoded texture.

[0095] Furthermore, this embodiment of the invention calculates pixel change information in the standardized face to compensate for the shortcomings of the texture information in fully reflecting the spatial distribution structure information of local grayscale changes and the difficulty in reflecting the inherent variation characteristics of the texture, thereby enhancing the relationship between texture and pixel change characteristics. Here, the pixel change information refers to the change information of pixel brightness intensity.

[0096] In one embodiment of the present invention, calculating the pixel change information in the standardized face includes: calculating the pixel intensity difference value in the standardized face using the following formula:

[0097]

[0098] Wherein, ε(y) c ) represents the pixel intensity difference value, q represents the total number of neighborhood points of any randomly selected pixel in the standardized face, p = 8, y c y represents the grayscale value of any pixel selected from the standardized face. j Indicates y c The gray values ​​of the neighboring pixels, where j represents the index of the neighboring pixels;

[0099] Calculate the pixel gradient direction values ​​in the standardized face; perform histogram transformation on the pixel intensity difference values ​​and the pixel gradient direction values ​​to obtain an intensity difference image and a gradient direction image; perform image merging processing on the intensity difference image and the gradient direction image to obtain the pixel change information.

[0100] Optionally, the process of merging the intensity difference image and the gradient direction image to obtain the pixel change information refers to the process of converting the two-dimensional array formed by combining the intensity difference image and the gradient direction image into a one-dimensional array.

[0101] Furthermore, embodiments of the present invention construct the three-dimensional structural information of the standardized human face to highlight the three-dimensional structural information of the living face based on the characteristics of non-living human faces, which mainly exist in two-dimensional planar media such as paper, video device screens, and photographs.

[0102] In one embodiment of the present invention, constructing the three-dimensional structural information of the standardized face includes: calculating the three-dimensional structural value of the standardized face using the following formula:

[0103] I(x,y)=ρ(x,y)n T(x,y)s

[0104] Where I(x,y) represents the three-dimensional structure value of the standardized face, ρ represents the albedo of the face image corresponding to the standardized face, and n T (x, y) represents the surface normal of the standardized face, which is represented by a 3D shape. s represents the light source illuminating the face image. Since the screens for photos and replaying videos are two-dimensional planar structures, n... T (x,y) is a constant;

[0105] Construct a three-dimensional structure vector of the three-dimensional structure value; perform image-based conversion on the three-dimensional structure vector to obtain the three-dimensional structure information.

[0106] Optionally, the process of constructing the three-dimensional structure vector of the three-dimensional structure values ​​is achieved by first converting the face image composed of the three-dimensional structure values ​​into a histogram image, and then performing vector combination on the pixel values ​​in the histogram image.

[0107] S3. Extract features from the texture information, pixel change information and three-dimensional structure information respectively to obtain texture features, pixel change features and three-dimensional structure features. Then, fuse the texture features, pixel change features and three-dimensional structure features to obtain fused features.

[0108] This invention employs feature extraction from the texture information, pixel variation information, and three-dimensional structural information respectively. This leverages the higher output accuracy of residual networks to improve the accuracy of feature extraction from low-level information in face images, ensuring that subsequent feature fusion results are not affected by the accuracy at this stage. The residual network refers to a deep residual network, composed of multiple convolutional layers, primarily used for feature extraction.

[0109] In one embodiment of the present invention, the feature extraction of the texture information, the pixel change information and the three-dimensional structure information is performed respectively to obtain texture features, pixel change features and three-dimensional structure features through a residual network.

[0110] Furthermore, embodiments of the present invention perform feature fusion on the texture features, the pixel variation features, and the three-dimensional structural features to improve the comprehensiveness of face liveness detection by associating the relationships between different features in a face image.

[0111] In one embodiment of the present invention, the feature fusion of the texture features, the pixel variation features, and the three-dimensional structural features to obtain fused features includes: calculating the importance of the texture features, the importance of the pixel variation features, and the importance of the three-dimensional structural features using the following formulas:

[0112] d i =q T f i

[0113] Wherein, d1 represents the importance of the texture feature, d2 represents the importance of the variation feature, d3 represents the importance of the three-dimensional feature, and d i The importance is indicated by i = 1, 2, 3, q ​​represents the query vector, f1 represents the texture feature, f2 represents the pixel change feature, f3 represents the three-dimensional structure feature, and T represents the transpose symbol.

[0114] Based on the importance of the texture features, the importance of the variation features, and the importance of the three-dimensional features, the texture feature weights, variation feature weights, and three-dimensional feature weights of the texture features, the pixel variation features, and the three-dimensional structural features are calculated using the following formulas:

[0115]

[0116] Wherein, ω1 represents the texture feature weight, ω2 represents the variation feature weight, ω3 represents the three-dimensional feature weight, d1 represents the texture feature importance, d2 represents the variation feature importance, d3 represents the three-dimensional feature importance, and d i Indicates importance, i = 1, 2, 3;

[0117] The fused feature is calculated using the following formula based on the texture feature weights, the variation feature weights, and the 3D feature weights:

[0118]

[0119] Where v represents the fusion feature, i = 1, 2, 3, f1 represents the texture feature, f2 represents the pixel variation feature, f3 represents the three-dimensional structure feature, ω1 represents the texture feature weight, ω2 represents the variation feature weight, and ω3 represents the three-dimensional feature weight.

[0120] S4. Based on the fusion features, calculate the liveness detection score of the standardized face, and use the liveness detection score to determine the face liveness detection result of the face image.

[0121] In this embodiment of the invention, the liveness detection score of the standardized face is calculated based on the fusion features, so as to aggregate the recognition results of features in the face image into a score value, which facilitates the judgment of the liveness detection result based on the value.

[0122] In one embodiment of the present invention, calculating the liveness detection score of the standardized face based on the fusion features includes: linearly fitting the fusion features using the following formula to obtain the fitted features:

[0123] Z = b + uv

[0124] Where Z represents the fitting feature, v represents the fusion feature, b represents the bias parameter of the model performing liveness detection, and u represents the weight parameter of the model performing liveness detection.

[0125] Based on the fitted features, the liveness detection score is calculated using the following formula:

[0126]

[0127] Among them, y ′ Z represents the liveness detection score, and Z represents the fitted feature.

[0128] In one embodiment of the present invention, see reference Figure 3 As shown, determining the face liveness detection result of the face image using the liveness detection score includes:

[0129] S301. Set the detection score threshold for the liveness detection score;

[0130] S302. When the liveness detection score is not greater than the detection score threshold, the failure of liveness detection of the face image is taken as the face liveness detection result.

[0131] S303. When the liveness detection score is greater than the detection score threshold, the successful liveness detection of the face image is taken as the face liveness detection result.

[0132] As can be seen, this embodiment of the invention first acquires face images in a moving scene for liveness detection of face information in the moving scene. Further, this embodiment performs face detection on the face images to extract the face portion of the image and ignores information unrelated to the face portion. Further, this embodiment standardizes the format of the detected face to extract accurate face images while retaining some background image information, thereby improving the detection performance of subsequent feature extraction and fixing the detected face to a standard format size. Secondly, this embodiment performs texture encoding on the standardized face to ensure the stability of the face information. Since non-live face information may exhibit local highlights and image blurring due to poor exposure and noise interference, encoding the texture details ensures that the information remains stable in non-live face information, facilitating subsequent feature extraction. Further, this embodiment extracts texture information from the standardized face based on the encoded texture for feature information extraction from the standardized face. Further, this embodiment calculates pixel change information in the standardized face to compensate for the incompleteness of the texture information. To address the shortcomings of reflecting the spatial distribution structure information of local grayscale changes within a window and the difficulty in reflecting the inherent variation characteristics of texture, this invention enhances the relationship between texture and pixel variation features. Furthermore, this embodiment constructs the three-dimensional structural information of the standardized face to highlight the three-dimensional structural information of a living face, based on the characteristics of non-living faces primarily existing in two-dimensional planar media such as paper, video device screens, and photographs. Further, this embodiment extracts features from the texture information, pixel variation information, and three-dimensional structural information respectively, leveraging the higher output accuracy of residual networks to improve the accuracy of feature extraction from low-level information in face images, ensuring that subsequent feature fusion results are not affected by this accuracy. Further, this embodiment fuses the texture features, pixel variation features, and three-dimensional structural features to correlate the relationships between different features in the face image, improving the comprehensiveness of face liveness detection. Finally, this embodiment calculates the liveness detection score of the standardized face based on the fused features, aggregating the recognition results of features in the face image into a score value, facilitating the judgment of liveness detection results based on the numerical value. Therefore, the face liveness detection method in motion scenarios proposed in this embodiment of the invention can enhance the focus on low-level information in face images and the interrelationships between features, thereby improving the comprehensiveness of face liveness detection in motion scenarios.

[0133] like Figure 4 The diagram shown is a functional block diagram of the face liveness detection device in motion scenarios according to the present invention.

[0134] The face liveness detection device 400 for motion scenarios described in this invention can be installed in an electronic device. Depending on the functions implemented, the face liveness detection device for motion scenarios may include a face standardization module 401, a 3D construction module 402, a feature fusion module 403, and a result determination module 404. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0135] In this embodiment of the invention, the functions of each module / unit are as follows:

[0136] The face standardization module 401 is used to acquire face images in a motion scene, perform face detection on the face images to obtain detected faces, and perform format standardization on the detected faces to obtain standardized faces.

[0137] The three-dimensional construction module 402 is used to perform texture encoding on the standardized face to obtain the encoded texture of the standardized face, extract texture information in the standardized face based on the encoded texture, calculate pixel change information in the standardized face, and construct the three-dimensional structure information of the standardized face.

[0138] The feature fusion module 403 is used to extract features from the texture information, the pixel change information and the three-dimensional structure information respectively to obtain texture features, pixel change features and three-dimensional structure features, and to fuse the texture features, pixel change features and three-dimensional structure features to obtain fused features;

[0139] The result determination module 404 is used to calculate the liveness detection score of the standardized face based on the fusion features, and use the liveness detection score to determine the face liveness detection result of the face image.

[0140] In detail, the modules in the face liveness detection device 400 in the motion state scenario described in this embodiment of the invention employ the same methods as described above during use. Figures 1 to 3 The method used is the same as the face liveness detection method in motion scenarios described above, and can produce the same technical effect, so it will not be elaborated here.

[0141] like Figure 5 The diagram shown is a structural schematic of an electronic device that implements a face liveness detection method in a motion-state scenario according to the present invention.

[0142] The electronic device may include a processor 50, a memory 51, a communication bus 52, and a communication interface 53. It may also include a computer program stored in the memory 51 and capable of running on the processor 50, such as a face liveness detection program in motion scenarios.

[0143] In some embodiments, the processor 50 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 50 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 51 (e.g., executing a face liveness detection program in a motion scenario) and calls data stored in the memory 51 to perform various functions of the electronic device and process data.

[0144] The memory 51 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 51 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 51 can include both internal and external storage units of the electronic device. The memory 51 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a database configuration connection program, but also to temporarily store data that has been output or will be output.

[0145] The communication bus 52 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 51 and at least one processor 50, etc.

[0146] The communication interface 53 is used for communication between the electronic device 5 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0147] Figure 5 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 5 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0148] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 50 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0149] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in terms of the scope of the patent invention.

[0150] The database configuration connection program stored in the memory 51 of the electronic device is a combination of multiple computer programs, which, when run in the processor 50, can achieve the following:

[0151] Acquire face images in a moving scene, perform face detection on the face images to obtain detected faces, and standardize the format of the detected faces to obtain standardized faces;

[0152] The standardized face is texture encoded to obtain the encoded texture of the standardized face. Based on the encoded texture, the texture information in the standardized face is extracted, the pixel change information in the standardized face is calculated, and the three-dimensional structural information of the standardized face is constructed.

[0153] Feature extraction is performed on the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features. Feature fusion is then performed on the texture features, pixel change features, and three-dimensional structure features to obtain fused features.

[0154] Based on the fusion features, the liveness detection score of the standardized face is calculated, and the liveness detection result of the face image is determined using the liveness detection score.

[0155] Specifically, the specific implementation method of the above-mentioned computer program by the processor 50 can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0156] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0157] The present invention also provides a storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0158] Acquire face images in a moving scene, perform face detection on the face images to obtain detected faces, and standardize the format of the detected faces to obtain standardized faces;

[0159] The standardized face is texture encoded to obtain the encoded texture of the standardized face. Based on the encoded texture, the texture information in the standardized face is extracted, the pixel change information in the standardized face is calculated, and the three-dimensional structural information of the standardized face is constructed.

[0160] Feature extraction is performed on the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features. Feature fusion is then performed on the texture features, pixel change features, and three-dimensional structure features to obtain fused features.

[0161] Based on the fusion features, the liveness detection score of the standardized face is calculated, and the liveness detection result of the face image is determined using the liveness detection score.

[0162] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0163] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0165] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0166] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0167] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0168] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for face liveness detection in a moving scene, characterized in that, The method includes: Acquire face images in a moving scene, perform face detection on the face images to obtain detected faces, and standardize the format of the detected faces to obtain standardized faces; The standardized face is texture encoded to obtain the encoded texture of the standardized face. Based on the encoded texture, the texture information in the standardized face is extracted, the pixel change information in the standardized face is calculated, and the three-dimensional structural information of the standardized face is constructed. The step of performing texture encoding on the standardized face to obtain the encoded texture of the standardized face includes: The standardized face is divided into regions to obtain the divided face regions; The pixel texture values ​​in the segmented face region are calculated using the following formula: ； ； in, This represents the pixel texture value. This represents the coordinates of the pixels in the defined face region. express The number of neighboring pixels of a pixel express The grayscale value of a pixel. Represents the grayscale value of neighboring pixels. Represents a symbolic function; The pixel texture values ​​are statistically analyzed to obtain the statistical frequency of the pixel texture values. The statistical frequencies are normalized to obtain normalized frequencies; The normalized frequency is vector-encoded to obtain the encoded texture; The process of extracting texture information from the standardized face based on the encoded texture is achieved by using a convolutional neural network to extract features from the encoded texture. The calculation of pixel change information in the standardized face includes: The pixel intensity difference value in the standardized face is calculated using the following formula: in, This represents the pixel intensity difference value. This represents the total number of neighborhood points of any randomly selected pixel in the standardized face. , This represents the grayscale value of any pixel selected arbitrarily within the standardized face. express The gray values ​​of the neighboring pixels, where j represents the index of the neighboring pixels; Calculate the pixel gradient direction values ​​in the standardized face; Perform histogram transformation on the pixel intensity difference value and the pixel gradient direction value to obtain an intensity difference image and a gradient direction image; The intensity difference image and the gradient direction image are merged to obtain the pixel change information; The construction of the standardized three-dimensional structural information of the face includes: The three-dimensional structural values ​​of the standardized face are calculated using the following formula: in, This represents the three-dimensional structural value of the standardized human face. This represents the albedo of the face image corresponding to the standardized face. This represents the surface normal of the standardized human face, where the surface normal is represented by a 3D shape. This represents the point source of light illuminating the image of a face. Since the screens for photos and replaying videos are two-dimensional planar structures, It is a constant; Construct the three-dimensional structure vector of the stated three-dimensional structure value; The three-dimensional structure vector is converted into an image format to obtain the three-dimensional structure information; Feature extraction is performed on the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features. Feature fusion is then performed on the texture features, pixel change features, and three-dimensional structure features to obtain fused features. Based on the fusion features, the liveness detection score of the standardized face is calculated, and the liveness detection result of the face image is determined using the liveness detection score.

2. The method according to claim 1, characterized in that, The process of standardizing the detected face to obtain a standardized face includes: Obtain the facial key points corresponding to the detected face; Calculate the similarity transformation matrix between the facial key points and the preset standard facial key points; The detected face is transformed using the similarity transformation matrix to obtain the standardized face.

3. The method according to claim 1, characterized in that, The process of fusing the texture features, pixel variation features, and three-dimensional structural features to obtain fused features includes: The importance of the texture features, the importance of the pixel variation features, and the importance of the three-dimensional structural features are calculated using the following formulas: in, Indicates the importance of the texture features. Indicates the importance of the aforementioned change features. Indicates the importance of the three-dimensional features. Indicates importance, i = 1, 2, 3, This represents the query vector. This represents the texture feature. This indicates the pixel change feature. This represents the three-dimensional structural features. Indicates the transpose symbol; Based on the importance of the texture features, the importance of the variation features, and the importance of the three-dimensional features, the texture feature weights, variation feature weights, and three-dimensional feature weights of the texture features, the pixel variation features, and the three-dimensional structural features are calculated using the following formulas: in, Indicates the texture feature weights, Represents the weight of the changing feature. Represents the weights of the three-dimensional features. Indicates the importance of the texture features. Indicates the importance of the aforementioned change features. Indicates the importance of the three-dimensional features. Indicates importance, i = 1, 2, 3; The fused feature is calculated using the following formula based on the texture feature weights, the variation feature weights, and the 3D feature weights: in, The fusion feature is represented by i = 1, 2, 3. This represents the texture feature. This indicates the pixel change feature. This represents the three-dimensional structural features. Indicates the texture feature weights, Represents the weight of the changing feature. This represents the weight of the three-dimensional feature.

4. The method according to claim 1, characterized in that, The step of calculating the liveness detection score of the standardized face based on the fused features includes: The fused features are linearly fitted using the following formula to obtain the fitted features: in, This represents the fitted features. This indicates the fusion feature. These represent the bias parameters of the model used for liveness detection. These represent the weight parameters of the model performing liveness detection; Based on the fitted features, the liveness detection score is calculated using the following formula: in, This represents the liveness detection score. This represents the fitted feature.

5. A face liveness detection device for motion scenarios, characterized in that, The device includes: The face standardization module is used to acquire face images in motion scenarios, perform face detection on the face images to obtain detected faces, and perform format standardization on the detected faces to obtain standardized faces. A 3D construction module is used to perform texture encoding on the standardized face to obtain the encoded texture of the standardized face, extract texture information from the standardized face based on the encoded texture, calculate pixel change information in the standardized face, and construct the 3D structural information of the standardized face. The step of performing texture encoding on the standardized face to obtain the encoded texture of the standardized face includes: The standardized face is divided into regions to obtain the divided face regions; The pixel texture values ​​in the segmented face region are calculated using the following formula: ； ； in, This represents the pixel texture value. This represents the coordinates of the pixels in the defined face region. express The number of neighboring pixels of a pixel express The grayscale value of a pixel. Represents the grayscale value of neighboring pixels. Represents a symbolic function; The pixel texture values ​​are statistically analyzed to obtain the statistical frequency of the pixel texture values. The statistical frequencies are normalized to obtain normalized frequencies; The normalized frequency is vector-encoded to obtain the encoded texture; The process of extracting texture information from the standardized face based on the encoded texture is achieved by using a convolutional neural network to extract features from the encoded texture. The calculation of pixel change information in the standardized face includes: The pixel intensity difference value in the standardized face is calculated using the following formula: in, This represents the pixel intensity difference value. This represents the total number of neighborhood points of any randomly selected pixel in the standardized face. , This represents the grayscale value of any pixel selected arbitrarily within the standardized face. express The gray values ​​of the neighboring pixels, where j represents the index of the neighboring pixels; Calculate the pixel gradient direction values ​​in the standardized face; Perform histogram transformation on the pixel intensity difference value and the pixel gradient direction value to obtain an intensity difference image and a gradient direction image; The intensity difference image and the gradient direction image are merged to obtain the pixel change information; The construction of the standardized three-dimensional structural information of the face includes: The three-dimensional structural values ​​of the standardized face are calculated using the following formula: in, This represents the three-dimensional structural value of the standardized human face. This represents the albedo of the face image corresponding to the standardized face. This represents the surface normal of the standardized human face, where the surface normal is represented by a 3D shape. This represents the point source of light illuminating the image of a face. Since the screens for photos and replaying videos are two-dimensional planar structures, It is a constant; Construct the three-dimensional structure vector of the stated three-dimensional structure value; The three-dimensional structure vector is converted into an image format to obtain the three-dimensional structure information; The feature fusion module is used to extract features from the texture information, the pixel change information, and the three-dimensional structure information respectively to obtain texture features, pixel change features, and three-dimensional structure features, and to fuse the texture features, pixel change features, and three-dimensional structure features to obtain fused features; The result determination module is used to calculate the liveness detection score of the standardized face based on the fusion features, and use the liveness detection score to determine the face liveness detection result of the face image.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the face liveness detection method in a motion state scenario as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the face liveness detection method in motion scenarios as described in any one of claims 1 to 4.

Images