Face authenticity detection method, apparatus, and computer readable medium

Abstract

The application provides a face authenticity detection method and device and a computer readable medium, the method comprising: acquiring a face detection image and depth data, and performing front-end processing on the face detection image and the depth data to obtain secondary adjustment values of the face detection image and the depth data; processing the secondary adjustment values of the face detection image and the depth data through a face feature extraction algorithm to respectively extract face image feature data and face depth feature data; performing a feature fusion operation on the face image feature data and the face depth feature data to obtain face feature fusion data; inputting the face feature fusion data into a face authenticity detection model to obtain a face authenticity detection judgment result; and the face authenticity detection model being configured to: classify the face feature fusion data through a linear classification algorithm to obtain a face authenticity detection evaluation value, and obtain the face authenticity judgment result according to the face authenticity detection evaluation value.

Description

Technical Field

[0001] This invention relates primarily to the field of face recognition, and more particularly to a method, apparatus, and computer-readable medium for detecting the authenticity of a face. Background Technology

[0002] With the rapid development of artificial intelligence and deep learning technologies, deep learning has been widely applied in facial recognition, and facial recognition technology has been used in many commercial scenarios, including facial payment, membership recognition, employee attendance, and identity verification. In these major facial recognition applications, face detection (or liveness detection) is extremely important. Face detection (or liveness detection) refers to distinguishing between real faces and fake faces, effectively preventing false recognition by the facial recognition system due to fake face attacks. However, some current face detection (or liveness detection) methods have certain limitations and cannot achieve very high recognition performance. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, apparatus and computer-readable medium for face authenticity detection, so as to achieve the requirement of high-performance face authenticity detection.

[0004] To address the aforementioned technical problems, this invention provides a method for face authenticity detection, comprising: acquiring face detection images and depth data, and performing front-end processing on the face detection images and depth data to obtain secondary adjustment values ​​for the face detection images and depth data; processing the secondary adjustment values ​​of the face detection images and depth data using a face feature extraction algorithm to extract face image feature data and face depth feature data respectively; performing feature fusion operation on the obtained face image feature data and face depth feature data to obtain face feature fusion data; inputting the face feature fusion data into a face authenticity detection model to determine a face authenticity detection result; wherein, the face authenticity detection model is configured to: classify the face feature fusion data using a linear classification algorithm to obtain a face authenticity detection evaluation value, and obtain the face authenticity judgment result based on the relationship between the face authenticity detection evaluation value and the corresponding face authenticity detection confidence threshold.

[0005] In one embodiment of the present invention, the front-end processing of the face detection image and depth data includes: scaling the face detection image to a first predetermined size to form a standard processed image; calculating the median value of the depth data of all pixels in the standard processed image; superimposing the median value with an extended range value to obtain a depth data threshold range; retaining the face depth data located within the depth data threshold range, and performing depth enhancement processing on the face depth data located outside the depth data threshold range to obtain a depth data modulation value, and obtaining a primary adjustment value for the overall face depth data; mapping the primary adjustment value for the overall face depth data to a second predetermined range to obtain a secondary adjustment value for the face detection image and depth data.

[0006] In one embodiment of the present invention, the intermediate value includes the median value or the average value.

[0007] In one embodiment of the present invention, the feature fusion operation on the face image feature data and face depth feature data includes:

[0008] The facial image feature data and facial depth feature data are concatenated according to their corresponding dimensions.

[0009] In one embodiment of the present invention, the face authenticity detection model includes a ResNet50 convolutional neural network, a VGG16 convolutional neural network, or an Inception network.

[0010] In one embodiment of the present invention, the facial feature extraction algorithm includes the RetinaFace facial detection algorithm.

[0011] In one embodiment of the present invention, the face authenticity detection model includes a convolutional neural network after model pruning.

[0012] In one embodiment of the present invention, the model pruning process includes filtering and deleting a set proportion of the feature channels of the convolutional neural network based on specific filtering parameters.

[0013] In one embodiment of the invention, the specific screening parameters include a loss function formed by the γ parameter of a model-based batch regularization layer.

[0014] In one embodiment of the present invention, the set ratio includes 30% to 50%.

[0015] In one embodiment of the present invention, face detection images and depth data are acquired by a camera device, the camera device including an image acquisition camera and a depth acquisition camera, and the depth data is calculated based on image data acquired by the image acquisition camera, depth data acquired by the depth acquisition camera, and the offset value of the depth acquisition camera relative to the image acquisition camera.

[0016] The present invention also provides a face authenticity detection device, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method as described in any of the preceding claims.

[0017] The present invention also provides a computer-readable medium storing computer program code that, when executed by a processor, implements the method as described in any of the preceding claims.

[0018] Compared with the prior art, the present invention has the following advantages: The technical solution of this application, by integrating face image detection and face depth liveness detection operations, extracts face image features and depth features, which can effectively improve the recognition accuracy of face authenticity detection and can be configured to form a good visualization effect. Attached Figure Description

[0019] The accompanying drawings are included to provide a further understanding of this application and form part of this application. The drawings illustrate embodiments of this application and, together with this specification, serve to explain the principles of this application.

[0020] In the attached image:

[0021] Figure 1 This is a flowchart of a face authenticity detection method according to an embodiment of this application.

[0022] Figure 2 This is a flowchart illustrating the steps of front-end processing of face detection images and depth data according to an embodiment of this application.

[0023] Figure 3 This is a diagram illustrating the face detection image and the secondary adjustment values ​​of depth data according to an embodiment of this application.

[0024] Figure 4 This is a schematic diagram of the composition of a face authenticity detection device according to an embodiment of this application. Detailed Implementation

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this application. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0026] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0027] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0028] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0029] Embodiments of this application describe a method, apparatus, and computer-readable medium for detecting the authenticity of a face.

[0030] Figure 1 This is a flowchart of a face authenticity detection method according to an embodiment of this application. Figure 1 As shown, the face authenticity detection method (or face liveness detection method) includes the following steps: Step 101, acquiring face detection images and depth data, and performing pre-processing on the face detection images and depth data to obtain secondary adjustment values ​​for the face detection images and depth data; Step 102, processing the secondary adjustment values ​​of the face detection images and depth data using a face feature extraction algorithm to extract face image feature data and face depth feature data respectively; Step 103, performing feature fusion operation on the obtained face image feature data and face depth feature data to obtain face feature fusion data; Step 104, inputting the face feature fusion data into the face authenticity detection model to determine the face authenticity detection result.

[0031] In some embodiments, the face authenticity detection model is configured to: classify the fused facial feature data using a linear classification algorithm to obtain a face authenticity detection evaluation value, and obtain a face authenticity judgment result based on the relationship between the face authenticity detection evaluation value and the corresponding face authenticity detection confidence threshold. For example, if the face authenticity detection confidence threshold is set to 0.5, a face authenticity detection evaluation value greater than 0.5 is judged as a live face; otherwise, it is judged as a non-live face. A non-live face is, for example, a 3D (three-dimensional) printed human head model or a 3D face mask.

[0032] In step 101, the face detection image and depth data are acquired, for example, by a camera device, which includes an image acquisition camera and a depth acquisition camera. The depth data is calculated based on the image data acquired by the image acquisition camera, the depth data acquired by the depth acquisition camera, and the offset value of the depth acquisition camera relative to the image acquisition camera.

[0033] Figure 2 This is a flowchart illustrating the steps of front-end processing of face detection images and depth data according to an embodiment of this application. In some embodiments, refer to... Figure 2 The front-end processing of face detection images and depth data includes: Step 201, scaling the face detection image to a first predetermined size to form a standard processed image; Step 202, calculating the median value of the depth data of all pixels in the standard processed image; the median value includes, for example, the median value or the average value; Step 203, superimposing the median value with an extended range value to obtain a depth data threshold range; Step 204, retaining the face depth data within the depth threshold range, and performing depth enhancement processing on the overall face depth data outside the depth data threshold range to obtain a depth data adjustment value, and obtaining a primary adjustment value for the face depth data; Step 205, mapping the primary adjustment value of the face depth data to a second predetermined range to obtain a secondary adjustment value for the face detection image and depth data.

[0034] In one specific embodiment, for example, the face detection image is scaled down to 64×64 pixels or 128×128 pixels to form a standard processed image. The median value ME of the depth data of all pixels in the standard processed image is calculated, specifically, for example, the average value. Next, the median value, i.e., the average value, is superimposed with an extended range value, such as ME±20mm, ME±30mm, or ME±40mm (millimeters), to obtain a depth data threshold range. Then, the face depth data within the depth data threshold range is retained, and the face depth data outside the depth data threshold range is subjected to depth enhancement processing to obtain a depth data adjustment value. For example, the face depth data greater than the depth extended range threshold is modified to ME+80mm, ME+85mm, or ME+90mm, and a primary adjustment value for the overall face depth data is obtained based on this. Then, the primary adjustment value of the face depth data is mapped to a second set range, such as the [0, 255] value range or the [0, 128] value range, to obtain the secondary adjustment value of the face detection image and depth data. Mapping methods may include, for example, obtaining a mapping ratio coefficient based on the numerical range before and after mapping, specifically a linear mapping ratio coefficient; then, calculating the mapped numerical value based on this mapping ratio coefficient. If decimal values ​​are present, rounding up, down, or to the nearest integer may be used. In pixel representation, the numerical range [0, 255] may correspond to a grayscale image from white to black, or an image corresponding to the RGB color space or other types of color spaces. Figure 3 This is a display diagram (or simply a display diagram of the secondary adjustment values ​​of face detection image and depth data) according to an embodiment of this application. Figure 3 This is an example of a black and white illustration. Figure 3 In the middle, the display diagram 300 corresponding to the secondary adjustment value of the face image data includes the display area 301 corresponding to the median superimposed extended range value data, and the area outside the display area 301 is the area corresponding to the adjustment value of the face depth data.

[0035] In step 102, a face feature extraction algorithm is used to process the secondary adjustment values ​​of the face detection image and depth data to extract face image feature data and face depth feature data, respectively. The face feature extraction algorithm may include, for example, the RetinaFace face detection algorithm, or other types of face detection algorithms.

[0036] Next, in step 103, a feature fusion operation is performed on the face image feature data and the face depth feature data to obtain face feature fusion data. In some embodiments, the feature fusion operation on the face image feature data and the face depth feature data includes concatenating the face image feature data and the face depth feature data in corresponding dimensions. For example, if the face image feature data is 512×1 dimension and the face depth feature data is 512×1 dimension, then the face image feature data and the face depth feature data can be concatenated in corresponding dimensions, for example, in a 1024×1 dimension or 512×2 dimension format.

[0037] In step 104, the fused facial feature data is input into the face authenticity detection model to obtain the face authenticity detection result. In some embodiments, the face authenticity detection model includes a convolutional neural network after model pruning. Model pruning includes filtering and deleting a set proportion of the feature channels of the convolutional neural network based on specific filtering parameters. A feature channel represents the number of neurons contained in the intermediate layer between the input and output layers of a neural network model. In a convolutional neural network, the number of feature channels corresponds to the number of filters in the convolutional layer, which determines the neural network model's ability to capture different types of features. Increasing the number of feature channels can improve the model's recognition accuracy, but it also increases the computational load of the neural network model. In this application, to ensure the timeliness of model operation when implementing face authenticity detection, model pruning is achieved by filtering and retaining a set proportion of the feature channels of the convolutional neural network based on specific filtering parameters. The specific filtering parameters include a loss function formed by the γ parameter (gamma parameter) of the batch regularization layer (BatchNorm layer) of the model. The expression of the batch regularization layer (BatchNorm layer) is, for example:

[0038]

[0039] Where x is the input, y is the output, and β is the shift factor. E[x] is the mean, and Var[x] is the variance.

[0040] The loss function is, for example, the L1 loss function. The specific form of the L1 loss function is, for example, L1 Loss = ∑|γ|. A set percentage of feature channels in the convolutional neural network is removed, for example, sorted from largest to smallest according to the gamma parameter (i.e., sorting the channel importance of each convolutional layer based on the gamma parameter). Feature channels corresponding to smaller gamma parameter values ​​are removed (i.e., lower-importance channels are removed), with a removal percentage, for example, between 30% and 50%, thus obtaining the pruned convolutional neural network. The pruned convolutional neural network can then be fine-tuned through further training.

[0041] In facial recognition technology applications, liveness detection is a crucial technique that effectively prevents spoofed faces from attacking facial recognition systems, thereby ensuring the security of the system. This invention provides a method for liveness detection that integrates facial image data (e.g., RGB image data) with 3D (three-dimensional) facial depth information and employs a convolutional neural model, achieving a high accuracy rate. By combining the 3D facial depth data with the face detection position results from the RGB image, the position and range of the face within the 3D depth data can be quickly located. Based on the position and range of the face within the 3D depth information, 3D facial depth data can be obtained, and a 3D facial depth image can be constructed based on the relative changes in this data, thus achieving better detection visibility.

[0042] Facial image features are extracted using a feature extraction algorithm. These features effectively determine the overall consistency of facial color, local texture, and facial feature positions, preventing attacks from 3D fake face models. Next, 3D depth image features are extracted using another feature extraction algorithm. This 3D depth information effectively determines facial depth characteristics, preventing photos or videos from impersonating real faces. Finally, RGB image features and 3D depth image features are fused through feature concatenation, and a deep learning neural network is used to determine whether the face is live. This achieves superior face authenticity detection results in various application scenarios and surrounding environments.

[0043] In summary, the technical solution of this application, by integrating face image detection and deep face liveness detection operations to extract face image features and depth features, can effectively improve the recognition accuracy of face authenticity detection (or face liveness detection) and can be configured to form a good visualization effect. The technical solution of this application performs pruning operations on the face authenticity detection model and the liveness detection model, thereby compressing the computational load during model runtime, reducing computational resource consumption, and improving the execution efficiency of the convolutional neural network model on electronic devices.

[0044] This application also provides a face authenticity detection device. Figure 4A schematic diagram of a face authenticity detection device according to an embodiment of this application is shown. The face authenticity detection device 400 may include an internal communication bus 401, a processor 402, a read-only memory (ROM) 403, a random access memory (RAM) 404, and a communication port 405. The face authenticity detection device 400 is connected to a network via the communication port and can be connected to other devices. The internal communication bus 401 enables data communication between components of the face authenticity detection device 400. The processor 402 can perform judgments and issue prompts. In some embodiments, the processor 402 may consist of one or more processors. The communication port 405 enables sending and receiving information and data from the network. The face authenticity detection device 400 may also include different forms of program storage units and data storage units, such as the read-only memory (ROM) 403 and the random access memory (RAM) 404, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor 402. The processor executes these instructions to implement the main part of the method. The results processed by the processor can be transmitted to the user device via the communication port and displayed on the user interface.

[0045] The aforementioned face authenticity detection device 400 can be implemented as a computer program, stored in a memory, and loaded into a processor 402 for execution to implement the face authenticity detection method of this application.

[0046] This application also provides a computer-readable medium storing computer program code, which, when executed by a processor, implements the above-described face authenticity detection method.

[0047] Some aspects of this application may be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a "data block," "module," "engine," "unit," "component," or "system." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. Furthermore, aspects of this application may be manifested as a computer product located on one or more computer-readable media, including computer-readable program code.

[0048] A computer-readable medium may contain a propagated data signal containing computer program encoding, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and so on, or suitable combinations thereof. The computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program encoding located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signals, or similar media, or any combination of the foregoing. It should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of the application requires more features than those mentioned in the claims. In fact, the embodiments have fewer features than all the features of the single embodiments disclosed above.

[0049] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this application are approximate values, in specific embodiments, such numerical values ​​are set as precisely as feasible.

[0050] Although this application has been described with reference to specific embodiments, those skilled in the art should recognize that the above embodiments are only used to illustrate this application, and various equivalent changes or substitutions can be made without departing from the spirit of this application. Therefore, any changes or modifications to the above embodiments within the essential spirit of this application will fall within the scope of the claims of this application.

Claims

1. A method for detecting the authenticity of a face, comprising: Acquire face detection images and depth data, and perform front-end processing on the face detection images and depth data to obtain secondary adjustment values ​​for the face detection images and depth data; The front-end processing of the face detection image and depth data includes: scaling the face detection image to a first predetermined size to form a standard processed image; calculating the median value of the depth data of all pixels in the standard processed image; superimposing the median value with an extended range value to obtain a depth data threshold range; retaining the face depth data within the depth data threshold range and performing depth enhancement processing on the face depth data outside the depth data threshold range to obtain a depth data adjustment value and a primary adjustment value for the overall face depth data; mapping the primary adjustment value for the overall face depth data to a second predetermined range to obtain a secondary adjustment value for the face detection image and depth data. The face detection image and the secondary adjustment value of the depth data are processed by the face feature extraction algorithm to extract face image feature data and face depth feature data respectively. A feature fusion operation is performed on the facial image feature data and facial depth feature data to obtain facial feature fusion data; The fused facial feature data is input into the facial authenticity detection model to obtain the facial authenticity detection result; The face authenticity detection model is configured to: classify the face feature fusion data using a linear classification algorithm to obtain a face authenticity detection evaluation value, and obtain the face authenticity judgment result based on the relationship between the face authenticity detection evaluation value and the corresponding face authenticity detection confidence threshold.

2. The face authenticity detection method according to claim 1, characterized in that, The intermediate value includes the median or average value.

3. The face authenticity detection method according to claim 1, characterized in that, The feature fusion operation on the face image feature data and face depth feature data includes: The facial image feature data and facial depth feature data are concatenated according to their corresponding dimensions.

4. The face authenticity detection method according to claim 1, characterized in that, The face authenticity detection model includes ResNet50 convolutional neural network, VGG16 convolutional neural network, or Inception network.

5. The face authenticity detection method according to claim 1, characterized in that, The facial feature extraction algorithm includes the RetinaFace face detection algorithm.

6. The face authenticity detection method according to claim 1, characterized in that, The face authenticity detection model includes a convolutional neural network that has undergone model pruning.

7. The face authenticity detection method according to claim 6, characterized in that, The model pruning process includes filtering and deleting a set proportion of the feature channels of the convolutional neural network based on filtering parameters.

8. The face authenticity detection method according to claim 7, characterized in that, The selection parameters include a loss function formed by the γ parameter of the model-based batch regularization layer.

9. The face authenticity detection method according to claim 7, characterized in that, The set ratio includes 30% to 50%.

10. The face authenticity detection method according to claim 1, characterized in that, Face detection images and depth data are acquired through a camera device, which includes an image acquisition camera and a depth acquisition camera. The depth data is calculated based on the image data acquired by the image acquisition camera, the depth data acquired by the depth acquisition camera, and the offset value of the depth acquisition camera relative to the image acquisition camera.

11. A face authenticity detection device, comprising: Memory is used to store instructions that can be executed by the processor; as well as A processor for executing the instructions to implement the method as described in any one of claims 1-10.

12. A computer-readable medium storing computer program code that, when executed by a processor, implements the method as claimed in any one of claims 1-10.

Images