A face recognition processing method, device and equipment based on machine dimension

Abstract

The embodiments of this specification disclose a method, device and equipment for face recognition processing based on machine dimensions. The scheme includes: obtaining a scene image of the scene in which the machine is located, which is pre-collected and retained by the machine, and extracting the global features of the scene image; obtaining a face image collected by the machine when the user performs face recognition, and extracting the global features and local features of the face image, wherein the local features include the features of the face area; performing feature comparison based on the global features of the scene image, the global features and local features of the face image, and obtaining a comparison result; performing fusion processing based on the global features and local features of the face image, and the comparison result, using a pre-trained live anti-attack model; judging whether the face in the face image is live based on the result of the fusion processing.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face recognition processing method, device and equipment based on machine dimension. Background Art

[0002] Face recognition technology has developed rapidly in recent years and has been widely used in scenarios such as payment, attendance, and authentication. Since face images are easy to obtain, many attackers use stolen user face images to perform live attacks on face recognition systems. For example, a common method of using a mobile phone screen to display and print photos in an attempt to deceive a face recognition system is a live attack.

[0003] In order to defend against liveness attacks, face recognition systems generally integrate liveness detection technology (which can be called liveness anti-attack technology in this scenario). Currently, the most common liveness anti-attack technology is based on interactive liveness detection, which requires the user to cooperate and perform some specified actions, such as instructing the user to blink, open his mouth, shake his head, etc.

[0004] Based on this, for face recognition, a live anti-attack solution with better user experience and greater security is also needed. Summary of the invention

[0005] One or more embodiments of the present specification provide a face recognition processing method, device, equipment and storage medium based on machine dimensions to solve the following technical problems: For face recognition, a live anti-attack solution with better user experience and greater security is also needed.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] One or more embodiments of this specification provide a face recognition processing method based on machine dimension, including:

[0008] Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image;

[0009] Acquire a facial image collected by the machine when the user performs facial recognition, and extract global features and local features of the facial image, wherein the local features include features of the facial area;

[0010] Performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0011] According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing;

[0012] According to the result of the fusion processing, it is determined whether the face in the face image is alive.

[0013] One or more embodiments of this specification provide a face recognition processing device based on machine dimension, including:

[0014] A scene storage module is used to obtain a scene image of the scene in which the machine is located, which is collected and stored in advance by the machine, and to extract global features of the scene image;

[0015] A face processing module, which obtains a face image collected by the machine when the user performs face recognition, and extracts global features and local features of the face image, wherein the local features include features of the face area;

[0016] A feature comparison module performs feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0017] A fusion processing module, which performs fusion processing using a pre-trained live anti-attack model according to the global features and local features of the face image and the comparison result;

[0018] The living body judgment module judges whether the human face in the face image is living according to the result of the fusion processing.

[0019] One or more embodiments of this specification provide a face recognition processing device based on machine dimension, including:

[0020] at least one processor; and,

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0023] Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image;

[0024] Acquire a facial image collected by the machine when the user performs facial recognition, and extract global features and local features of the facial image, wherein the local features include features of the facial area;

[0025] Performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0026] According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing;

[0027] According to the result of the fusion processing, it is determined whether the face in the face image is alive.

[0028] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to:

[0029] Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image;

[0030] Acquire a facial image collected by the machine when the user performs facial recognition, and extract global features and local features of the facial image, wherein the local features include features of the facial area;

[0031] Performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0032] According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing;

[0033] According to the result of the fusion processing, it is determined whether the face in the face image is alive.

[0034] At least one of the above-mentioned technical solutions adopted in one or more embodiments of the present specification can achieve the following beneficial effects: utilizing the scene information of the machine dimension, through the global and local feature comparison learning technology, the scene changes caused by the liveness attack are detected and sensitive to it, and the risk judgment of face recognition is made based on this, and the result of the risk judgment is integrated into the classification-based liveness detection process to obtain the final judgment result, thereby effectively improving the security of liveness detection, and having a good recall for high-difficulty liveness attacks, which are not easily bypassed, and do not require additional cooperation from the user, which helps to improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1A conceptual diagram of a face recognition processing solution based on machine dimensions provided for one or more embodiments of this specification;

[0037] Figure 2 A schematic flow chart of a face recognition processing method based on machine dimension provided for one or more embodiments of this specification;

[0038] Figure 3 A schematic diagram of the structure of a face recognition processing device based on machine dimension provided for one or more embodiments of this specification;

[0039] Figure 4 A schematic diagram of the structure of a face recognition processing device based on machine dimensions provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0040] The embodiments of this specification provide a face recognition processing method, device, equipment and storage medium based on machine dimension.

[0041] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0042] The common liveness protection technology mentioned in the background technology is based on interactive liveness detection. However, since this type of solution requires the user to cooperate in performing specified actions, it takes longer time. Moreover, if the user's actions are not standardized, they will be required to retry or directly judged as authentication failure, so the user experience is poor; on the other hand, this type of solution may also be bypassed by using simple action generation technology, so the security capability is limited.

[0043] The following embodiments provide corresponding solutions to the above problems, which are referred to as this solution. In order to better illustrate the concept of this solution, the following analysis is conducted.

[0044] From the perspective of a facial recognition machine (for example, an automatic face-scanning vending machine, a smart phone, etc., referred to as the machine in the embodiment), the object to be recognized will face the camera of the machine so that the machine can collect the facial image. The facial image collected by the machine includes a foreground part and a background part. The foreground part is the object to be recognized (usually the face area, and may also include other body areas below the head), and the background part is the scene where the machine and the object to be recognized are currently located, such as a shopping mall, a store, a bank lobby, a user's home, a company workstation, around a gate access control, etc.

[0045] For the current common liveness attacks on face recognition, for example, the attacker will use a mobile phone screen (called a screen attack) or a printed photo (called a photo attack) containing the face area of ​​a normal user (the person being impersonated) to impersonate the live face of the normal user, and show it to the camera of the machine in an attempt to deceive the machine.

[0046] Compared to the normal face recognition of the normal user facing the camera of the machine, this type of attack often changes or affects the scene in the camera field of view of the machine more obviously and drastically. For example, the scene may be more or completely covered, leaving only the mobile phone photo or printed photo to basically fill the camera field of view. Not only that, because the shape of the photo is often inconsistent with the real person, and lacks a natural sense of dynamics, especially the edge area of ​​the face, the transition between the foreground and the scene will be more unnatural. Based on these situations, this solution wants to make full use of the scene information of the machine dimension (currently mostly using the information of the dimension of the object to be identified itself), so that the machine can pay attention to the scene in a targeted manner, and keep the corresponding scene information, and apply it to the subsequent recognition process, and perform more detailed global and local feature recognition to improve recognition accuracy and security.

[0047] Figure 1 A schematic diagram of the concept of a face recognition processing solution based on machine dimensions provided for one or more embodiments of this specification. The concept mainly includes the following four parts, which are briefly introduced as examples.

[0048] The first part is the collection and retention of scene information from the machine dimension. For example, for each face recognition machine, regular scene information collection is performed, such as collecting the scene image it is currently facing as a reference for subsequent comparison and learning. Of course, for the scene information of the machine dimension, in addition to the scene information in the form of images, other forms of information can also be used as auxiliary. For example, ambient noise, machine location data, machine usage distribution data (such as frequency of use, peak and valley data, etc.), and so on. The following embodiments mainly describe the solution using scene images as an example.

[0049] The second part is image acquisition and preprocessing. The object to be identified (normal user or live impersonator or live impersonator, etc.) is placed in front of the camera of the machine. After face recognition begins, face image acquisition and preprocessing are attempted. Preprocessing is performed according to actual needs, such as selecting more suitable images, enlarging the face area, image denoising, and image enhancement.

[0050] The third part is global and local feature comparison. Based on the newly proposed solution, the face images collected during recognition and the backup images of the scene are compared globally and locally to measure the risk and clues of live attack.

[0051] Part 4: Liveness judgment based on fusion processing. Global and local features and comparison results are fused with classification-based model features to perform liveness judgment.

[0052] Based on this concept, the following is further described in detail.

[0053] Figure 2 A flowchart of a face recognition processing method based on the machine dimension provided for one or more embodiments of this specification. The method can be applied to face recognition processes in different business fields to defend against live face attacks, such as financial business fields, e-commerce business fields, instant messaging business fields, game business fields, official business fields, etc. The process can be executed by computing devices in the corresponding fields (such as smart phones, public face recognition machines, risk control servers corresponding to payment services, etc.), and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0054] Figure 2 The process in may include the following steps:

[0055] S202: Acquire a scene image of the scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image.

[0056] The machine has a camera, and the user needs to face the camera when performing face recognition so that the machine can collect face images. In addition to collecting face images in normal work, the machine can also actively collect scene images of the scene in which it is located at an appropriate time. In particular, some scenes that are relatively consistent with the scene in which the user is when scanning the face can be collected. For machines with relatively fixed positions (for example, automatic face scanning vending machines), the surrounding scenery generally does not change much. In this case, the scene factors such as time and light can be considered to be as consistent as possible. For example, the automatic face scanning vending machine is generally used more by users in the evening, so the scene images of the evening period can be collected. For machines with easily changing positions (for example, smart phones), the surrounding scenery often changes dynamically, so some typical scenes of the user (where face recognition may be used) can be collected. Images, such as home, company cafeteria, and shops that have been visited many times, etc.

[0057] In one or more embodiments of the present specification, the machine collects scene images in advance during idle time. For example, when it is not performing face recognition, it collects scene images multiple times (for example, once every hour, 10 images at a time, etc.). Each time it collects, it randomly selects a backup image from the scene images collected this time and retains it. The backup image can be updated or increased in a timely manner later.

[0058] Furthermore, when collecting scene images, it is possible to accidentally capture faces of people around you. These faces may not necessarily be for face recognition, but are just passers-by. In order to protect the privacy of passers-by, face detection can be performed on the collected scene images, and scene images with detected faces can be discarded, and scene images without detected faces can be retained to avoid privacy issues. In addition, for some scenes in non-public areas, privacy protection of the scenes can also be performed. For example, for scene images of home or workplace, the framework in the scene (such as room configuration, furniture outlines, etc.) can be extracted and retained, and the details (such as small objects, furniture wall textures, etc.) can be replaced with some general maps.

[0059] In one or more embodiments of the present specification, the global features of the scene image reflect the global features of the corresponding scene and can be used to determine the similarity of the scenes.

[0060] S204: Obtaining a facial image collected by the machine when the user performs facial recognition, and extracting global features and local features of the facial image, wherein the local features include features of the facial area.

[0061] In one or more embodiments of the present specification, the user confirms the start of face recognition by, for example, clicking on the machine screen, and the machine starts trying to collect face images. It can perform face detection on the collected images. If a face is detected, the image with the face is saved and the collection is terminated. The machine can also perform human body detection on the collected images (for example, detecting whether a human body part or a human body part other than a face exists, or whether the human body part is coordinated with the face), and save the results of human body detection, which is also helpful for live body attack prevention, because, in some live body attacks, the attacker may simply simulate a human face, but may not simulate other human body parts accordingly, for example, facing the machine camera with a face photo.

[0062] The face image mentioned in step S204, in addition to the face area, generally also includes a background area (in the case of a live attack, or when a normal user is too close to the machine-based camera, the background area may not exist), and the background area reflects the scene.

[0063] In one or more embodiments of the present specification, a face region can be determined in a face image, and then features can be extracted from the face region as local features of the face image. It should be noted that when a normal user is identified, the face region is live (i.e., the face of the normal user), while under a live attack, the face region is often a non-live impersonation of the face of a normal user. For ease of description, they are collectively referred to as face regions. Features other than local features of a face image are mainly background features, which can more specifically and accurately reflect the scene in which the identified object was located when the face image was collected.

[0064] In one or more embodiments of the present specification, the local features of the facial image may include other foreground features in addition to the features of the facial area. Since non-living impostors also belong to the foreground, such local features help to more accurately perceive the non-living impostors in the future and help to more accurately determine whether the scene changes caused by facial recognition are normal.

[0065] S206: Perform feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result.

[0066] In one or more embodiments of the present specification, the features of the scene image can be compared with the features of the face image to determine the change of the scene. According to the previous description, the local features of the face image can more accurately reflect the situation of the foreground, and the foreground can directly affect (at least including occlusion) the change of the scene. Based on this, the three features in step S206 are used as the basis for a more detailed and hierarchical feature comparison. The comparison results can more accurately reflect the risk of live attack.

[0067] Through feature comparison, we can directly obtain the global and local similarities. If the similarity is low, it means that the risk of live attack is high, which is generally caused by photo attack or screen attack, which causes a significant change in the image. In practical applications, attackers can also use injection attacks to carry out live attacks. This attack method does not deceive the camera, but directly injects illegal data (for example, fake images) into the machine or its backend, which can completely change the image actually collected by the machine. The above scene-based similarity calculation also has a good protective effect against injection attacks.

[0068] S208: Perform fusion processing using a pre-trained liveness anti-attack model based on the global features and local features of the face image and the comparison result.

[0069] In one or more embodiments of this specification, a live anti-attack model is constructed based on a classifier (for example, a binary classification of live or non-live, and non-live is considered to be an attack), and a large amount of labeled attack and live data is used in advance to train the model. Of course, this alone is not enough, and the security capabilities for some higher-precision attacks such as using high-definition screens and high-precision printing paper may not meet expectations. Therefore, global features, local features, and comparison results are further integrated into the model to assist the model in more accurately identifying attacks.

[0070] S210: judging whether the human face in the facial image is alive according to the result of the fusion processing.

[0071] In one or more embodiments of the present specification, a final prediction is made based on the result of the fusion processing to determine whether the facial image is a live attack. Under a live attack, the face in the facial image can be considered as non-live, otherwise the face can be considered as live.

[0072] pass Figure 2 The method utilizes the scene information of the machine dimension, and through the global and local feature comparison learning technology, it detects the scene changes caused by liveness attacks and is sensitive to them, and makes risk judgments on face recognition based on this. The result of risk judgment is integrated into the classification-based liveness detection process to obtain the final judgment result, thereby effectively improving the security of liveness detection, and having a good recall for high-difficulty liveness attacks, which is not easily bypassed and does not require additional cooperation from users, which helps to improve user experience.

[0073] based on Figure 2 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0074] The comparison of global features and local features, as well as their fusion during the classification process, are the key to improving the security of this solution, which will be further explained in detail below.

[0075] In one or more embodiments of the present specification, as a basis, when performing feature comparison, similarity calculation can be performed based on the global features of the scene image and the global features of the face image, and the comparison result can be determined based on the result of the similarity calculation. In this process, when the security requirements are relatively low, the similarity between the global features of the scene image and the global features of the face image can be directly calculated as the comparison result. A live attack may cause a large change in the scene, and the similarity will become smaller accordingly, thereby measuring the risk of a live attack to a certain extent. However, in actual applications, when the face occupies a large area in the face image, the similarity may be low regardless of whether it is a live attack. Taking this into account, the local features of the face image are further used to reduce the impact of the face itself on the similarity calculation, so as to more focusedly identify the changes in the scene.

[0076] For example, according to the local features of the face image, the corresponding features in the global features of the scene image and the local features of the face image are generalized, and the similarity is calculated according to the global features of the scene image and the local features of the face image after the generalization. In this case, assuming that the local features are the features of the face area, the features of the area in the scene image that corresponds to the position of the face area can be determined according to the local features (i.e., the corresponding features mentioned above). The features belong to the scene features in the scene image. Originally, whether it is an attack or not, the scene features and the local features are obviously different. By generalizing the scene features and the local features, the difference can be reduced or even eliminated. Then, the similarity between the global features of the scene image and the global features of the face image after the generalization can be calculated, thereby helping to more reasonably measure the changes in the scene. The generalization method is not specifically limited. For example, the corresponding features can be directly eliminated, or blurred, or replaced with other similar or identical features.

[0077] Furthermore, in current liveness attacks, the edges of the foreground (for example, the edges of a square screen or photo) are often very different from the normal situation (the edges of a living face). Even if the contours are more similar, such as using a face mask for liveness attacks, the difference may be reduced, but the difference still exists, for example, the hair area, the elastic dynamics of the edge, the connection between the mask and other human body areas, etc.

[0078] In order to identify such differences more accurately, during the feature comparison process, special processing can be performed on edge features and special attention can be paid to them.

[0079] For example, the global features and local features of the face image are compared to obtain an intermediate result of the comparison. If the similarity reflected by the intermediate result of the comparison (the similarity between the global feature and the local feature) is greater than the first set threshold (it can be considered that at this time, the proportion of the face area in the face image is large enough, and the impact on the global situation is large enough, then the impact of the scene is relatively small, and it is necessary to pay more attention to the scene area more accurately), then according to the local features of the face image, the actual face edge scene features are extracted from the global features of the face image (based on the edge scene features to more accurately pay attention to the scene area), and the edge scene features of the imaginary face (there is actually no face in the scene image, but if there is a face, it should normally be in this area) are extracted from the global features of the scene image (that is, the edge scene features of the area corresponding to the position of the face area mentioned above), and the actual face edge scene features are compared with the imaginary face edge scene features to obtain a comparison result. If the similarity of the edge scene features of the two is smaller, it reflects that the risk of live attack is greater.

[0080] Furthermore, in addition to the similarity, in order to facilitate fusion processing in the model, the comparison result includes, for example, a feature difference set between the actual face edge scene features and the imaginary face edge scene features (which finely reflects the changes in the scene, especially the edge scene). In this case, the above-mentioned fusion processing includes, for example: combining the global features and local features of the face image, as well as the feature difference set (for example, connecting the corresponding vectors in a set order), and then inputting them into the pre-trained live anti-attack model for fusion processing in order to perform classification prediction.

[0081] In addition, if the similarity reflected by the above-mentioned intermediate comparison result is less than the second set threshold (it can be considered that at this time, the proportion of the face area in the face image is small enough, and the impact on the global situation is small enough, then the impact of the scene is relatively large, and the scene area can be paid more attention to in a macro sense), then the global features of the scene image can be directly compared with the global features of the face image to obtain the comparison result. The second set threshold is less than (or even may be much less than) the first set threshold. For example, the first set threshold is set to 80%, the second set threshold is set to 20%, and so on.

[0082] In one or more embodiments of the present specification, feature extraction and feature comparison may be performed by a corresponding feature processing model, and the feature processing model may be an independent model or integrated into a living body anti-attack model.

[0083] For example, the feature processing model includes two feature extraction parts. The first part is a global feature extraction sub-model, denoted as M1, and the second part is a local feature extraction sub-model, denoted as M2. The features extracted by M1 and M2 are connected in series and processed by network structures such as a fully connected layer to achieve feature comparison, and finally the comparison result is output (for example, expressed as a risk score, which reflects the high risk of a live attack, but it is not the final result, but only serves as one of the reference data). The available structures of M1 and M2 are diverse, and no specific limitation is made here, such as using ResNet18.

[0084] Some specific implementation methods of feature comparison are listed above. Taking the simple case of global feature comparison as an example, other cases can be implemented similarly. Human body detection can be performed on the face image, and the detected human body area (at least including the face area) can be cropped out as the input of M2, and the original entire face image can be used as the input of M1 for processing by the feature processing model and the live anti-attack model. Among them, when calculating the risk score, the global features of the face image collected during the current face recognition (denoted as f1) and the retained global features (denoted as f2, which can be obtained by inputting the retained scene image into M1) can be used to calculate the risk score (denoted as s) according to the following formula 1, where<x,y> Indicates calculating the similarity between x and y, for example, using cosine similarity.

[0085] s=1.0-<f1,f2> ; (Formula 1)

[0086] In one or more embodiments of the present specification, the comparison result includes some image features, such as the above-mentioned feature difference set. In this case, the comparison result can be fused with the global features and local features of the face image, and then prediction can be performed. For example, the output features of the pre-trained live classification model for the face image (which can be the features of the hidden layer output) are obtained, the global features and local features of the face image (which can also include the global features of the scene image), and the output features are first fused, and input into the pre-trained live anti-attack model, and then classification prediction is performed to obtain reference classification data, and the comparison result and the reference classification data are fused for the second time through the live anti-attack model.

[0087] Based on this idea, it is assumed that the comparison result includes a normalized risk score reflecting the similarity between the scene image and the part other than the face area in the face image, such as the above s, and the reference classification data includes the classification probability of classifying the face in the face image as alive or not alive. The above feature processing model and liveness classification model are integrated into the liveness anti-attack model, which also includes a fusion model. The fusion model includes multiple fully connected layers and is pre-trained based on a binary classification loss function.

[0088] The initialization model (such as ResNet18) can also be trained for binary classification to obtain a live classification model. The features of the face image are input into the live classification model for classification to obtain the above-mentioned output features. The outputs of M1 and M2 and the output features are input into the feature fusion model to output the classification probability (denoted as P) as the reference classification data. Next, P and s are fused. When both P and s are in normalized form, P and s are superimposed to obtain the judgment score. Then, based on the judgment score and the corresponding set threshold, it is determined whether the face in the face image is live or whether it has been attacked by a live body. For example, assuming that the value of s is positively correlated with the risk of live attacks, P specifically represents the probability of classification as non-live. If the judgment score is greater than the corresponding set threshold (denoted as T), that is, when P+s>T, it can be determined that the face in the face image is non-live and has been attacked by a live body.

[0089] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 3 , Figure 4 shown.

[0090] Figure 3 A schematic diagram of a face recognition processing device based on machine dimension provided for one or more embodiments of this specification, the device comprising:

[0091] The scene record module 302 acquires a scene image of the scene in which the machine is located, which is collected and stored in advance by the machine, and extracts global features of the scene image;

[0092] The face processing module 304 acquires the face image collected by the machine when the user performs face recognition, and extracts the global features and local features of the face image, wherein the local features belong to the features of the face area;

[0093] A feature comparison module 306 performs feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0094] A fusion processing module 308 performs fusion processing using a pre-trained live anti-attack model according to the global features and local features of the face image and the comparison result;

[0095] The living body determination module 310 determines whether the human face in the face image is living according to the result of the fusion processing.

[0096] Optionally, the scene record module 302 collects scene images of the scene where the machine is located multiple times through the machine when the machine is not performing face recognition;

[0097] Perform face detection on each of the scene images, and retain the scene images in which no face is detected.

[0098] Optionally, the feature comparison module 306 performs similarity calculation based on the global features of the scene image and the global features of the face image;

[0099] The comparison result is determined according to the result of the similarity calculation.

[0100] Optionally, the feature comparison module 306 generalizes corresponding features in the global features of the scene image and the global features of the face image according to the local features of the face image;

[0101] Similarity calculation is performed based on the global features of the scene image and the global features of the face image after the generalization processing.

[0102] Optionally, the feature comparison module 306 performs feature comparison on the global features and local features of the face image to obtain an intermediate comparison result;

[0103] If the similarity reflected by the intermediate comparison result is greater than a first set threshold, extracting actual face edge scene features from the global features of the face image and extracting imaginary face edge scene features from the global features of the scene image based on the local features of the face image;

[0104] The actual face edge scene feature is compared with the imaginary face edge scene feature to obtain a comparison result.

[0105] Optionally, the feature comparison module 306 performs a feature comparison on the global features of the facial image with the local features to obtain an intermediate comparison result. If the similarity reflected by the intermediate comparison result is less than a second set threshold, the global features of the scene image are compared with the global features of the facial image to obtain a comparison result, and the second set threshold is less than the first set threshold.

[0106] Optionally, the comparison result includes a feature difference set between the actual face edge scene feature and the imaginary face edge scene feature;

[0107] The fusion processing module 308 connects the global features and local features of the face image and the feature difference set, and inputs them into a pre-trained liveness anti-attack model for fusion processing so as to perform classification prediction.

[0108] Optionally, the fusion processing module 308 obtains a classification result of the face image by a pre-trained living body classification model;

[0109] The global features and local features of the face image and the classification result are fused and input into a pre-trained live attack prevention model to perform classification prediction to obtain reference classification data;

[0110] The comparison result and the reference classification data are fused through the living body anti-attack model.

[0111] Optionally, the comparison result includes a normalized risk score reflecting the similarity between the scene image and the portion outside the face region in the face image; the reference classification data includes a classification probability of classifying the face in the face image as a living body or a non-living body;

[0112] The fusion processing module 308 superimposes the normalized risk score and the classification probability to obtain a decision score, so as to determine whether the face in the face image is alive based on the decision score and a corresponding set threshold.

[0113] Figure 4 A schematic diagram of a face recognition processing device based on machine dimension provided for one or more embodiments of this specification, the device comprising:

[0114] at least one processor; and,

[0115] a memory communicatively connected to the at least one processor; wherein,

[0116] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0117] Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image;

[0118] Acquire a face image collected by the machine when the user performs face recognition, and extract global features and local features of the face image, wherein the local features belong to features of the face area;

[0119] Performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0120] According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing;

[0121] According to the result of the fusion processing, it is determined whether the face in the face image is alive.

[0122] The processor and the memory may communicate with each other via a bus, and the device may further include an input / output interface for communicating with other devices.

[0123] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method, storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0124] Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image;

[0125] Acquire a face image collected by the machine when the user performs face recognition, and extract global features and local features of the face image, wherein the local features belong to features of the face area;

[0126] Performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result;

[0127] According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing;

[0128] According to the result of the fusion processing, it is determined whether the face in the face image is alive.

[0129] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0130] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0131] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0132] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0133] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0134] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0138] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0139] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0141] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0142] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A face recognition processing method based on machine dimension, comprising: Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image; Acquire a facial image collected by the machine when the user performs facial recognition, and extract global features and local features of the facial image, wherein the local features include features of the facial area; Performing feature comparison according to the global features of the scene image and the global features and local features of the face image to obtain a comparison result, including: comparing the features of the scene image with the features of the face image to determine the change of the scene, performing similarity calculation according to the global features of the scene image and the global features of the face image, and determining the comparison result according to the result of the similarity calculation; According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing; According to the result of the fusion processing, it is determined whether the face in the face image is alive.

2. The method according to claim 1, wherein the machine collects a scene image of the scene in which the machine is located in advance, specifically comprising: When the machine is not performing face recognition, the machine collects scene images of the scene where the machine is located multiple times; Perform face detection on each of the scene images, and retain the scene images in which no face is detected.

3. The method according to claim 1, wherein the similarity calculation based on the global features of the scene image and the global features of the face image specifically comprises: According to the local features of the face image, generalizing the corresponding features in the global features of the scene image and the global features of the face image; Similarity calculation is performed based on the global features of the scene image and the global features of the face image after the generalization processing.

4. The method according to claim 1, wherein the step of performing feature comparison based on the global features of the scene image and the global features and local features of the face image to obtain a comparison result specifically comprises: Performing feature comparison on the global features and local features of the face image to obtain an intermediate comparison result; If the similarity reflected by the intermediate comparison result is greater than a first set threshold, extracting actual face edge scene features from the global features of the face image and extracting imaginary face edge scene features from the global features of the scene image based on the local features of the face image; The actual face edge scene feature is compared with the imaginary face edge scene feature to obtain a comparison result.

5. The method according to claim 4, wherein after comparing the global features and the local features of the face image to obtain an intermediate comparison result, the method further comprises: If the similarity reflected by the intermediate comparison result is less than a second set threshold, a feature comparison is performed on the global features of the scene image and the global features of the face image to obtain a comparison result, and the second set threshold is less than the first set threshold.

6. The method according to claim 4, wherein the comparison result comprises a feature difference set between the actual face edge scene feature and the imaginary face edge scene feature; The method of performing fusion processing using a pre-trained live anti-attack model based on the global features and local features of the face image and the comparison result specifically includes: After the global features and local features of the face image and the feature difference set are combined, they are input into a pre-trained live anti-attack model for fusion processing so as to perform classification prediction.

7. The method according to claim 1, wherein the fusion processing is performed using a pre-trained live anti-attack model based on the global features and local features of the face image and the comparison result, specifically comprising: Obtaining output features of a pre-trained living body classification model for the face image; The global features and local features of the face image and the output features are fused and input into a pre-trained live attack prevention model to perform classification prediction to obtain reference classification data; The comparison result and the reference classification data are fused through the living body anti-attack model.

8. The method of claim 7, wherein the comparison result comprises a normalized risk score reflecting the similarity between the scene image and the portion other than the face region in the face image; the reference classification data comprises a classification probability of classifying the face in the face image as a living object or a non-living object; The fusing processing of the comparison result and the reference classification data specifically includes: The normalized risk score and the classification probability are superimposed to obtain a decision score, so as to determine whether the face in the face image is alive according to the decision score and a corresponding set threshold.

9. A face recognition processing device based on machine dimension, comprising: A scene storage module is used to obtain a scene image of the scene in which the machine is located, which is collected and stored in advance by the machine, and to extract global features of the scene image; A face processing module, which obtains a face image collected by the machine when the user performs face recognition, and extracts global features and local features of the face image, wherein the local features include features of the face area; A feature comparison module performs feature comparison according to the global features of the scene image and the global features and local features of the face image to obtain a comparison result, including: comparing the features of the scene image with the features of the face image to determine the change of the scene, performing similarity calculation according to the global features of the scene image and the global features of the face image, and determining the comparison result according to the result of the similarity calculation; A fusion processing module, which performs fusion processing using a pre-trained live anti-attack model according to the global features and local features of the face image and the comparison result; The living body judgment module judges whether the human face in the face image is living according to the result of the fusion processing.

10. The device according to claim 9, wherein the scene record module collects scene images of the scene where the machine is located multiple times through the machine when the machine is not performing face recognition; Perform face detection on each of the scene images, and retain the scene images in which no face is detected.

11. The device according to claim 9, wherein the feature comparison module generalizes the corresponding features in the global features of the scene image and the global features of the face image according to the local features of the face image; Similarity calculation is performed based on the global features of the scene image and the global features of the face image after the generalization processing.

12. The device according to claim 9, wherein the feature comparison module compares the global features and local features of the face image to obtain an intermediate comparison result; If the similarity reflected by the intermediate comparison result is greater than a first set threshold, extracting actual face edge scene features from the global features of the face image and extracting imaginary face edge scene features from the global features of the scene image based on the local features of the face image; The actual face edge scene feature is compared with the imaginary face edge scene feature to obtain a comparison result.

13. In the device as claimed in claim 12, the feature comparison module performs feature comparison on the global features of the scene image and the local features of the facial image to obtain an intermediate comparison result. If the similarity reflected by the intermediate comparison result is less than a second set threshold, the global features of the scene image and the global features of the facial image are compared to obtain a comparison result, and the second set threshold is less than the first set threshold.

14. The device of claim 12, wherein the comparison result comprises a feature difference set between the actual face edge scene feature and the imaginary face edge scene feature; The fusion processing module combines the global features and local features of the face image and the feature difference set, and inputs them into a pre-trained live anti-attack model for fusion processing so as to perform classification prediction.

15. The device according to claim 9, wherein the fusion processing module obtains output features of a pre-trained living body classification model for the face image; The global features and local features of the face image and the output features are fused and input into a pre-trained live attack prevention model to perform classification prediction to obtain reference classification data; The comparison result and the reference classification data are fused through the living body anti-attack model.

16. The apparatus of claim 15, wherein the comparison result comprises a normalized risk score reflecting the similarity between the scene image and the portion other than the face region in the face image; the reference classification data comprises a classification probability of classifying the face in the face image as a living object or a non-living object; The fusion processing module superimposes the normalized risk score and the classification probability to obtain a decision score, so as to determine whether the face in the face image is alive based on the decision score and a corresponding set threshold.

17. A face recognition processing device based on machine dimension, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire a scene image of a scene in which the machine is located, which is collected and retained in advance by the machine, and extract global features of the scene image; Acquire a facial image collected by the machine when the user performs facial recognition, and extract global features and local features of the facial image, wherein the local features include features of the facial area; Performing feature comparison according to the global features of the scene image and the global features and local features of the face image to obtain a comparison result, including: comparing the features of the scene image with the features of the face image to determine the change of the scene, performing similarity calculation according to the global features of the scene image and the global features of the face image, and determining the comparison result according to the result of the similarity calculation; According to the global features and local features of the face image and the comparison result, a pre-trained liveness anti-attack model is used for fusion processing; According to the result of the fusion processing, it is determined whether the face in the face image is alive.

Images