Biometric feature identification method, server and client
By randomly converting and filtering biometric data bits and employing multi-check verification with negative adversarial samples, the solution strengthens biometric systems' resistance to adversarial attacks, ensuring secure authentication.
Patent Information
- Application Number
- JP2024533875
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-01-28
- Filing Date
- 2022-08-19
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2042-08-19
AI Technical Summary
Biometric systems, particularly facial recognition, are vulnerable to adversarial attacks using countervailing samples that generate incorrect authentication results, and current systems lack effective defenses against such attacks.
The solution involves randomly converting the least significant bits of biometric data, applying filtering algorithms to preprocess the data, and implementing a multi-check verification strategy with a blacklist for terminals that fail consistency checks, along with embedding and verifying negative adversarial samples to counteract potential attacks.
This approach effectively neutralizes adversarial noise, making it impossible for attackers to train effective counter-attack models and prevents unauthorized access by adding terminals to a blacklist, thus enhancing security against camera hijacking and other adversarial threats.
Smart Images

Figure 2024544215000001_ABST
Abstract
Description
[Technical field]
[0001] This application claims priority to a Chinese patent application (application number: 202210108922.9).
[0002] The present application relates to an identity identification solution, and more particularly, to a biometric feature identification method, a biometric feature identification server, a client, a computer storage medium and a computer program product. [Background technology]
[0003] In recent years, biometric features (e.g., face and fingerprint) have become the mainstream identity identification method due to their close association with an individual's identity, and have many applications in important financial transactions, such as withdrawals, payments, and bank registration via mobile phone.
[0004] Recently, the Science and Technology Department of the People's Bank of China issued a "Fintech Application Risk Statement" in which it pointed out the potential risks of facial technology applications in the field of financial payments, in response to media reports that "face recognition systems in multiple Android mobile phones have been breached." Among these, a method for breaching a facial recognition system is an attack technique based on adversarial samples that generate a adversarial network.
[0005] Therefore, a biometric-based identification solution that can effectively prevent the challenge sample attack technique is desirable. Summary of the Invention [Means for solving the problem]
[0006] According to one aspect of the present invention, there is provided a method for identifying a biometric feature, including step A of receiving biometric feature data, step B of randomly converting lower-order bit data in the received biometric feature data, step C of preprocessing the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms, and step D of performing identity recognition based on the preprocessed biometric feature data.
[0007] Optionally, in the above method, the biometric data includes facial image data.
[0008] Optionally, in the above method, randomly transforming less significant bits in the received biometric data comprises randomly transforming least significant bits in the biometric data.
[0009] Optionally, in the above method, randomly converting the least significant bits in the biometric feature data includes any one of: setting all least significant bits in the biometric feature data to 0; setting all least significant bits in the biometric feature data to 1; randomly rearranging the least significant bits in the biometric feature data; and randomly generating the least significant bits in the biometric feature data.
[0010] Optionally, in the above method, in step D, identity identification is performed based on the pre-processed biometric feature data to obtain a first matching result, and the method further includes repeatedly performing steps B to D to obtain a second matching result, and directly performing identity identification based on the received biometric feature data to obtain a third matching result.
[0011] Optionally, the method further includes performing a consistency check on the first matching result, the second matching result, and the third matching result, and feeding back the first matching result when all of the first matching result, the second matching result, and the third matching result match.
[0012] Optionally, the method further includes: when there is a discrepancy among the first matching result, the second matching result and the third matching result, feeding back that the identification process is abnormal, and adding the collection terminal of the corresponding data to a blacklist.
[0013] Optionally, in the above method, the biometric data further includes a timestamp, and between step A and step B, the method further includes: decoding the biometric data to obtain the face image data and the timestamp; and feeding back an abnormality when a difference value between a system time and the timestamp does not meet a requirement.
[0014] Optionally, the method further comprises, between step A and step B, detecting a face position in the face image data if a difference between the system time and the timestamp satisfies a requirement.
[0015] Optionally, the method further includes, between step A and step B, adding an adversarial sample to the face position to counteract the effect of a negative adversarial sample added at a data collection terminal, where the negative adversarial sample is a negative value of the adversarial sample.
[0016] Optionally, in the above method, the oppositional samples are pre-generated by performing back conduction training on a generator based on face recognition results.
[0017] According to another aspect of the present invention, there is provided a biometric feature identification server comprising: a receiving device for receiving biometric feature data; a low-bit random filter for randomly converting lower-bit data in the received biometric feature data; a preprocessor for pre-processing the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms; and an identification device for performing identification based on the preprocessed biometric feature data.
[0018] Optionally, in the server, the biometric data includes facial image data.
[0019] Optionally, in the server, the low order random filter is arranged to randomly transform the least significant bits of the biometric feature data.
[0020] Optionally, in the server, the low order random filter is configured to perform any one of: setting all least significant bits in the biometric feature data to 0; setting all least significant bits in the biometric feature data to 1; randomly rearranging the least significant bits in the biometric feature data; and randomly generating the least significant bits of the biometric feature data.
[0021] Optionally, the server further includes an identification determination device for checking whether the identification results are consistent with each other multiple times, and feeding back the result when all the identification results are consistent.
[0022] Optionally, in the server, the identification determination device is configured to feed back that the identification process is abnormal when there is a discrepancy in the identification result, and add the collection terminal of the corresponding data to a blacklist.
[0023] Optionally, at the server, the biometric characteristic data further includes a timestamp.
[0024] Optionally, the server further includes a decoding device for decoding the biometric feature data to obtain the facial image data and the timestamp, and a verification device for feeding back an anomaly if the difference between the system time and the timestamp does not satisfy a requirement.
[0025] Optionally, in the server, the verification device is further arranged to detect a face position in the face image data if a difference between the system time and the timestamp satisfies a requirement.
[0026] Optionally, in the server, the verification device is further configured to add a hostility sample to the face position to counteract the effect of a negative hostility sample added at the data collection terminal, wherein the negative hostility sample is a negative value of the hostility sample.
[0027] Optionally, in the server, the oppositional samples are pre-generated by performing back conduction training on a generator based on face recognition results.
[0028] According to another aspect of the present invention, there is provided a client adapted to cooperate with the server, the client including a face collecting device for collecting face image data and an embedding device for embedding negative oppositional samples into face positions, wherein the negative oppositional samples are opposite numbers of oppositional samples, and the oppositional samples are additive noise superimposed on a real face image to cause a face recognition model to produce an erroneous recognition result.
[0029] Optionally, at the client, the embedding device is configured to detect the face location based on the face image data, embed the negative anti-aliasing sample at the face location, and encrypt and transmit the processed face image and a timestamp.
[0030] According to another aspect of the present invention, there is provided a method for generating a challenging sample using the biometric feature identification server, the method including: step A of configuring a challenging generation network including a generator and an authenticator; step B of generating noise information using the generator and adding the noise information to a real face image to obtain generated face data; step C of authenticating the generated face data using a face recognition model; step D of performing back conduction training on the generator based on the authentication result; step E of repeating steps B to D until the face recognition model generates an incorrect authentication result; and step F of determining the noise information corresponding to the incorrect authentication result as a challenging sample.
[0031] According to another aspect of the invention, there is provided a computer storage medium containing instructions which, when executed, perform the method.
[0032] According to another aspect of the invention there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method.
[0033] Compared with the conventional biometric authentication solutions, the biometric authentication solutions according to one or more embodiments of the present invention randomly convert the least significant bit data in the received biometric data, and perform identity verification based on the pre-processed biometric data. This solution is very destructive to the counter-reactive noise in the image, and the counter-reactive attack function is lost. Therefore, even if an attacker obtains the parameters of the biometric authentication model, the training process of the counter-reactive attack model cannot stably converge, and an effective attack model cannot be formed. [Brief description of the drawings]
[0034] The above and other objects and advantages of the present invention will become more fully apparent from the following detailed description taken in conjunction with the drawings, in which the same or similar elements are represented using the same reference numerals.
[0035] [Figure 1] 1 is a flow diagram of a biometric feature identification method according to one embodiment of the present invention; [Diagram 2] The principle of attack on the resistance sample is shown. [Diagram 3] FIG. 2 is a schematic diagram showing the structure of a biometric feature identification server according to one embodiment of the present invention; [Figure 4] FIG. 2 is a flow diagram illustrating a flow chart of face recognition according to one embodiment of the present invention. [Diagram 5] FIG. 1 is a schematic diagram of a face recognition system including a data collection terminal and a face recognition service platform according to one embodiment of the present invention. [Figure 6] FIG. 11 is a flow diagram showing a flowchart of face recognition according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0036] Hereinafter, the embodiments of the present invention will be described in more detail in combination with drawings and embodiments. The following examples can be used to explain the present invention, but cannot be used to limit the scope of the present invention.
[0037] In the description herein, a description that refers to terms such as "one embodiment," "several embodiments," "examples," "particular examples," or "several examples" refers to the inclusion of a particular feature, structure, material, or characteristic in the embodiment or example in combination with the particular embodiment or example in at least one embodiment or example of the present invention. In the present specification, general expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. Furthermore, a person skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described herein without mutual contradiction.
[0038] In the context of the present invention, the term "client", also called a user terminal, refers to an appliance or device that corresponds to a server and provides local services to the client. In one or more embodiments, the client has some basic functions, including a collection function for collecting biometric information of the user. The client is a smart device of the user, including but not limited to a mobile phone, a laptop, and a headset of the user.
[0039] The term "server", also referred to as a remote server or cloud server, refers to an apparatus or device that corresponds to a "client" and provides remote services to the client. In one or more embodiments, the server can provide the client with a biometric feature authentication (e.g., face authentication) function to prevent attacks by challenging samples, by performing some pre-processing (e.g., noise removal) on the biometric feature data received from the client before identifying the biometric feature.
[0040] FIG. 1 is a flow diagram illustrating a method 1000 for identifying a biometric feature according to one embodiment of the present invention. As shown in FIG. 1, the method 1000 includes: A step S110 of receiving biometric characteristic data; A step S120 of randomly converting the lower bits of the received biometric characteristic data; A step S130 of pre-processing the randomly transformed biometric feature data by randomly selecting and employing one or more filtering algorithms; and performing identity verification based on the pre-processed biometric feature data S140.
[0041] In the context of the present invention, the term "biometric data" refers to data inherent to a human body and auxiliary information related to the data used for personal identification. For example, biometric data includes, but is not limited to, physiological characteristics (e.g., fingerprints, irises, faces, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.).
[0042] In one embodiment, the biometric data includes facial image data. In one embodiment, the biometric data includes, in addition to the facial image data, a timestamp associated with the facial image data. The timestamp may be the time when the facial image data was recorded or the time when the facial image data was uploaded.
[0043] FIG. 2 shows the principle of the attack of the challenging sample. The "challenging sample" can guarantee that it does not affect the recognition by human eyes, and can make the classifier generate an incorrect prediction. For example, the challenging sample can be an abnormal sample generated by an attacker by slightly disturbing a normal sample. In one embodiment, the challenging sample can enable the CNN model to output an incorrect output with high reliability. FIG. 2 shows a challenging sample in a face matching algorithm. It can be seen that the matching score of the face drops sharply from 100 points to below 60 points when the challenging noise is superimposed. That is, the face matching algorithm cannot successfully distinguish between the two people at this point.
[0044] The attack on the opposing sample can be divided into white-box attack and black-box attack. In a white-box attack, the attacker knows all the information of the authentication model, including the training data, model architecture, superparameters, number of layers, activation function and model weights, and performs the attack by calculating the gradient of the model to generate the opposing sample. In a black-box attack, the attacker performs the attack without knowing the above information.
[0045] As can be seen from the principle of the resilience sample attack, the root cause of false recognition in a biometric authentication system (e.g., a face recognition system) is that an attacker adds additive resilience noise to the biometric data (e.g., face image data).
[0046] The inventors of the present application have found that most of the data in the resistive noise or resistive sample has a very low numerical value, for example, only the lower bits of 8-bit image data are converted. Therefore, randomly converting the lower bits of the biometric feature data may destroy most of the resistive noise data. Here, the "lower bits of data" refers to the least significant bit data, the last two digits, or the last three digits of the data unique to the human body that can be used for personal identification.
[0047] In one embodiment, step S120 includes randomly converting the least significant bit of the biometric data. Of course, those skilled in the art will appreciate that in one embodiment, the last two or three digits of the biometric data can be randomly converted.
[0048] In one embodiment, randomly converting the least significant bits of the biometric data includes any one of setting all least significant bits of the biometric data to 0, setting all least significant bits of the biometric data to 1, randomly rearranging the least significant bits of the biometric data, and randomly generating the least significant bits of the biometric data. For example, for 8-bit image data, randomly converting the least significant bits includes four algorithms: (1) setting all 8 least significant bits to 0, (2) setting all 8 least significant bits to 1, (3) randomly rearranging the 8 least significant bits, and (4) randomly generating the 8 least significant bits. In one or more embodiments, the algorithm for randomly converting the least significant bits is not fixed, but is randomly selected from a plurality of algorithms (e.g., the above four algorithms), thereby preprocessing the input image.
[0049] After randomly transforming the least significant bits of the biometric data, the randomly transformed biometric data is preferably filtered (pre-processed) again to smooth the biometric data (e.g., smooth the facial image data). For example, the randomly transformed biometric data may also be pre-processed by randomly selecting one or more filtering algorithms.
[0050] Taking image data (e.g., face image data) as an example, there are multiple image denoising / filtering algorithms, such as, but not limited to, BM3D denoising, DCT denoising, PCA denoising, K-SVD denoising, non-local means denoising, WNNM denoising, image denoising algorithms based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold denoising, Contourlet transform, and pan-invariant Contourlet transform based SAR image denoising, etc. In one or more embodiments, one or more of the above eleven kinds of filtering algorithms are randomly selected to preprocess the input image. In normal cases, denoising does not affect the recognition results of the face recognition system, but it is very destructive to the repulsive noise in the image, and the repulsive attack function is lost.
[0051] In one embodiment, in step S140, identity verification is performed based on the pre-processed biometric feature data to obtain a first matching result. Although not shown in Fig. 1, the method 1000 further includes repeatedly performing steps S120 to S140 to obtain a second comparison result (wherein the filtering algorithm used to re-perform steps S120 and S130 is different or partially different from the filtering algorithm used in the initial execution), and directly performing authentication based on the received biometric feature data to obtain a third matching result.
[0052] In the above embodiment, the method 1000 further includes: performing a consistency check on the first matching result, the second matching result, and the third matching result; and feeding back the first matching result when the first matching result, the second matching result, and the third matching result are all consistent; on the other hand, feeding back an anomaly in the identification process when there is a mismatch among the first matching result, the second matching result, and the third matching result, and adding the collection terminal of the corresponding data to a blacklist.
[0053] As can be seen from the above, the third authentication is performed on the original biometric feature data (i.e., the lower bits are not randomly converted), while the first and second authentications are performed on the biometric feature data with the lower bits randomly converted (different filter algorithms can be adopted). If the results of these three authentications are consistent, the authentication result can be fed back directly. On the other hand, if the results of multiple authentications (e.g., the results of three authentications) are different, an abnormality in the authentication process can be fed back (probable attack of a counter-sample), and the terminal that collected the abnormal data can be added to a blacklist, and the authentication service for the terminal can be refused thereafter.
[0054] In this way, by adding the multi-graph verification strategy and the blacklist denial strategy, we can effectively prevent attackers from conducting black-box attack training.
[0055] At present, the biggest problem affecting face recognition is that camera hijacking attacks can greatly damage the security of face recognition systems. The basic principle is that the attacker first "breaks into" the payment app to hijack the camera data, then cooperates with the attacker to complete the liveness detection process in face recognition, and finally replaces the video data uploaded from the camera with the victim's pre-forged video data, and uploads the forged data to the background face recognition system to complete the entire attack process. At present, the vast majority of face recognition systems do not have the ability to defend against such camera hijacking attacks.
[0056] To address this issue, in one embodiment, the method 1000 further includes, between step S110 and step S120, decoding the biometric data to obtain the face image data and the timestamp, and feeding back an anomaly if the difference between the system time and the timestamp does not meet a requirement. If the difference between the system time and the timestamp meets a requirement, detecting a face position in the face image data, and adding a counter sample to the face position for face recognition.
[0057] Here, the reason for adding the oppositional sample to the face position is to offset the effect of embedding the negative oppositional sample during face image acquisition. That is, in the face image collection process, the face collection terminal (e.g., client) needs to detect the face position, embed the negative oppositional sample in the face detection area, and send the processed face image, face position, and time stamp to the face recognition server in an encrypted manner. In this way, even if the camera is hijacked, the oppositional sample is added to the image that does not have the negative oppositional sample embedded at the time of collection before the authentication service performs face recognition, so that the face recognition process cannot obtain a correct result.
[0058] In one embodiment, the opposing sample is generated in advance by performing back-conduction training on the generator based on the face recognition result. For example, the opposing sample is generated by the steps of (1) forming an opposing generation network, a generation model generating a fake face image, a face recognition model as the authenticator, fixing the parameters of the face recognition model, and optimizing the parameters of only the generator, (2) the generation unit generating noise information and adding the noise information to the real face image to form fake data and fake tags, (3) the face recognition model authenticating the generated face data, (4) performing back-conduction training on the generator based on the face recognition result, and (5) repeating steps (2) to (4) until the face recognition model produces an incorrect authentication result. At this time, the image generated by the generator is an opposing sample A, and its reciprocity number -A is a negative opposing sample. The oppositional samples are additive noise and the negative oppositional samples are the negative of the oppositional samples and are used to counteract the effect of the oppositional samples.
[0059] In one embodiment, the negative oppositional samples are generated only once, and the process of embedding and removing them from the image is computationally easy and efficient, and the face data embedded with the negative oppositional samples is transparent to the human eye and cannot be removed even if the data is decrypted during transmission.
[0060] 3 is a structural schematic diagram showing a biometric feature identification server 3000 according to an embodiment of the present invention. As shown in FIG. 3, the biometric feature identification server 3000 includes a receiving device 310, a low-bit random filter 320, a pre-processor 330, and an authentication device 340. Here, the receiving device 310 receives biometric feature data, the low-bit random filter 320 randomly converts low-bit data in the received biometric feature data, the pre-processor 330 pre-processes the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms, and the identification device 340 performs identity identification based on the pre-processed biometric feature data.
[0061] In the context of the present invention, the term "biometric data" refers to data inherent to a human body and auxiliary information related to the data used for personal identification. For example, biometric data includes, but is not limited to, physiological characteristics (e.g., fingerprints, irises, faces, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.).
[0062] In one embodiment, the biometric data includes facial image data. In one embodiment, the biometric authentication data includes, in addition to the facial image data, a timestamp associated with the facial image data. The timestamp may be the time when the facial image data was recorded or the time when the facial image data was uploaded.
[0063] The term "challenging sample" refers to a sample that can guarantee that it does not affect the authentication by human eyes and causes the classifier to generate an incorrect prediction. For example, the challenging sample may be an abnormal sample generated by an attacker by slightly disturbing a normal sample. In one embodiment, the challenging sample can enable the CNN model to output a highly reliable incorrect output. As shown in FIG. 2, it provides a challenging sample in the face matching algorithm. When the challenging noise is superimposed, it can be seen that the face matching score drops sharply from 100 points to 60 or less. That is, the face matching algorithm cannot successfully authenticate the two people at this point. The attack on the challenging sample can be divided into a white box attack and a black box attack. A white box attack refers to an attack that the attacker knows all the information to authenticate the model, including the training data, the model architecture, the superparameters, the number of layers, the activation function, and the model weights, and performs the attack by calculating the gradient of the model to generate the challenging sample. A black box attack refers to an attack that the attacker performs without knowing the above information.
[0064] As can be seen from the principle of the resilience sample attack, the fundamental reason why a biometric authentication system (e.g., a face recognition system) causes false recognition is that an attacker adds additive resilience noise to the biometric data (e.g., face image data).
[0065] The inventors of the present application have found that most of the data in the resistive noise or resistive sample has a very low numerical value, for example, only the lower bits of 8-bit image data are converted. Therefore, randomly converting the lower bits of the data in the biometric feature data may destroy most of the resistive noise data. Here, the "lower bits of data" refers to the lowest data, the last two digits, or the last three digits of the data unique to the human body that can be used for personal identification.
[0066] In one embodiment, the least significant bit random filter 320 is arranged to randomly transform the least significant bits of the biometric data. Of course, it will be appreciated by those skilled in the art that in one embodiment, the random filter unit 320 is not limited to the least significant bits of the biometric data, but can also randomly transform the last two or last three digits of the data.
[0067] In one embodiment, the least significant bit random filter 320 is configured to perform any one of: setting all least significant bits of the biometric feature data to 0; setting all least significant bits of the biometric feature data to 1; randomly rearranging the least significant bits of the biometric feature data; and randomly generating the least significant bits of the biometric feature data. For example, for 8-bit image data, the least significant bit random transformation is configured to perform any one of: (1) setting all 8 least significant bits to 0; (2) setting all 8 least significant bits to 1; (3) randomly rearranging the 8 least significant bits; and (4) randomly generating the 8 least significant bits. In one or more embodiments, the algorithm employed by the least significant bit random filter 320 is not fixed, but is randomly selected from a plurality of algorithms (e.g., the above four algorithms) to preprocess the input image.
[0068] After the least significant bit random filter 320 randomizes the least significant bits of the biometric data, the pre-processor 330 preferably re-filters (pre-processes) the randomized biometric data to smooth the biometric data (e.g., smooth the facial image data). For example, the pre-processor 330 is arranged to pre-process the randomized biometric data by randomly selecting one or more filtering algorithms.
[0069] Taking image data (e.g., face image data) as an example, there are a number of image denoising / filtering algorithms, such as, but not limited to, BM3D denoising, DCT denoising, PCA denoising, K-SVD denoising, non-local means denoising, WNNM denoising, image denoising algorithms based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold denoising, Contourlet transform, and pan-invariant Contourlet transform based SAR image denoising, etc. In one or more embodiments, the pre-processor is configured to randomly select one or more of the above eleven kinds of filtering algorithms to pre-process the input image. In normal cases, denoising does not affect the recognition results of the face recognition system, but is highly destructive to the oppositional noise in the image, which makes the function of oppositional attacks disappear.
[0070] In one embodiment, the server 3000 further includes an identity determination device 350, which is used to check the identity of multiple identification results and feed back the result when the identification results are all consistent. For example, the first identification is to perform identity determination on the original biometric feature data (i.e., without low-order bit random conversion), and the second and third identification are to perform identity determination on the biometric feature data that has been low-order bit random converted (different filtering algorithms can be used). If the three identification results are consistent, the identity determination device can directly feed back the identification results.
[0071] In one embodiment, the identification determination device 350 is further configured to, if there is a discrepancy in the identification results, feed back that the identification process is abnormal, and add the terminal that collects the corresponding data to a blacklist. That is, if the results of multiple identifications (e.g., the results of three identifications) are different, feed back that the verification process is abnormal (probable attack of countervailing samples), add the terminal that collects the abnormal data to a blacklist, and then refuse to provide the identification service to the terminal. In this way, by adding the multi-graph verification strategy and the blacklist refusal strategy, it is possible to effectively prevent an attacker from conducting black-box attack training.
[0072] At present, the biggest problem affecting face recognition is that camera hijacking attacks can greatly damage the security of face recognition systems. The basic principle is that the attacker first "breaks into" the payment app to hijack the camera data, then cooperates with the attacker to complete the liveness detection process in face recognition, and finally replaces the video data uploaded from the camera with the victim's pre-forged video data, and uploads the forged data to the background face recognition system to complete the entire attack process. At present, the vast majority of face recognition systems do not have the ability to defend against such camera hijacking attacks.
[0073] To address this issue, in one embodiment, the server 3000 further includes a decoding device 360 for decoding the biometric data to obtain the face image data and the timestamp, and a verification device 370 for feeding back an anomaly when the difference between the system time and the timestamp does not meet a requirement. In one embodiment, the verification device 370 is further configured to detect a face position in the face image data and add a challenge sample to the face position for face authentication when the difference between the system time and the timestamp meets a requirement.
[0074] The verification device 370 adds a countervailing sample to the face position in order to counteract the effect of embedding a negative countervailing sample in the process of collecting a face image. That is, in the process of collecting a face image, the face collection terminal (e.g., a client) needs to detect the face position, embed a negative countervailing sample in the face detection area, and transmit the processed face image, the face position, and the timestamp in an encrypted format to a server (e.g., a face recognition authentication server). In this way, even if a camera hijack occurs, even in an image that does not have a negative countervailing sample embedded at the time of collection, a countervailing sample is added before the authentication service performs face authentication, and therefore, the correct result cannot be obtained in the process of face authentication.
[0075] In one embodiment, the oppositional sample is generated in advance by performing back conduction training on the generator based on the face recognition result. For example, the oppositional sample may be generated by a biometric feature identification server, and a specific generation method includes the steps of: forming an oppositional generation network including a generator and an authenticator; generating noise information by the generator, and adding the noise information and a real face image to obtain generated face data; authenticating the generated face data using a face recognition model; performing back conduction training on the generator based on the authentication result; repeating steps B to D until the face recognition model generates an incorrect authentication result; and determining the noise information corresponding to the incorrect authentication result as the oppositional sample A. The reciprocal number -A of the oppositional sample A is a negative oppositional sample. It can be seen that the oppositional sample is an additive noise, and the negative oppositional sample is a negative value of the oppositional sample, which is used to offset the effect of the oppositional sample.
[0076] In one embodiment, the negative oppositional samples are generated only once, and then the process of embedding and removing them from the image is computationally easy and efficient. Also, the face data embedded with the negative oppositional samples has no effect on the human eye, and the data cannot be removed even if the data is decrypted during transmission.
[0077] FIG. 4 is a diagram showing a flowchart of face recognition according to one embodiment of the present invention. As shown in FIG. 4, the face recognition system may include multiple devices or modules, such as a face collection module, a pixel low-bit random filter, a random noise filter, a face recognition function module, and an identification determination module, as shown in FIG. 4. In one embodiment, the face collection module may be located on a client side, such as included in a user's smartphone. The pixel low-bit random filter, the random noise filter, the face recognition function module, and the identification determination module may be located on a server side, such as a face recognition service platform.
[0078] The face collection module is used to collect face data. The numerical value of most of the data of the resistive noise is very low, and only the lower bits of the 8-bit image data are converted, and the pixel lower bit random filter randomly converts the least significant bit of the image data, thereby destroying most of the data of the resistive noise. In addition, the filtering algorithms included in the pixel lower bit random filter mainly include: making all the least significant bits of the 8 bits 0, making all the least significant bits of the 8 bits 1, randomly rearranging the least significant bits of the 8 bits, and randomly generating the least significant bits of the 8 bits. In the filtering process, the pixel lower bit random filter can randomly select from the above four types of algorithms to perform preprocessing on the input image. In the random noise filter, the image denoising algorithm includes, but is not limited to, BM3D noise reduction, BM3D noise reduction, DCT noise reduction, PCA noise reduction, K-SVD noise reduction, NL-Means non-local means noise reduction, WNNM noise reduction, image noise reduction algorithm based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold noise reduction, Contourlet transform, and SAR image noise reduction based on translation invariant Contourlet transform. The random noise filter randomly selects one or several of the above eleven filtering algorithms to preprocess the input image.
[0079] The face recognition function module is for authenticating the face in the image. The identification decision module performs a match check on the results of multiple face matching, feeds back the result if the verification result is a match, or feeds back the abnormality of the identification process if the verification result is not a match, and adds the corresponding data collection terminal to the blacklist.
[0080] In one embodiment, after the face collection module completes data collection, the face recognition function module performs a first face recognition on the original face data, and feeds back the recognition result to the identification determination module. Then, the original face data is subjected to a first pixel low-bit random filtering and a first random noise filtering, and the filtered data is subjected to a second face matching. Then, the original face data is subjected to a second pixel low-bit random filtering and a second random noise filtering (ensuring that the second filtering method is different from the first filtering method), and the filtered data is subjected to a third face matching. Finally, the identification determination module compares the results of the three filterings, and if the three identification results match, it feeds back the normal value of face recognition; if the three authentication results do not match, it feeds back that there is an abnormality in the authentication (abnormal value is S), and adds the terminal that collects the abnormal data to a blacklist, and then the terminal is refused to be provided with face recognition services.
[0081] 5 is a schematic diagram of a face recognition system including a data collection terminal and a face recognition service platform according to an embodiment of the present invention. The data collection terminal is located at a user side, and the face recognition service platform is located at a remote server side.
[0082] In the embodiment of Fig. 5, the data collection terminal provides operating resources including a camera for collecting face data and a TEE for providing a secure operating and storage environment for the application and driver program of the data collection terminal. As shown in Fig. 5, the data collection terminal further includes a negative resistance sample embedding module for detecting a face position, embedding a negative resistance sample in the face detection area, and transmitting the processed face image, face position and timestamp in an encrypted form to a face recognition service platform located at a remote location.
[0083] In one embodiment, the facial recognition service platform is a facial recognition service, and the negative opposing sample verification module is for decrypting the facial image, the facial position and the timestamp, while the facial recognition model is for authenticating the authenticated facial recognition image and feeding back the identity information of the authenticated person.
[0084] FIG. 6 is a diagram showing a flowchart of face recognition by the face recognition system shown in FIG. 5. As shown in FIG. 6, before performing face recognition, the data collection terminal performs application initialization, and stores the face detection model and the negative opposition sample-A in the TEE. Next, the camera collects a face image. The negative opposition sample embedding module detects the face position P in the face image, embeds the negative opposition sample-A in the P area, and then encrypts the face image and the embedded timestamp t and uploads it to the face recognition service platform. Then, on the server side, the negative opposition sample verification module decrypts the face data and the embedded timestamp t, verifies the embedded timestamp t, and if the difference between the system time and t does not meet the requirements, feeds back an abnormality to the platform; otherwise, detects the face position P and embeds the opposition sample A at the position of P. Finally, the face recognition model executes the face recognition flow and feeds back the recognition result.
[0085] In one embodiment, the embedding process of negative directional resistivity includes the steps of detecting a face position P (x1, y1, x2, y2) in an input image F (x and y are the horizontal and vertical coordinates of the face frame, respectively); the content of the pixel block determined by the position P in the image is S, and S1=(SA) mod 256; replacing the S value with S1, and then converting the image to F1; and recording the embedding time t of the negative directional sample and encrypting (F1, t).
[0086] In one embodiment, the process of verifying a negative opposition sample includes the steps of decrypting (F1, t), verifying that the difference between system time t1 and timestamp t is within the range of (-a, a), detecting a face position P (x1, y1, x2, y2) in an input image F, defining the content of a pixel block determined by position P in the image as S1, setting S = (S1 + A) mod 256, replacing the value of S1 with S, and then converting the image to F.
[0087] In addition, in FIG. 6, the negative resistance sample embedding module and the negative resistance sample verification module are shown as single modules, but these modules may be further subdivided depending on the function or need, and it will be understood by those skilled in the art that, for example, the negative resistance sample verification module may be further subdivided into a decoding module and a verification module.
[0088] Moreover, the above device embodiment is merely schematic, and the modules described as separate components may not be physically separated, i.e., may be located in one place or distributed in multiple network modules. According to actual needs, some or all of these modules may be selected to realize the purpose of the solution of the embodiment. Those skilled in the art can understand and implement it without creative labor.
[0089] From the above description of the various embodiments, it will be clearly understood by those skilled in the art that each embodiment can be realized in the form of adding a general-purpose hardware platform required for software, but of course it may be implemented through hardware. Based on such understanding, the above technical solutions can be substantially embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium including any mechanism for storing or transmitting information in a form readable by a computer (e.g., a computer). For example, the machine-readable medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium, an optical storage medium, a flash storage medium, an electrical, optical, acoustic, or other form of propagated signal (e.g., a carrier wave, an infrared signal, a digital signal, etc.), and the computer software product includes some commands, and one computer device (which may be a personal computer, a server, or a network device, etc.) is used to execute the method described in the various embodiments or a part of the embodiments.
[0090] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program commands. These computer program commands are provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, and the commands executed by the processor of the computer or other programmable data processing device can generate a device for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.
[0091] These computer program commands may further be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, where the commands stored in the computer readable memory produce an article of manufacture that includes a command device that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.
[0092] These computer program instructions may be loaded into a computer or other programmable data processing device and a sequence of operational steps executed on the computer or other programmable data processing device to produce a computer-implemented process, whereby the commands executed on the computer or other programmable data processing device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.
[0093] As described above, the biometric authentication solution according to one or more embodiments of the present invention randomly converts the lower bit data of the received biometric data, and performs identity verification according to the pre-processed biometric data. The solution is very destructive to the counter-reactive noise in the image, and the function of counter-reactive attack is lost. Therefore, even if an attacker obtains the parameters of the biometric authentication model, the training process of the counter-reactive attack model cannot stably converge, and an effective attack model cannot be formed.
[0094] Furthermore, in one or more embodiments, a multi-graph verification strategy and a blacklist rejection strategy are added to effectively prevent an attacker from training an attack in black box mode. In addition, by adding a flow of embedding negative oppositional samples to the process of collecting face images and adding a process of verifying (removing) negative oppositional samples to the face recognition system, images that do not have negative oppositional samples embedded therein at the time of collection will be added to the oppositional samples before the authentication service performs face recognition, so that the correct result cannot be obtained during the face recognition process. The negative oppositional samples only need to be generated once, and the subsequent embedding and removal processes in the images are computationally simple and efficient, and the face data embedded with the negative oppositional samples will not affect the human eye, and even if the data is decrypted during the transmission process, the data cannot be removed.
[0095] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have acquired the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all modifications and modifications that fall within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application are within the scope of the claims of the present application and the equivalent technology thereof, the present application is intended to include these modifications and variations as well. [Explanation of symbols]
[0096] 1000 Identification Methods 3000 Identity Server 310 Receiving Equipment 320 Low-bit Random Filter 330 Preprocessor 340 Identification Device 350 Identification and determination device 360 Decoding Device 370 Verification Device
Claims
1. A step A of receiving biometric characteristic data; A step B of randomly converting the lower bits of the received biometric characteristic data; C. pre-processing the randomly transformed biometric data by randomly selecting and employing one or more filtering algorithms; a step D of performing identity verification based on the preprocessed biometric data; A method for identifying a biometric feature comprising:
2. The method of claim 1 , wherein the biometric data includes facial image data.
3. The method of claim 1 , wherein randomly transforming less significant bits of received biometric data comprises randomly transforming least significant bits of the biometric data.
4. Randomly converting the least significant bits of the biometric characteristic data includes: setting all least significant bits of the biometric characteristic data to 0; setting all of the least significant bits of the biometric characteristic data to 1; randomly rearranging the least significant bits of the biometric data; and randomly generating the least significant bits of the biometric characteristic data.
5. In step D, performing identity verification based on the pre-processed biometric feature data to obtain a first matching result; The method comprises: repeatedly performing steps B to D to obtain a second matching result; The method of claim 1 , further comprising: directly performing identity identification based on the received biometric characteristic data to obtain a third matching result.
6. performing a match check on the first match result, the second match result, and the third match result; feeding back the first matching result when all of the first matching result, the second matching result, and the third matching result match; The method of claim 5 further comprising:
7. feedback that the identification process is abnormal when there is a mismatch between the first matching result, the second matching result, and the third matching result; The method of claim 6, further comprising: adding a collection terminal of the corresponding data to a blacklist.
8. The biometric data further includes a time stamp; Between step A and step B, decoding the biometric data to obtain the facial image data and the timestamp; The method of claim 2 , further comprising: feeding back an anomaly if the difference between a system time and the timestamp does not meet a requirement.
9. Between step A and step B, The method of claim 8 , further comprising: detecting a face location in the facial image data if a difference between the system time and the time stamp satisfies a requirement.
10. Between step A and step B, 10. The method of claim 9, further comprising adding an oppositional sample to the face position to counteract the effect of a negative oppositional sample added at a data collection terminal, wherein the negative oppositional sample is a negative value of the oppositional sample.
11. The method of claim 10 , wherein the oppositional samples are pre-generated by training a generator on back conduction based on face recognition results.
12. a receiving device for receiving biometric characteristic data; a low-bit random filter for randomly converting low-bit data in the received biometric characteristic data; a pre-processor for pre-processing the randomly transformed biometric feature data by randomly selecting and employing one or more filtering algorithms; an identification device for performing identification based on the preprocessed biometric feature data; A biometric feature identification server comprising:
13. The server of claim 12 , wherein the biometric data includes facial image data.
14. 13. The server of claim 12, wherein the least significant bit random filter is arranged to randomly transform the least significant bits in the biometric characteristic data.
15. The lower bit random filter is setting all least significant bits of the biometric characteristic data to 0; setting all of the least significant bits of the biometric characteristic data to 1; randomly rearranging the least significant bits of the biometric data; and randomly generating the least significant bits of the biometric characteristic data.
16. 13. The server according to claim 12, further comprising an identification determination device that checks whether the identification results are consistent with each other multiple times and feeds back the identification result when all the identification results are consistent.
17. The server according to claim 16, wherein the identification determination device is configured to: feed back that the identification process is abnormal when there is a discrepancy in the identification result, and add the collection terminal of the corresponding data to a blacklist.
18. The server of claim 13 , wherein the biometric data further includes a timestamp.
19. a decoding device for decoding the biometric data to obtain the facial image data and the time stamp; The server according to claim 18 , further comprising: a verification device that feeds back an anomaly if a difference between a system time and the timestamp does not satisfy a requirement.
20. 20. The server of claim 19, wherein the verification device is further arranged to detect a face position in the facial image data if a difference between the system time and the time stamp satisfies a requirement.
21. The server of claim 20, wherein the verification device is further configured to add a hostility sample to the face position to counteract the effect of a negative hostility sample added at a data collection terminal, wherein the negative hostility sample is a negative value of the hostility sample.
22. The server of claim 21 , wherein the oppositional samples are pre-generated by performing back conduction training on a generator based on face recognition results.
23. a face collection device for collecting face image data; 22. A client adapted to cooperate with the server of claim 21, comprising: an embedder for embedding negative oppositional samples into face locations.
24. 24. The client of claim 23, wherein the embedding device is configured to detect the face location based on the facial image data, embed the negative antipathy sample at the face location, and encrypt and transmit the processed facial image and a timestamp.
25. A step of constructing a competitive generation network including generators and authenticators; A step B of generating noise information using the generator and adding the noise information to a real face image to obtain generated face data; A step C of authenticating the generated face data using a face recognition model; A step D of training the generator in retrograde conduction based on the authentication result; Step E repeats steps B to D until the face recognition model produces an incorrect recognition result; A method for generating a challenge sample using a biometric feature identification server according to any one of claims 12 to 22, comprising: a step F of determining noise information corresponding to the erroneous authentication result as a challenge sample.
26. A computer storage medium comprising instructions which, when executed, perform the method of any one of claims 1 to 11.
27. There is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 11.
Citation Information
Patent Citations
Continuous data processing method
JP2001078008A
Authentication system for cellphone for use in private school such as children music school, and program
JP2008059543A
Method and system for approving user transaction
JP2015109621A
Image conversion apparatus and image conversion method
JP2019148980A
Method and apparatus for training a hand keypoint recognition model, method and apparatus for recognizing hand keypoints, and computer program
JP2021518022A
