Method for identifying biometric characteristics, server and client

Abstract

The present application relates to a biometric identification method, including: receiving biometric data, randomly converting lower-order bit data in the received biometric data, randomly selecting one or more filtering algorithms to pre-process the randomly converted biometric data, and performing identity identification based on the pre-processed biometric data. The present application further relates to a method for generating a challenge sample, a biometric identification server, a client, a computer storage medium, and a computer program product.

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application (application number: 202210108922.9).

[0002] The present application relates to identity identification solutions, and more particularly to a biometric feature identification method, a biometric feature identification server, a client, a computer storage medium and a computer program product. [Background technology]

[0003] In recent years, biometric features (e.g., face and fingerprint) have become the mainstream identity identification method due to their close association with personal identity, and have many applications in important financial transactions such as withdrawals, payments, and bank registration via mobile phones.

[0004] Recently, the Department of Science and Technology of the People's Bank of China issued a "Fintech Application Risk Statement," in which it pointed out the potential risks of applying facial technology in the financial payment field in response to media reports that "face recognition systems on multiple Android mobile phones have been breached." One method for breaching facial recognition systems is an attack technique based on adversarial samples that generate adversarial networks.

[0005] Therefore, a biometric-based identification solution that can effectively prevent the counter-sample attack technique is desired. Summary of the Invention [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided a method for identifying biometric features, including step A of receiving biometric feature data; step B of randomly converting lower-order bit data in the received biometric feature data; step C of preprocessing the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms; and step D of performing identity recognition based on the preprocessed biometric feature data.

[0007] Optionally, in the above method, the biometric data includes facial image data.

[0008] Optionally, in the above method, randomly transforming the least significant bits in the received biometric characteristic data comprises randomly transforming the least significant bits in the biometric characteristic data.

[0009] Optionally, in the above method, randomly converting the least significant bits in the biometric feature data includes any one of setting all least significant bits in the biometric feature data to 0, setting all least significant bits in the biometric feature data to 1, randomly rearranging the least significant bits in the biometric feature data, and randomly generating the least significant bits in the biometric feature data.

[0010] Optionally, in the above method, in step D, identity identification is performed based on the pre-processed biometric feature data to obtain a first matching result, and the method further includes repeatedly performing steps B to D to obtain a second matching result, and directly performing identity identification based on the received biometric feature data to obtain a third matching result.

[0011] Optionally, the method further includes performing a match check on the first matching result, the second matching result, and the third matching result, and feeding back the first matching result when all of the first matching result, the second matching result, and the third matching result match.

[0012] Optionally, the method further includes, when there is a discrepancy between the first matching result, the second matching result and the third matching result, feeding back that the identification process is abnormal, and adding the collection terminal of the corresponding data to a blacklist.

[0013] Optionally, in the above method, the biometric data further includes a timestamp, and between step A and step B, the method further includes: decoding the biometric data to obtain the face image data and the timestamp; and feeding back an abnormality when a difference value between the system time and the timestamp does not meet a requirement.

[0014] Optionally, the method further includes, between step A and step B, detecting a face position in the face image data if the difference between the system time and the timestamp satisfies a requirement.

[0015] Optionally, the method further includes, between step A and step B, adding an adversarial sample to the face position to counteract the effect of a negative adversarial sample added at the data collection terminal, wherein the negative adversarial sample is a negative value of the adversarial sample.

[0016] Optionally, in the above method, the opposing samples are pre-generated by back-conduction training a generator based on face recognition results.

[0017] According to another aspect of the present invention, there is provided a biometric feature identification server comprising: a receiving device for receiving biometric feature data; a low-bit random filter for randomly converting low-order bits of the received biometric feature data; a preprocessor for pre-processing the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms; and an identification device for performing identification based on the preprocessed biometric feature data.

[0018] Optionally, in the server, the biometric data includes facial image data.

[0019] Optionally, in the server, the low order random filter is arranged to randomly transform the least significant bits in the biometric feature data.

[0020] Optionally, in the server, the low order random filter is configured to perform any one of: setting all least significant bits in the biometric feature data to 0; setting all least significant bits in the biometric feature data to 1; randomly rearranging the least significant bits in the biometric feature data; and randomly generating the least significant bits of the biometric feature data.

[0021] Optionally, the server further includes an identification determination device for checking whether the identification results are consistent with each other multiple times and feeding back the result when all the identification results are consistent.

[0022] Optionally, in the server, the identification determination device is configured to, when there is a discrepancy in the identification result, feed back that the identification process is abnormal, and add the collection terminal of the corresponding data to a blacklist.

[0023] Optionally, at the server, the biometric data further includes a timestamp.

[0024] Optionally, the server further includes a decoding device for decoding the biometric feature data to obtain the facial image data and the timestamp, and a verification device for feeding back an abnormality if the difference between the system time and the timestamp does not meet requirements.

[0025] Optionally, in the server, the verification device is further arranged to detect a face position in the face image data if the difference between the system time and the timestamp satisfies a requirement.

[0026] Optionally, in the server, the verification device is further configured to add a repulsive sample to the face position to counteract the effect of a negative repulsive sample added at the data collection terminal, wherein the negative repulsive sample is a negative value of the repulsive sample.

[0027] Optionally, in the server, the oppositional samples are pre-generated by performing back conduction training on a generator based on face recognition results.

[0028] According to another aspect of the present invention, there is provided a client adapted to cooperate with the server, the client including a face collecting device for collecting face image data and an embedding device for embedding negative opposing samples into face locations, wherein the negative opposing samples are opposite numbers of opposing samples, and the opposing samples are additive noise superimposed on real face images to cause a face recognition model to produce erroneous recognition results.

[0029] Optionally, at the client, the embedding device is configured to detect the face location based on the face image data, embed the negative opposition sample at the face location, and encrypt and transmit the processed face image and a timestamp.

[0030] According to another aspect of the present invention, there is provided a method for generating a challenging sample using the biometric feature identification server, the method including: step A of configuring a challenging generation network including a generator and an authenticator; step B of generating noise information using the generator and adding the noise information to a real face image to obtain generated face data; step C of authenticating the generated face data using a face recognition model; step D of performing back conduction training on the generator based on the authentication result; step E of repeating steps B to D until the face recognition model generates an incorrect authentication result; and step F of determining the noise information corresponding to the incorrect authentication result as a challenging sample.

[0031] According to another aspect of the invention, there is provided a computer storage medium containing instructions which, when executed, perform the method.

[0032] According to another aspect of the invention there is provided a computer program product comprising a computer program which when executed by a processor implements the method.

[0033] Compared with conventional biometric authentication solutions, the biometric authentication solution according to one or more embodiments of the present invention randomly transforms the least significant bits of received biometric data and performs identity verification based on the pre-processed biometric data. This solution is highly destructive to counter-noise in the image, making counter-attacks ineffective. As a result, even if an attacker obtains parameters in the biometric authentication model, the training process for the counter-attack model cannot converge stably, and an effective attack model cannot be formed. [Brief explanation of the drawings]

[0034] These and other objects and advantages of the present invention will become more fully apparent from the following detailed description taken in conjunction with the drawings, in which the same or similar elements are represented using the same reference numerals.

[0035] [Figure 1] 1 is a flow diagram of a method for identifying biometric features according to one embodiment of the present invention; [Figure 2] The principles of attack on the resistance sample are presented. [Figure 3] FIG. 2 is a schematic diagram illustrating the structure of a biometric feature identification server according to one embodiment of the present invention; [Figure 4] FIG. 1 is a flow diagram illustrating a flowchart of face recognition according to one embodiment of the present invention. [Figure 5] 1 is a schematic diagram of a face recognition system including a data collection terminal and a face recognition service platform according to an embodiment of the present invention; [Figure 6] FIG. 10 is a flow diagram showing a flowchart of face recognition according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0036] Hereinafter, the embodiments of the present invention will be described in more detail in combination with the drawings and embodiments. The following examples can be used to explain the present invention, but cannot be used to limit the scope of the present invention.

[0037] In the description herein, any reference to terms such as "one embodiment," "some embodiments," "examples," "particular examples," or "some examples" indicates that the particular feature, structure, material, or characteristic described in combination with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, general expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. Furthermore, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described herein without mutual contradiction.

[0038] In the context of the present invention, the term "client," also referred to as a user terminal, refers to a device or apparatus that corresponds to a server and provides local services to the client. In one or more embodiments, the client has some basic functions, including a collection function for collecting biometric information of the user. The client is a user's smart device, including, but not limited to, a user's mobile phone, laptop, and headset.

[0039] The term "server," also known as a remote server or cloud server, refers to a device or apparatus that serves a "client" and provides remote services to the client. In one or more embodiments, the server may provide the client with a biometric feature authentication (e.g., face authentication) function to prevent attacks using counterfeit samples, by performing some pre-processing (e.g., noise removal) on biometric feature data received from the client before identifying the biometric feature.

[0040] 1 is a flow diagram illustrating a method 1000 for identifying biometric features according to one embodiment of the present invention. As shown in FIG. 1, the method 1000 includes: a step S110 of receiving biometric feature data; Step S120: randomly converting the lower bits of the received biometric feature data; a step S130 of preprocessing the randomly transformed biometric feature data by randomly selecting and employing one or more filtering algorithms; and step S140 of performing identity verification based on the pre-processed biometric feature data.

[0041] In the context of the present invention, the term "biometric data" refers to data inherent to a human body and auxiliary information related to the data used for personal identification. For example, biometric data includes, but is not limited to, physiological characteristics (e.g., fingerprints, irises, faces, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.).

[0042] In one embodiment, the biometric data includes facial image data. In one embodiment, the biometric data includes, in addition to the facial image data, a timestamp associated with the facial image data. The timestamp may be the time the facial image data was recorded or the time the facial image data was uploaded.

[0043] FIG. 2 illustrates the principle of a challenging sample attack. A "challenging sample" is a sample that can be guaranteed not to affect human visual recognition but can cause a classifier to generate incorrect predictions. For example, a challenging sample may be an abnormal sample generated by an attacker by slightly disturbing a normal sample. In one embodiment, the challenging sample can enable a CNN model to output an incorrect output with high reliability. FIG. 2 illustrates a challenging sample in a face matching algorithm. It can be seen that when the challenging noise is added, the face matching score drops sharply from 100 to below 60. This means that the face matching algorithm can no longer successfully distinguish between the two people.

[0044] Attacks on challenging samples can be divided into white-box attacks and black-box attacks. A white-box attack is one in which the attacker knows all the information about the authentication model, including the training data, model architecture, superparameters, number of layers, activation function, and model weights, and then calculates the gradient of the model to generate challenging samples. A black-box attack is one in which the attacker does not know the above information.

[0045] As can be seen from the principle of resilience sample attack, the fundamental cause of false recognition in biometric feature authentication systems (e.g., face recognition systems) is that an attacker adds additive resilience noise to biometric feature data (e.g., face image data).

[0046] The inventors of the present application have discovered that most of the data in the resistive noise or resistive sample has a very low numerical value, for example, only the lower bits of 8-bit image data are converted. Therefore, randomly converting the lower bits of the biometric feature data may destroy most of the resistive noise data. Here, "lower bits of data" refers to the least significant bit, the last two digits, or the last three digits of the data unique to the human body that can be used for personal identification.

[0047] In one embodiment, step S120 includes randomly converting the least significant bit of the biometric data. Of course, those skilled in the art will understand that in one embodiment, the last two or three digits of the biometric data can be randomly converted.

[0048] In one embodiment, randomly converting the least significant bits of the biometric feature data includes any one of: setting all least significant bits of the biometric feature data to 0; setting all least significant bits of the biometric feature data to 1; randomly rearranging the least significant bits of the biometric feature data; and randomly generating the least significant bits of the biometric feature data. For example, for 8-bit image data, randomly converting the least significant bits includes four algorithms: (1) setting all 8 least significant bits to 0; (2) setting all 8 least significant bits to 1; (3) randomly rearranging the 8 least significant bits; and (4) randomly generating the 8 least significant bits. In one or more embodiments, the algorithm for randomly converting the least significant bits is not fixed but is randomly selected from a plurality of algorithms (e.g., the four algorithms described above), thereby preprocessing the input image.

[0049] After randomly transforming the least significant bits of the biometric data, it is preferable to filter (pre-process) the randomly transformed biometric data again to smooth the biometric data (e.g., smooth the facial image data). For example, the randomly transformed biometric data can also be pre-processed by randomly selecting one or more filtering algorithms.

[0050] Taking image data (e.g., facial image data) as an example, there are multiple image denoising / filtering algorithms, such as, but not limited to, BM3D noise reduction, DCT noise reduction, PCA noise reduction, K-SVD noise reduction, non-local means noise reduction, WNNM noise reduction, image noise reduction algorithms based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold noise reduction, Contourlet transform, and pan-invariant Contourlet transform-based SAR image denoising. In one or more embodiments, one or more of the above eleven filtering algorithms are randomly selected to preprocess the input image. While denoising typically does not affect the authentication results of a facial recognition system, it is highly destructive to the resistance noise in the image, eliminating the ability to perform resistance attacks.

[0051] In one embodiment, in step S140, identity verification is performed based on the pre-processed biometric feature data to obtain a first matching result. Although not shown in Fig. 1, the method 1000 further includes repeatedly performing steps S120 to S140 to obtain a second comparison result (wherein the filtering algorithm used to re-perform steps S120 and S130 is different or partially different from the filtering algorithm used in the initial execution), and directly performing authentication based on the received biometric feature data to obtain a third matching result.

[0052] In the above embodiment, the method 1000 further includes: performing a consistency check on the first matching result, the second matching result, and the third matching result; and feeding back the first matching result if the first matching result, the second matching result, and the third matching result all match; on the other hand, feeding back an anomaly in the identification process if there is a mismatch among the first matching result, the second matching result, and the third matching result, and adding the corresponding data collection terminal to a blacklist.

[0053] As can be seen from the above, the third authentication is performed on the original biometric data (i.e., the lower bits are not randomly converted), while the first and second authentications are performed on the biometric data with the lower bits randomly converted (different filter algorithms can be used). If the results of these three authentications are consistent, the authentication result can be directly fed back. On the other hand, if the results of multiple authentications (e.g., the results of three authentications) are different, an abnormality in the authentication process can be fed back (high probability of a counter-sample attack), and the terminal that collected the abnormal data can be added to a blacklist and subsequently refused to provide authentication services to the terminal.

[0054] In this way, by adding the multi-graph verification strategy and the blacklist denial strategy, we can effectively prevent attackers from conducting black-box attack training.

[0055] Currently, the biggest problem affecting facial recognition is camera hijacking attacks, which significantly undermine the security of facial recognition systems. The basic principle is that an attacker first "breaks into" a payment app to hijack camera data. Next, the attacker cooperates with the payment app to complete the liveness detection process in facial recognition. Finally, the attacker replaces the video data uploaded from the camera with pre-forged video data of the victim, and uploads the forged data to the background facial recognition system, completing the entire attack process. Currently, the vast majority of facial recognition systems lack the ability to defend against such camera hijacking attacks.

[0056] To address this issue, in one embodiment, the method 1000 further includes, between step S110 and step S120, decoding the biometric data to obtain the facial image data and the timestamp, and feeding back an anomaly if the difference between the system time and the timestamp does not meet a requirement. If the difference between the system time and the timestamp meets a requirement, detecting a face position in the facial image data and adding a counter sample to the face position for facial recognition.

[0057] Here, the reason for adding the countervailing samples to the face position is to offset the effect of embedding the negative countervailing samples during face image acquisition. That is, in the face image collection process, the face collection terminal (e.g., client) needs to detect the face position, embed the negative countervailing samples in the face detection area, and send the processed face image, face position, and timestamp to the face recognition server in an encrypted form. In this way, even if a camera hijack occurs, even images that do not have the negative countervailing samples embedded at the time of collection will have the countervailing samples added before the authentication service performs face recognition, which will result in an incorrect result in the face recognition process.

[0058] In one embodiment, the oppositional sample is generated in advance by training a generator through back-conduction based on the face recognition result. For example, the oppositional sample is generated by the following steps: (1) constructing an oppositional generative network, a generative model generating a fake face image, a face recognition model as the authenticator, fixing the parameters of the face recognition model, and optimizing the parameters of only the generator; (2) a generator generating noise information and adding the noise information to real face images to form fake data and fake tags; (3) a face recognition model authenticating the generated face data; (4) training the generator through back-conduction based on the face recognition result; and (5) repeating steps (2) to (4) until the face recognition model produces an incorrect authentication result. In this case, the image generated by the generator is oppositional sample A, and its reciprocal number −A is the negative oppositional sample. The opposing samples are additive noise, and the negative opposing samples are the negative of the opposing samples and are used to cancel the effect of the opposing samples.

[0059] In one embodiment, the negative opposition samples are generated only once, making the process of embedding and removing them from the image computationally easy and efficient. Furthermore, the face data embedded with the negative opposition samples is transparent to the human eye and cannot be removed even if the data is decoded during transmission.

[0060] 3 is a structural schematic diagram showing a biometric feature identification server 3000 according to one embodiment of the present invention. As shown in FIG. 3, the biometric feature identification server 3000 includes a receiving device 310, a low-bit random filter 320, a pre-processor 330, and an authentication device 340. Here, the receiving device 310 receives biometric feature data, the low-bit random filter 320 randomly converts low-bit data in the received biometric feature data, the pre-processor 330 pre-processes the randomly converted biometric feature data by randomly selecting and employing one or more filtering algorithms, and the identification device 340 performs identity identification based on the pre-processed biometric feature data.

[0061] In the context of the present invention, the term "biometric data" refers to data inherent to a human body and auxiliary information associated with the data that is used for personal identification. For example, biometric data includes, but is not limited to, physiological characteristics (e.g., fingerprints, irises, faces, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.).

[0062] In one embodiment, the biometric data includes facial image data. In one embodiment, the biometric authentication data includes, in addition to the facial image data, a timestamp associated with the facial image data. The timestamp may be the time the facial image data was recorded or the time the facial image data was uploaded.

[0063] A "challenging sample" refers to a sample that can guarantee no impact on human recognition but that can cause a classifier to generate an incorrect prediction. For example, a challenging sample can be an abnormal sample generated by an attacker by slightly perturbing a normal sample. In one embodiment, a challenging sample can enable a CNN model to output a highly reliable incorrect output. As shown in Figure 2, this provides a challenging sample for a face matching algorithm. When challenging noise is added, the face matching score drops sharply from 100 to below 60. At this point, the face matching algorithm cannot successfully authenticate the two people. Attacks on challenging samples can be divided into white-box attacks and black-box attacks. A white-box attack refers to an attack in which an attacker knows all the information required to authenticate a model, including the training data, model architecture, superparameters, number of layers, activation function, and model weights, and then calculates the gradient of the model to generate a challenging sample. A black-box attack refers to an attack in which the attacker does not know the above information.

[0064] As can be seen from the principle of resilience sample attack, the fundamental reason why biometric feature authentication systems (e.g., face recognition systems) cause false authentication is that an attacker adds additive resilience noise to biometric feature data (e.g., face image data).

[0065] The inventors of the present application have discovered that most of the data in the resistive noise or resistive sample has a very low numerical value, for example, only the lower bits of 8-bit image data are converted. Therefore, randomly converting the lower bits of the biometric feature data may destroy most of the resistive noise data. Here, "lower bits of data" refers to the lowest, last two, or last three digits of the data unique to the human body that can be used for personal identification.

[0066] In one embodiment, the least significant bit random filter 320 is configured to randomly transform the least significant bits of the biometric data. Of course, those skilled in the art will appreciate that in one embodiment, the random filter unit 320 is not limited to the least significant bits of the biometric data, but can also randomly transform the last two or last three digits of the data.

[0067] In one embodiment, the least significant bit random filter 320 is configured to perform any of the following: set all least significant bits of the biometric feature data to 0; set all least significant bits of the biometric feature data to 1; randomly rearrange the least significant bits of the biometric feature data; or randomly generate the least significant bits of the biometric feature data. For example, for 8-bit image data, the least significant bit random transformation is configured to perform any one of (1) set all 8 least significant bits to 0; (2) set all 8 least significant bits to 1; (3) randomly rearrange the 8 least significant bits; or (4) randomly generate the 8 least significant bits. In one or more embodiments, the algorithm employed by the least significant bit random filter 320 is not fixed, but is randomly selected from a plurality of algorithms (e.g., the four algorithms described above) to preprocess the input image.

[0068] After the low-bit random filter 320 randomizes the least significant bits of the biometric data, the pre-processor 330 preferably re-filters (pre-processes) the randomized biometric data to smooth the biometric data (e.g., smooth the facial image data). For example, the pre-processor 330 is arranged to pre-process the randomized biometric data by randomly selecting one or more filtering algorithms.

[0069] Taking image data (e.g., facial image data) as an example, there are multiple image denoising / filtering algorithms, such as, but not limited to, BM3D noise reduction, DCT noise reduction, PCA noise reduction, K-SVD noise reduction, non-local means noise reduction, WNNM noise reduction, image noise reduction algorithms based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold noise reduction, Contourlet transform, and pan-invariant Contourlet transform-based SAR image denoising. In one or more embodiments, the pre-processor is configured to randomly select one or more of the above eleven filtering algorithms to preprocess the input image. While denoising does not generally affect the recognition results of a facial recognition system, it is highly destructive to the reactive noise in the image, eliminating the ability of reactive attacks.

[0070] In one embodiment, the server 3000 further includes an identification determination device 350, which performs identity verification on multiple identification results and feeds back the result if all the identification results match. For example, the first identification is performed on the original biometric feature data (i.e., without low-order bit randomization), and the second and third identifications are performed on the biometric feature data that has been low-order bit randomized (different filtering algorithms can be used). If these three identification results match, the identification determination device can directly feed back the identification results.

[0071] In one embodiment, the identification determination device 350 is further configured to, if there is a discrepancy in the identification results, provide feedback that the identification process is abnormal and add the terminal collecting the corresponding data to a blacklist. That is, if the results of multiple identifications (e.g., the results of three identifications) are different, provide feedback that the verification process is abnormal (high probability of a counterfeit sample attack), add the terminal collecting the abnormal data to a blacklist, and thereafter refuse to provide identification services to the terminal. In this way, by adding the multi-graph verification strategy and the blacklist rejection strategy, it is possible to effectively prevent attackers from conducting black-box attack training.

[0072] Currently, the biggest problem affecting facial recognition is camera hijacking attacks, which significantly undermine the security of facial recognition systems. The basic principle is that an attacker first "breaks into" a payment app to hijack camera data. Next, the attacker cooperates with the payment app to complete the liveness detection process in facial recognition. Finally, the attacker replaces the video data uploaded from the camera with pre-forged video data of the victim, and uploads the forged data to the background facial recognition system, completing the entire attack process. Currently, the vast majority of facial recognition systems lack the ability to defend against such camera hijacking attacks.

[0073] To address this issue, in one embodiment, the server 3000 further includes a decoding device 360 ​​for decoding the biometric feature data to obtain the facial image data and the timestamp, and a verification device 370 for feeding back an abnormality when a difference between a system time and the timestamp does not meet a requirement. In one embodiment, the verification device 370 is further configured to detect a face position in the facial image data and add a challenge sample to the face position for face authentication when the difference between the system time and the timestamp meets a requirement.

[0074] The verification device 370 adds the countervailing samples to the face positions to counteract the effect of embedding the negative countervailing samples in the process of collecting the face images. That is, in the process of collecting the face images, the face collection terminal (e.g., client) needs to detect the face positions, embed the negative countervailing samples in the face detection areas, and send the processed face images, face positions, and timestamps in encrypted form to a server (e.g., face recognition authentication server). In this way, even if a camera is hijacked, even images that do not have the negative countervailing samples embedded in them at the time of collection will have the countervailing samples added before the authentication service performs face authentication, and therefore the face authentication process will not produce correct results.

[0075] In one embodiment, the oppositional sample is pre-generated by performing back-conduction training on a generator based on the face recognition result. For example, the oppositional sample may be generated by a biometric feature identification server. A specific generation method includes: step A: configuring an oppositional generation network including a generator and an authenticator; step B: generating noise information using the generator and adding the noise information to an actual face image to obtain generated face data; step C: authenticating the generated face data using a face recognition model; step D: performing back-conduction training on the generator based on the authentication result; step E: repeating steps B to D until the face recognition model generates an incorrect authentication result; and step F: determining the noise information corresponding to the incorrect authentication result as oppositional sample A. The reciprocal number -A of oppositional sample A is a negative oppositional sample. It can be seen that the oppositional sample is additive noise, and the negative oppositional sample is the negative value of the oppositional sample and is used to offset the effect of the oppositional sample.

[0076] In one embodiment, the negative opposition samples are generated only once, after which the process of embedding and removing them from the image is computationally easy and efficient. Additionally, the face data embedded with the negative opposition samples is transparent to the human eye and cannot be removed even if the data is decoded during transmission.

[0077] 4 is a flowchart illustrating face recognition according to one embodiment of the present invention. As shown in FIG. 4, the face recognition system may include multiple devices or modules, such as a face collection module, a pixel low-bit random filter, a random noise filter, a face recognition function module, and an identity determination module, as shown in FIG. 4. In one embodiment, the face collection module may be located on a client side, such as a user's smartphone. The pixel low-bit random filter, the random noise filter, the face recognition function module, and the identity determination module may be located on a server side, such as a face recognition service platform.

[0078] The face collection module is used to collect face data. Most of the data from the resistive noise has a very low numerical value, so only the lower bits of the 8-bit image data are converted. The pixel lower bit random filter randomly converts the least significant bits of the image data, destroying most of the resistive noise data. The filtering algorithms included in the pixel lower bit random filter mainly include: setting all 8 least significant bits to 0, setting all 8 least significant bits to 1, randomly rearranging the 8 least significant bits, and randomly generating the 8 least significant bits. During the filtering process, the pixel lower bit random filter can randomly select from the above four algorithms to perform preprocessing on the input image. In the random noise filter, the image denoising algorithms include, but are not limited to, BM3D noise reduction, BM3D noise reduction, DCT noise reduction, PCA noise reduction, K-SVD noise reduction, NL-Means non-local means noise reduction, WNNM noise reduction, image noise reduction algorithms based on principal component analysis and bilateral filtering, wavelet transform, wavelet threshold noise reduction, Contourlet transform, and SAR image noise reduction based on translation-invariant Contourlet transform. The random noise filter randomly selects one or several of the above 11 filtering algorithms to preprocess the input image.

[0079] The face recognition function module is for recognizing the face in the image, and the identification determination module performs a match check on the results of multiple face matching, and if the verification result is a match, it feeds back the result, and if the verification result is not a match, it feeds back the abnormality of the identification process, and adds the corresponding data collection terminal to the blacklist.

[0080] In one embodiment, after the face collection module completes data collection, the face recognition function module performs a first round of face recognition on the original face data and feeds back the recognition result to the identification determination module. Next, the original face data undergoes a first round of pixel low-order bit random filtering and a first round of random noise filtering, and the filtered data undergoes a second round of face matching. Then, the original face data undergoes a second round of pixel low-order bit random filtering and a second round of random noise filtering (ensuring that the second filtering method is different from the first filtering method), and the filtered data undergoes a third round of face matching. Finally, the identification determination module compares the results of the three filtering rounds, and if the three recognition results match, it feeds back a normal value for face recognition. If the three recognition results do not match, it feeds back an abnormality in recognition (abnormal value S), and adds the terminal that collected the abnormal data to a blacklist, after which the terminal is denied face recognition service.

[0081] 5 is a schematic diagram of a face recognition system including a data collection terminal and a face recognition service platform according to one embodiment of the present invention, where the data collection terminal is located at the user side and the face recognition service platform is located at the remote server side.

[0082] In the embodiment of Figure 5, the data collection terminal provides operating resources including a camera for collecting face data and a TEE for providing a secure operating and storage environment for applications and driver programs of the data collection terminal. As shown in Figure 5, the data collection terminal further includes a negative resistance sample embedding module for detecting a face position, embedding a negative resistance sample in the face detection area, and transmitting the processed face image, face position, and timestamp in an encrypted format to a face recognition service platform located at a remote location.

[0083] In one embodiment, the facial recognition service platform is a facial recognition service, and the negative opposing sample verification module decodes the facial image, face position and timestamp, while the facial recognition model authenticates the authenticated facial recognition image and feeds back the identity information of the authenticated person.

[0084] FIG. 6 is a flowchart of face recognition using the face recognition system shown in FIG. 5. As shown in FIG. 6, before performing face recognition, the data collection terminal initializes the application and stores the face detection model and negative adversarial sample A in the TEE. Next, the camera collects face images. The negative adversarial sample embedding module detects face position P in the face image and embeds negative adversarial sample A in the P region. The face image and the embedded timestamp t are then encrypted and uploaded to the face recognition service platform. Then, on the server side, the negative adversarial sample verification module decrypts the face data and the embedded timestamp t, verifies the embedded timestamp t, and if the difference between the system time and t does not meet the requirements, reports an abnormality to the platform. Otherwise, it detects face position P and embeds adversarial sample A at the position of P. Finally, the face recognition model executes the face recognition flow and reports the authentication result.

[0085] In one embodiment, the negative directional embedding process includes the steps of detecting a face position P (x1, y1, x2, y2) in an input image F (x and y are the horizontal and vertical coordinates of the face frame, respectively); determining that the content of the pixel block determined by the position P in the image is S, and setting S1 = (SA) mod 256, replacing the S value with S1, and then converting the image to F1; and recording the embedding time t of the negative directional sample and encrypting (F1, t).

[0086] In one embodiment, the verification process of the negative opposition sample includes the steps of: decoding (F1, t); verifying that the difference between the system time t1 and the timestamp t is within the range of (-a, a); detecting the face position P(x1, y1, x2, y2) in the input image F; defining the content of the pixel block determined by the position P in the image as S1, setting S=(S1+A) mod 256, replacing the value of S1 with S; and then converting the image to F.

[0087] In addition, in Figure 6, the negative countermeasure sample embedding module and the negative countermeasure sample verification module are shown as single modules, but it will be understood by those skilled in the art that these modules may be further subdivided depending on function or need, for example, the negative countermeasure sample verification module may be further subdivided into a decoding module and a verification module.

[0088] Furthermore, the above device embodiment is merely schematic, and the modules described as separate components may not be physically separated, i.e., may be located in one place or distributed across multiple network modules. According to actual needs, some or all of these modules may be selected to achieve the purpose of the solution of the present embodiment. Those skilled in the art can understand and implement it without any creative work.

[0089] From the above description of various embodiments, it will be clear to those skilled in the art that each embodiment can be realized in the form of software plus a required general-purpose hardware platform, although of course it may be implemented via hardware. Based on this understanding, the technical solutions described above, or portions that contribute to the prior art, can be substantially embodied in the form of a software product, and the computer software product can be stored on a computer-readable storage medium, which includes any mechanism for storing or transmitting information in a form readable by a computer (e.g., a computer). For example, machine-readable media include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), etc., and the computer software product includes several commands, and a computer device (which may be a personal computer, a server, a network device, etc.) is used to execute the methods described in various embodiments or parts of the embodiments.

[0090] The present application will be described with reference to flowcharts and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, and the commands executed by the processor of the computer or other programmable data processing device can generate an apparatus for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0091] These computer program commands may further be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, the commands stored in the computer-readable memory producing an article of manufacture that includes a command device that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0092] These computer program commands may be loaded into a computer or other programmable data processing device and executed on the computer or other programmable data processing device to produce a computer-implemented process, whereby the commands executed on the computer or other programmable data processing device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0093] As described above, the biometric authentication solution according to one or more embodiments of the present invention randomly transforms the least significant bits of received biometric data and performs identity verification based on the pre-processed biometric data. This solution is highly destructive to counter-noise in the image, making counter-attacks ineffective. Therefore, even if an attacker obtains the parameters of the biometric authentication model, the training process for the counter-attack model cannot stably converge, and an effective attack model cannot be formed.

[0094] Furthermore, in one or more embodiments, the addition of a multi-graph verification strategy and a blacklist rejection strategy can effectively prevent attackers from conducting black-box mode attack training. Furthermore, by adding a negative countervailing sample embedding process to the facial image collection process and a negative countervailing sample verification (removal) process to the facial recognition system, images that do not have negative countervailing samples embedded during collection are added to the countervailing samples before the authentication service performs facial recognition, thereby preventing accurate results from being obtained during the facial recognition process. The negative countervailing samples only need to be generated once, and the subsequent embedding and removal processes are computationally simple and efficient. Furthermore, facial data embedded with negative countervailing samples does not affect the human eye, and the data cannot be removed even if the data is decrypted during transmission.

[0095] Although preferred embodiments of the present application have been described, those skilled in the art may make additional modifications and amendments to these embodiments once they have grasped the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all modifications and amendments that fall within the scope of the preferred embodiments and the present application. Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations as well. [Explanation of symbols]

[0096] 1000 Identification Methods 3000 Identity Server 310 Receiving device 320 Low-order bit random filter 330 Preprocessor 340 Identification Device 350 Identification and Determination Device 360 Decoding Device 370 Verification Device

Claims

1. A method for identifying biometric features performed by an identification server, comprising: Step A of receiving biometric feature data; Step B: randomly converting the lower bits of the received biometric characteristic data; a step C of pre-processing the randomly transformed biometric feature data by randomly selecting and employing one or more filtering algorithms; and step D of performing identity verification based on the pre-processed biometric feature data, the biometric data includes facial image data; Randomly converting the data of the least significant bits of the received biometric characteristic data includes randomly converting the least significant bits of the biometric characteristic data; the biometric data further includes a timestamp; Between step A and step B, decoding the biometric data to obtain the facial image data and the timestamp; If the difference between the system time and the timestamp does not meet the requirement, feeding back an abnormality; Between step A and step B, If the difference between the system time and the timestamp satisfies a requirement, detecting a face position in the face image data; Between step A and step B, The method further includes adding a counteraction sample to the face position to counteract the effect of a negative counteraction sample added at a data collection terminal, wherein the negative counteraction sample is a negative value of the counteraction sample. Biometric feature identification methods.

2. Randomly converting the least significant bits of the biometric characteristic data includes: setting all least significant bits of the biometric feature data to 0; setting all of the least significant bits of the biometric feature data to 1; randomly rearranging the least significant bits of the biometric data; and randomly generating the least significant bits of the biometric feature data.

3. In step D, performing identity verification based on the pre-processed biometric feature data to obtain a first matching result; The method comprises: repeatedly performing steps B to D to obtain a second matching result; The method of claim 1 , further comprising: directly performing identity identification based on the received biometric feature data to obtain a third matching result.

4. performing a match check on the first match result, the second match result, and the third match result; The method of claim 3 , further comprising: feeding back the first matching result when the first matching result, the second matching result, and the third matching result all match.

5. feedback that the identification process is abnormal when there is a discrepancy between the first matching result, the second matching result, and the third matching result; The method of claim 4, further comprising: adding the collection terminal of the corresponding data to a blacklist.

6. The method of claim 1 , wherein the oppositional samples are pre-generated by training a generator on back conduction based on face recognition results.

7. a receiving device for receiving biometric feature data; a low-bit random filter for randomly converting low-bit data in the received biometric characteristic data; a preprocessor for preprocessing the randomly transformed biometric feature data by randomly selecting and employing one or more filtering algorithms; an identification device for performing identification based on the preprocessed biometric feature data; the biometric data includes facial image data; the least significant bit random filter is arranged to randomly transform the least significant bits of the biometric feature data; the biometric data further includes a timestamp; a decoding device for decoding the biometric feature data to obtain the facial image data and the timestamp; a verification device that outputs an abnormality feedback signal when the difference between the system time and the timestamp does not satisfy a requirement; the verification device is further arranged to detect a face position in the facial image data if a difference between the system time and the timestamp satisfies a requirement; The verification device is further configured to add a countervailing sample to the face position to counteract the effect of a negative countervailing sample added at the data collection terminal, wherein the negative countervailing sample is a negative value of the countervailing sample. Biometric feature identification server.

8. The lower bit random filter setting all least significant bits of the biometric feature data to 0; setting all of the least significant bits of the biometric feature data to 1; randomly rearranging the least significant bits of the biometric data; and randomly generating the least significant bits of the biometric feature data.

9. 8. The server according to claim 7, further comprising an identification determination device that checks whether the identification results obtained a plurality of times are consistent, and feeds back the identification result when all the identification results match.

10. The server according to claim 9, wherein the identification determination device is configured to, when there is a discrepancy in the identification result, feed back that the identification process is abnormal, and add the collection terminal of the corresponding data to a blacklist.

11. The server of claim 7 , wherein the oppositional samples are pre-generated by performing back conduction training on a generator based on face recognition results.

12. A client adapted to cooperate with the server of claim 7, comprising: a face collection device that collects face image data; an embedding device for detecting a face position based on the face image data and embedding a negative opposition sample at the face position; Including, The client is operable to encrypt and transmit the processed facial image data and a timestamp to the server.

13. Step A of constructing a competitive generation network including a generator and a validator; Step B: generating noise information using the generator and adding the noise information to a real face image to obtain generated face data; Step C of authenticating the generated face data using a face authentication model; Step D: training the generator in retrograde conduction based on the authentication result; Step E: repeating steps B to D until the face recognition model produces an incorrect recognition result; A method for generating a contestable sample by using a server according to any one of claims 7 to 11, the method including a step F of determining noise information corresponding to the incorrect authentication result as a contestable sample.

14. A computer storage medium containing instructions which, when executed, perform the method of any one of claims 1 to 6.

15. There is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 6.

Images