A biometric processing method, apparatus and device

Abstract

The embodiment of the specification discloses a kind of biometric processing method, device and equipment, the method includes: obtaining the biometric request of target user, the biometric request includes the user biological information of target user;The user biological information is respectively input into the encoder for carrying out privacy protection to user biological information, to carry out privacy protection processing to the user biological information by encoder, obtain the user biological information after privacy protection, encoder is obtained by first model training mode model training and / or by second model training mode model training, first model training mode is the mode that model training is carried out by successively through the training sample data containing difficult sample data and model training is carried out by multiple sets of adversarial sample data, second model training mode is the mode that model training is carried out by pre-acquired multiple adversarial features;Biometric processing is carried out to target user based on the user biological information after privacy protection.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of computer, and particularly relates to a biological recognition processing method, device and equipment. BACKGROUND

[0002] In recent years, biological recognition technology has developed rapidly, and the application of biological recognition has entered people's work and life, such as face access control in a community, face cash register in a supermarket, fingerprint unlocking of a mobile phone, etc. However, while the biological recognition system provides convenience for users, the biological recognition system needs to collect, transmit, process and store user biological information, so that the user's private information (i.e. user biological information) is in a high-risk state, and once the user's private information is leaked, the property and information security will be greatly threatened.

[0003] The privacy protection capability becomes an important capability of the biological recognition system. Generally, the privacy protection processing can be performed through information encryption. Specifically, the user biological information is encrypted or processed by row and column confusion using simple linear operations, but the above-mentioned method has simple atomic operations, single process, and is easy to be cracked by methods such as brute force cracking. In addition, the privacy protection processing can also be performed through deep learning. Specifically, the user biological information is processed through training and using a deep learning model (such as a neural network model) to obtain the user biological information after privacy protection, but since the training of the deep learning model is driven by data, the security capability of the above-mentioned deep learning model will seriously decrease for the data type of the training sample data that does not appear or has a small amount in the training sample data. Therefore, it is necessary to provide a user biological recognition technical solution with higher security and stronger privacy protection capability. SUMMARY

[0004] The technical solution of the present specification embodiment is to provide a user biological recognition technical solution with higher security and stronger privacy protection capability.

[0005] In order to achieve the above technical solution, the present specification embodiment is implemented as follows:

[0006] The method for biological recognition processing provided in the embodiments of the present specification comprises: obtaining a biological recognition request of a target user, wherein the biological recognition request comprises user biological information of the target user; inputting the user biological information into an encoder for privacy protection of user biological information respectively, so as to perform privacy protection processing on the user biological information through the encoder to obtain user biological information after privacy protection, wherein the encoder is obtained through model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a manner of sequentially performing model training through training sample data containing difficult sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner is a manner of performing model training through multiple adversarial features obtained in advance; and performing biological recognition processing on the target user based on the user biological information after privacy protection.

[0007] The biological recognition processing device provided in the embodiments of the present specification comprises: a request module configured to obtain a biological recognition request of a target user, wherein the biological recognition request comprises user biological information of the target user; a first privacy protection module configured to input the user biological information into an encoder for privacy protection of user biological information respectively, so as to perform privacy protection processing on the user biological information through the encoder to obtain user biological information after privacy protection, wherein the encoder is obtained through model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a manner of sequentially performing model training through training sample data containing difficult sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner is a manner of performing model training through multiple adversarial features obtained in advance; and a biological recognition module configured to perform biological recognition processing on the target user based on the user biological information after privacy protection.

[0008] The biological recognition processing device provided by the embodiments of the present specification comprises a processor and a memory arranged to store computer executable instructions, which, when executed, cause the processor to: acquire a biological recognition request of a target user, wherein the biological recognition request comprises user biological information of the target user; input the user biological information into an encoder for privacy protection of user biological information respectively, to perform privacy protection processing on the user biological information through the encoder, to obtain user biological information after privacy protection, wherein the encoder is obtained through model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a manner of sequentially performing model training through training sample data containing difficult sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner is a manner of performing model training through multiple adversarial features obtained in advance; and perform biological recognition processing on the target user based on the user biological information after privacy protection.

[0009] The embodiments of the present specification also provide a storage medium for storing computer executable instructions, which, when executed by a processor, implement the following processes: acquiring a biological recognition request of a target user, wherein the biological recognition request comprises user biological information of the target user; inputting the user biological information into an encoder for privacy protection of user biological information respectively, to perform privacy protection processing on the user biological information through the encoder, to obtain user biological information after privacy protection, wherein the encoder is obtained through model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a manner of sequentially performing model training through training sample data containing difficult sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner is a manner of performing model training through multiple adversarial features obtained in advance; and performing biological recognition processing on the target user based on the user biological information after privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0011] Figure 1 A biological recognition processing method embodiment of the present specification;

[0012] Figure 2This is another embodiment of the biometric processing method described in this specification;

[0013] Figure 3 This is a schematic diagram of the structure of a biometric processing system described in this specification;

[0014] Figure 4 This is yet another embodiment of a biometric processing method described in this specification;

[0015] Figure 5 This is yet another embodiment of a biometric processing method described in this specification;

[0016] Figure 6 This is an embodiment of a biometric processing device described in this specification;

[0017] Figure 7 This is an embodiment of a biometric processing device described in this specification. Detailed Implementation

[0018] This specification provides a biometric processing method, apparatus, and device through its embodiments.

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0020] Example 1

[0021] like Figure 1 As shown in the embodiments of this specification, a biometric processing method is provided. The execution subject of this method can be a terminal device or a server. The terminal device can be a mobile phone, tablet computer, or a computer device such as a laptop or desktop computer, or an IoT device (specifically, a smartwatch, in-vehicle device, etc.). The server can be a single server or a server cluster composed of multiple servers. The server can be a backend server for financial services or online shopping services, or a backend server for an application. The method specifically includes the following steps:

[0022] In step S102, a biometric request from the target user is obtained, which includes the target user's biometric information.

[0023] The target user can be any user, such as the owner of the terminal device, and the target user can initiate a biometric identification request through the terminal device. The user biometric information can include various types, such as fingerprint information, palmprint information, facial information, or iris information, etc. In actual applications, the carrier of the user biometric information can include various types, such as an image, and the specific configuration can be determined according to actual conditions, which is not limited in the embodiments of the present specification.

[0024] In practice, in recent years, biometric identification technology has developed rapidly, and biometric identification applications have entered people's work and life, such as face access control in a certain community, face cash register in a certain supermarket, and fingerprint unlocking of a mobile phone, etc. However, while the biometric identification system provides convenience for users, the biometric identification system needs to collect, transmit, process, and store user biometric information, which puts the user's privacy information (i.e., user biometric information) in a high-risk state, and once the user's privacy information is leaked, the user's property and information security will be greatly threatened.

[0025] Privacy protection capability has become an important capability of the biometric identification system. Typically, privacy protection can be performed by information encryption. Specifically, simple linear operations are used to encrypt or row-column confusion user biometric information, etc. However, the above method has simple atomic operations, single process, and is easily broken by methods such as brute force cracking. In addition, privacy protection can also be performed by deep learning. Specifically, a deep learning model (such as a neural network model) is trained and used to perform privacy protection on user biometric information to obtain privacy-protected user biometric information. However, since the training of the deep learning model is data-driven, the security capability of the deep learning model will decrease significantly for data types that do not appear or have very few training sample data in the training sample data. Therefore, a technical solution for user biometric identification with higher security and stronger privacy protection capability is needed. The embodiments of the present specification provide a feasible technical solution, which can specifically include the following contents:

[0026] When a user needs to perform a specified service (such as a payment service, a login service, etc.), the execution mechanism of the specified service can be triggered. Before executing the specified service, the identity of the user often needs to be identified. At this time, the terminal device can start the corresponding information collection component, such as a fingerprint collection component, a camera component, a palmprint collection component, etc. The user biometric information of the user can be collected through the information collection component, such as a fingerprint image, a facial image, a palmprint image, or an iris image, etc. The terminal device can generate a biometric identification request based on the collected user biometric information, and the terminal device can obtain the biometric identification request.

[0027] In step S104, the above user biological information is respectively input into an encoder for privacy protection of the user biological information, to obtain privacy-protected user biological information by privacy protection processing of the user biological information by the encoder, the encoder being obtained by model training through a first model training manner and / or model training through a second model training manner, the first model training manner being a manner of sequentially performing model training through training sample data containing difficult sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner being a manner of performing model training through multiple pre-acquired adversarial features.

[0028] The encoder can be a model for privacy protection processing of specified data, and the encoder can be constructed through multiple different algorithms, for example, the encoder can be constructed through a neural network algorithm, or the encoder can be constructed through a random forest algorithm, etc., which can be set according to actual conditions, and the present specification embodiments do not limit this. The difficult sample data can also be difficult sample data, and when the model is used to predict the label of the difficult sample data, the obtained label information has a large error (i.e., the error is greater than a preset error threshold) from the true label information. The adversarial sample data can be the same as the original sample data in visual perception, but the recognition result obtained by using a pre-trained model to recognize the adversarial sample data is different from the original sample data (or the similarity between the features of the adversarial sample data and the features of the original sample data is less than a preset threshold, i.e., the features of the adversarial sample data and the features of the original sample data are not the same or similar). The adversarial features can be features obtained by adding disturbance data to original features, and the disturbance data can include multiple types, such as specified noise data, random noise data, or other specified data, etc., which can be set according to actual conditions, and the present specification embodiments do not limit this. The training sample data can be sample data for training the encoder.

[0029] In implementation, an initial architecture of the encoder can be constructed through a preset algorithm, then user biological information of different users can be acquired and used as training sample data, wherein the training sample data can include difficult sample data and simple sample data, then a corresponding loss function can be set, the encoder can be trained using the acquired user biological information and the loss function to obtain a trained encoder, in actual application, in order to ensure the training effect of the encoder, a corresponding decoder can be constructed and trained jointly with the encoder to obtain a trained encoder. In order to improve the generalization ability of the encoder to data, the above trained encoder can be further trained using multiple groups of adversarial sample data, wherein the multiple groups of adversarial sample data can be acquired in multiple different ways, for example, multiple original sample data can be acquired, the original sample data can be input into a pre-trained adversarial network model (such as a neural network model, etc.), and corresponding adversarial sample data can be generated through the adversarial network model, through the above method, a group of adversarial sample data composed of original sample data and generated adversarial sample data can be obtained, and in this way, multiple groups of adversarial sample data can be obtained, or multiple original sample data can be acquired, then random noise data can be generated for each original sample data, the random noise data can be added to the corresponding original sample data to obtain adversarial sample data satisfying the adversarial sample property (that is, the original sample data is the same in visual perception, but the recognition result obtained by identifying the adversarial sample data through the pre-trained model is different from the original sample data (or the similarity between the features of the adversarial sample data and the features of the original sample data is less than a preset threshold, that is, the features of the adversarial sample data and the features of the original sample data are not the same or similar)), a group of adversarial sample data composed of original sample data and adversarial sample data can be obtained, and in this way, multiple groups of adversarial sample data can be obtained, etc., which can be set according to actual conditions, and the embodiments of the present specification are not limited in this regard.

[0030] In addition, the encoder can also be trained in the manner of adversarial features. Specifically, adversarial features can be acquired, which can be determined based on the features corresponding to the specified training sample data (such as adding disturbance data to the features corresponding to the specified training sample data to obtain adversarial features satisfying the adversarial sample property, etc.), or can be obtained through a pre-trained model, which can be set according to actual conditions. A network layer can be selected as a feature space in the network layer of the encoder, then the adversarial features can be input into the selected network layer, and the encoder can be trained in combination with a preset loss function to obtain a trained encoder, so that the trained encoder has high generalization ability to data.

[0031] When the biometric recognition request is acquired, the user biological information can be input into the trained encoder, and the user biological information is processed by the encoder for privacy protection, so that sensitive information (such as information of fingerprint lines, clear contour information of the face, etc.) contained in the user biological information is removed or hidden, and finally the user biological information processed for privacy protection output by the encoder can be obtained.

[0032] In step S106, the target user is processed for biometric recognition based on the user biological information processed for privacy protection.

[0033] In implementation, the determined user biological information processed for privacy protection can be calculated for similarity with the reference user biological information (which can not contain sensitive information, i.e., still desensitized information) pre-stored in the local (or server), if the obtained similarity value is greater than the preset similarity threshold, the result of the biometric recognition processing of the target user is pass, at this time, the specified business processing can be continued, if the obtained similarity value is less than the preset similarity threshold, the result of the biometric recognition processing of the target user is failure, and the specified business processing is terminated. In actual application, the above processing process is only an optional way, and various different processing ways can also be included, which can be set according to actual conditions.

[0034] The biometric recognition processing method provided by the embodiments of the present specification includes: acquiring a biometric recognition request of a target user, the biometric recognition request including user biological information of the target user; inputting the user biological information into an encoder for privacy protection of the user biological information, to process the user biological information for privacy protection by the encoder, to obtain user biological information processed for privacy protection, the encoder being obtained by model training by a first model training manner and / or model training by a second model training manner, the first model training manner being a manner of sequentially performing model training by using training sample data including difficult sample data and performing model training by using multiple sets of adversarial sample data, and the second model training manner being a manner of performing model training by using multiple pre-acquired adversarial features; and processing the target user for biometric recognition based on the user biological information processed for privacy protection. In this way, on the one hand, adversarial noise data is added to known sample data, so that the robustness and security capability of the encoder itself are stronger, and on the other hand, adversarial feature sampling processing is added in the feature space, unknown data simulating outofdistribution is mined, so that the generalization capability of the encoder for unknown data is obviously improved, so that comprehensive coverage of various types of data can be realized, so that there is no longer a long-tail problem in the data, and algorithm discrimination problem is avoided.

[0035] Embodiment two

[0036] As Figure 2As shown, the embodiment of the present specification provides a biometric processing method, the execution subject of the method can be a terminal device or a server, wherein the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, etc., can also be a computer device such as a notebook computer or a desktop computer, or can also be an IoT device (such as a smart watch, a vehicle-mounted device, etc.). The server can be an independent server, or can also be a server cluster composed of multiple servers, etc. The server can be a background server of a financial service or a network shopping service, or can also be a background server of an application program, etc. The method can specifically include the following steps:

[0037] In step S202, training sample data containing difficult sample data is obtained, and the training sample data includes user biometric information of a user.

[0038] The training sample data can include multiple training sample data, which can be composed of user biometric information of the same user, or can be composed of user biometric information of multiple different users, and can be set according to actual conditions.

[0039] In implementation, the user biometric information of the user can be obtained from multiple different users with the consent of the user, or the user biometric information of the user can be obtained from a specified database, etc. The obtained user biometric information can be classified to determine which user biometric information has a large error between the label information obtained by the model and the real label information when the model is used to predict the label, and the user biometric information can be marked as difficult sample data. The remaining user biometric information can be used as simple sample data. The difficult sample data and the simple sample data in the training sample data can be selected according to the combination requirements (such as the ratio of difficult sample data to simple sample data is 1:1 or 1:2, etc.), a certain number of difficult sample data can be selected from the above difficult sample data, and a corresponding number of simple sample data can be selected from the above simple sample data. The selected certain number of difficult sample data and the corresponding number of simple sample data can be combined into training sample data.

[0040] In step S204, the encoder, the decoder and the difficult sample discriminator are jointly trained by the training sample data and the preset first loss function to obtain the initially trained encoder, the initially trained decoder and the initially trained difficult sample discriminator. The decoder is used for restoring the training sample data after privacy protection, and the difficult sample discriminator is used for judging whether the training sample data processed by the encoder satisfies the condition corresponding to the difficult sample data.

[0041] The first loss function can be determined in various ways, for example, a corresponding loss function can be set based on the encoder, the decoder, and the difficult sample discriminator, or a loss function corresponding to the input data and the final output data can be set, or a suitable loss function can be set for the joint training according to actual conditions. The condition corresponding to the difficult sample data can refer to whether the judged object is difficult sample data, i.e., the condition that needs to be met when the judged object is difficult sample data. The condition can be, for example, that when the model is used to predict the label of the judged object, the error between the obtained label information and the real label information is large. The condition can be set according to actual conditions, and the embodiments of the present specification do not limit the condition.

[0042] In implementation, the difficult sample data and the easy sample data in the training sample data can be input into the encoder to obtain output data (i.e., privacy-protected training sample data). The decoder can be used to restore the output data, and the difficult sample discriminator can be used to determine whether the training sample data processed by the encoder meets the condition corresponding to the difficult sample data. Then, the first loss function can be used to calculate the corresponding loss value, and the convergence of the encoder, the decoder, and the difficult sample discriminator can be determined based on the calculated loss value. If the convergence is met, the initial training of the encoder, the decoder, and the difficult sample discriminator is completed, and if the convergence is not met, the training of the encoder, the decoder, and the difficult sample discriminator is continued based on the training sample data until the convergence of the encoder, the decoder, and the difficult sample discriminator is met, and the initial training of the encoder, the decoder, and the difficult sample discriminator is completed.

[0043] The specific processing mode of the step S204 can be various, and an optional processing mode is provided as follows, which can include the following steps A2 to A8.

[0044] In step A2, the training sample data is input into the encoder to obtain the privacy-protected training sample data.

[0045] In step A4, the privacy-protected training sample data is input into the decoder corresponding to the encoder to restore the privacy-protected training sample data by the decoder to obtain the reconstructed training sample data.

[0046] In step A6, the privacy-protected training sample data is input into the difficult sample discriminator to determine whether the privacy-protected training sample data meets the condition corresponding to the difficult sample data by the difficult sample discriminator to obtain the corresponding determination result.

[0047] In step A8, based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, the judgment result and the preset first loss function, it is determined whether the encoder, the decoder and the hard sample recognizer converge. If not, the training sample data containing the hard sample data is obtained to continue the model training of the encoder, the decoder and the hard sample recognizer until the encoder, the decoder and the hard sample recognizer converge, and the primary trained encoder, the primary trained decoder and the primary trained hard sample recognizer are obtained.

[0048] The first loss function is determined by the maximum value of the similarity between the privacy-protected training sample data and the training sample data, whether the privacy-protected training sample data includes the identity information of the user, and the preset classification sub-loss function, specifically, Lt=L1(I, It)+L2(It, Ir)+L3(p, y), wherein I represents the training sample data, It represents the privacy-protected training sample data, Ir represents the reconstructed training sample data, Lt represents the first loss function corresponding to the training sample data, p and y respectively represent the categories obtained after classification, L1(I, It) guarantees the privacy protection effect, so that the privacy-protected training sample data is as inconsistent as possible with the training sample data in vision, L2(I, Ir) ensures that the privacy-protected training sample data still contains identity information and other characteristics, so that the original training sample data can be recovered, and L3(p, y) can be a binary classification sub-loss function, used to distinguish whether the privacy-protected training sample data is difficult sample data (the privacy protection effect of the difficult sample data is poor, and it is easy to be attacked by others). The encoder and the decoder can be constructed based on multiple different ways, for example, can be constructed based on U-Net, the U-Net is constructed by a fully connected network, the U-Net presents a structure similar to the letter "U", which is composed of a left half compression channel (Contracting Path) and a right half expansion channel (Expansive Path), the compression channel can be constructed by a convolutional neural network, and the structure of 2 convolutional layers and 1 maximum pooling layer can be repeatedly used, and after each pooling operation, the dimension of the data is increased. In the expansion channel, first, perform 1 deconvolution operation to reduce the dimension of the data by half, then splice it to the corresponding compression channel for cutting, and the corresponding feature data can be obtained, based on the above feature data, new feature data is reconstituted, and then 2 convolutional layers are used for feature extraction, and the above structure is repeated, and in the last output layer, 2 convolutional layers are used to map the high-dimensional feature data to low-dimensional output data. The U-Net can be divided into two parts of upsampling and downsampling, the downsampling part mainly uses continuous convolution pooling layers to extract feature information in the data, and gradually maps the feature information to high dimensions, and the highest dimension of the entire network exists rich feature information in the entire data. The U-Net can not need to directly pool the data and directly upsample the output data to the same size as the original data, but through deconvolution processing, the high-dimensional features are mapped to low dimensions again. In order to enhance the accuracy of segmentation, the data with the same dimension in the downscaling network of the same dimension is fused in the mapping process. Since the dimension will become twice the original dimension in the fusion process, convolution processing is needed again to ensure that the dimension after processing is the same as the dimension before the fusion operation, so that after the deconvolution processing again, the data can be fused twice with the data of the same dimension, until the dimension of the output data is the same as the original data.The structure of the encoder and the decoder in the embodiment can be composed of a certain number of network layers of the U-Net, specifically, can be composed of the U-Net with 8 or 10 network layers, and the like, which can be set according to actual conditions. For example, the encoder and the decoder can be constructed by a multi-layer perception (MLP), which has multiple hidden layers in addition to the input layer and the output layer. The simplest MLP only contains one hidden layer, i.e., a three-layer structure. The layers of the MLP are fully connected. The bottom layer of the MLP is the input layer, the middle layer is the hidden layer, and the last layer is the output layer. The encoder and the decoder can be constructed by a three-layer MLP, which can be set according to actual conditions. The hard sample discriminator can be constructed by a specified classification algorithm, specifically, a binary classification algorithm, and the like, which can be selected according to actual conditions.

[0049] In the implementation, the training sample data can be obtained and input into the encoder to obtain the privacy-protected training sample data. The privacy-protected training sample data can be input into the decoder to obtain the reconstructed training sample data. Meanwhile, the privacy-protected training sample data can be input into the hard sample discriminator to obtain the judgment result of whether the privacy-protected training sample data is hard sample data. The decoder can also be constructed based on the U-Net or based on the MLP. The input data of the decoder is the privacy-protected training sample data, and the output data is the reconstructed original training sample data. Then, based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, and the judgment result of whether the privacy-protected training sample data is hard sample data, the first preset loss function is used to determine whether the encoder, the decoder, and the hard sample discriminator converge. If not, the training sample data is obtained to continue training the encoder, the decoder, and the hard sample discriminator until the encoder, the decoder, and the hard sample discriminator converge, thereby obtaining the initially trained encoder, the initially trained decoder, and the initially trained hard sample discriminator.

[0050] In step S206, a set of adversarial sample data is generated, and the adversarial sample data is labeled as hard sample data.

[0051] The specific processing method of generating a set of adversarial sample data can include multiple methods, which can be set according to actual conditions. An optional processing method is provided below, which can include the following steps B2 to B6.

[0052] In step B2, the first sample data is obtained.

[0053] In implementation, the first sample data can be any sample data obtained, and in actual application, the first sample data can include all or part of the sample data in the training sample data, or can be sample data different from the training sample data.

[0054] In step B4, the first sample data is input into the preliminarily trained encoder, the output data of the preliminarily trained encoder are input into the preliminarily trained decoder and the preliminarily trained hard sample discriminator respectively, and a preset optimization algorithm is used to maximize the first loss function and minimize the regularization term of the preset multi-norm constraint, to obtain the adversarial sample data corresponding to the first sample data.

[0055] The optimization algorithm can include various algorithms, such as a stochastic gradient descent (SGD) algorithm and a gradient descent algorithm, and can be set according to actual conditions.

[0056] In implementation, the adversarial sample data can be mined based on the multi-norm constraint maximized by the first loss function. Specifically, an optimization target can be preset: maximizing the first loss function, i.e., mining the encoder, decoder and hard sample discriminator that can make the existing encoder, decoder and hard sample discriminator perform poorly. The corresponding optimization constraint can be: in the process of maximizing the first loss function, the first sample data can collapse or produce abnormal sample data. To avoid the above situation, a regularization term of multi-norm constraint can be used to set the optimization constraint, which acts on the adversarial noise mask adv , so that the adversarial noise satisfies various norm constraints while generating the adversarial sample data, achieving better physical characteristics. For example, the regularization term of 1-norm constraint requires the adversarial noise to be sparse, the regularization term of 2-norm constraint requires the adversarial noise to be smooth, and the regularization term of multi-norm constraint requires the noise to have multiple excellent characteristics (such as sparsity and smoothness of the adversarial noise). For details, see the regularization term of multi-norm constraint below:

[0057]

[0058] Specifically, the first sample data can be input into the preliminarily trained encoder, the output data of the preliminarily trained encoder can be input into the preliminarily trained decoder and the preliminarily trained hard sample discriminator respectively, and the first loss function can be maximized and the regularization term of the multi-norm constraint can be minimized by using an algorithm such as the stochastic gradient descent algorithm. The obtained gradient information is back-propagated to the first sample data to obtain mask adv . Finally, the obtained adversarial sample data can be as follows:

[0059] I adv = I + mask adv

[0060] wherein, I adv represents the adversarial sample data, I represents the first sample data.

[0061] In step B6, a set of adversarial sample data is generated based on the adversarial sample data corresponding to the first sample data and the first sample data.

[0062] In step S208, the model training of the encoder, the decoder and the difficult sample discriminator is continued using the adversarial sample data, and it is judged whether the judgment result output by the difficult sample discriminator meets the preset stability condition. If not, a set of adversarial sample data is generated again, and the model training of the encoder, the decoder and the difficult sample discriminator is continued using the adversarial sample data generated again until the accuracy of the judgment result output by the difficult sample discriminator meets the preset stability condition, and the encoder after retraining is obtained.

[0063] The stability condition can include various conditions, for example, the accuracy of the judgment result output by the difficult sample discriminator changes within a preset accuracy range, or the accuracy of the judgment result output by the difficult sample discriminator tends to a stable value, etc. The specific condition can be set according to the actual situation.

[0064] In implementation, the encoder, the decoder and the difficult sample discriminator can be trained using the adversarial sample data through the training process of steps A2-A8, and it is judged whether the accuracy of the judgment result output by the difficult sample discriminator meets the preset stability condition. If yes, the encoder after retraining can be obtained, and if no, a set of adversarial sample data can be generated again using the processing of steps B2-B6, and the model training of the encoder, the decoder and the difficult sample discriminator is continued using the adversarial sample data generated again until the accuracy of the judgment result output by the difficult sample discriminator meets the preset stability condition, and the encoder after retraining is obtained.

[0065] In actual application, perturbation data can also be made in the dimension of sample data to mine corresponding adversarial sample data. The above method is simple and easy to operate, but the generated adversarial sample data has high similarity with the original sample data, so the improvement of the generalization ability of the encoder to data is very limited. Therefore, a method of adversarial feature mining in feature space is provided, so that the encoder and other models have better generalization coverage to the entire feature space, thereby fundamentally solving the generalization problem. For details, please refer to the following related content.

[0066] In step S210, the adversarial feature corresponding to the feature included in the second sample data is obtained, and the adversarial feature is a feature determined based on the feature included in the second sample data and the first loss function.

[0067] The second sample data can be any sample data obtained in actual application. The second sample data can include all or part of the training sample data or the first sample data, or can be sample data different from the training sample data or the first sample data.

[0068] In implementation, the feature extraction algorithm can be preset, and the feature extraction algorithm can be used to extract features from the obtained second sample data. Alternatively, the obtained second sample data can be input into the trained encoder, and the feature extraction part in the trained encoder can be used to extract features from the obtained second sample data. Noise data can be obtained, and the noise data can be added to the features contained in the second sample data to obtain processed features. The processed features can be input into the encoder, and the encoder, the difficult sample discriminator, and the first loss function can be used for joint training to select the adversarial features corresponding to the features contained in the second sample data that meet the preset requirements.

[0069] The specific processing mode of the step S210 can be various, and an optional processing mode is provided below. The processing can include the following steps C2 to C8.

[0070] In step C2, the second sample data is obtained.

[0071] In step C4, the target noise data is collected from the noise data meeting the preset distribution, and the target noise data is added to the features contained in the second sample data to obtain noise features.

[0072] The preset distribution can include various distributions, such as Gaussian distribution, and can be set according to actual conditions. The embodiments of the present application do not limit this.

[0073] In implementation, the feature space of the adversarial features can be selected from the intermediate network layer of the encoder. The intermediate network layer can be any network layer in the encoder, and the resolution of the intermediate network layer can be [C, H, W], where C, H, and W are resolution values of three dimensions in the resolution. The mean vector can be 0, the covariance matrix can be a diagonal matrix, and a Gaussian distribution with the same dimension as the features (or adversarial features) contained in the second sample data can be selected as the preset distribution. The noise data can be randomly sampled from the noise data of the Gaussian distribution to obtain the target noise data, and the target noise data can be superimposed on the features contained in the second sample data to obtain noise features.

[0074] In step C6, the noisy feature is input into the encoder, and the data output by the encoder is input into the decoder and the hard sample discriminator respectively to obtain output data of the decoder and output data of the hard sample discriminator, and based on the output data of the decoder and the output data of the hard sample discriminator, a corresponding loss value is calculated through the first loss function.

[0075] In implementation, the forward propagation of the network composed of the encoder, the decoder and the hard sample discriminator can be performed by using the noisy feature, that is, the noisy feature is input into the encoder, and the data output by the encoder is input into the decoder and the hard sample discriminator respectively to obtain output data of the decoder and output data of the hard sample discriminator, and based on the output data of the decoder and the output data of the hard sample discriminator, a corresponding loss value is calculated through the first loss function.

[0076] In step C8, based on the calculated loss value and the loss value corresponding to the feature contained in the second sample data, the adversarial feature corresponding to the feature contained in the second sample data is determined.

[0077] In implementation, a threshold value can be preset, and the difference between the loss value obtained above and the loss value corresponding to the feature contained in the second sample data (that is, the difference of the loss function before and after adding the noise data) can be calculated, and if the difference is greater than the threshold value, the noisy feature can be identified as the adversarial feature corresponding to the feature contained in the second sample data. The above processing can be performed on each second sample data in the above manner to obtain the adversarial feature corresponding to the feature contained in each second sample data.

[0078] In step S212, the second sample data is input into the encoder or the retrained encoder to obtain the feature of the preset network layer in the encoder or the retrained encoder.

[0079] The preset network layer can be the network layer corresponding to the selected feature space.

[0080] In step S214, the adversarial feature is input into the preset network layer in the encoder or the retrained encoder to obtain the privacy-protected second sample data.

[0081] In step S216, the feature corresponding to the privacy-protected second sample data is input into the decoder or the retrained decoder to obtain the reconstructed feature of the preset network layer in the encoder or the retrained encoder.

[0082] In step S218, the privacy-protected second sample data is input into the hard sample discriminator to obtain a corresponding judgment result.

[0083] In step S220, based on the second sample data, the privacy-protected second sample data, the features of the preset network layer in the encoder or the retrained encoder, the reconstructed features of the preset network layer in the encoder or the retrained encoder, the judgment result and the preset second loss function, it is determined whether the encoder, the decoder and the difficult sample discriminator converge, and if not, the features corresponding to the adversarial features contained in the third sample data continue to be used for model training of the encoder, the decoder and the difficult sample discriminator until the encoder, the decoder and the difficult sample discriminator converge, and the retrained encoder is obtained.

[0084] The specific processes of steps S212 to S220 can refer to the related content of the above model training, which will not be described here again.

[0085] Since the sampling is performed on the adversarial features rather than the sample data, it is necessary to redesign the corresponding loss function so that the privacy protection can be trained at the adversarial feature level. Therefore, the second loss function can be as follows: the second loss function is determined by the maximum value of the similarity between the privacy-protected second sample data and the second sample data, whether the features corresponding to the privacy-protected second sample data include the features of the preset network layer in the encoder or the retrained encoder, and the preset classification sub-loss function.

[0086] Specifically, Ls = L4(I, It) + L5(ft, fr) + L6(p, y), where ft represents the features corresponding to the privacy-protected second sample data, fr represents the reconstructed features, Ls represents the second loss function, p and y represent the categories obtained after classification respectively, L4(I, It) ensures the effect of privacy protection, so that the privacy-protected training sample data is as inconsistent as possible with the training sample data in vision, L5(ft, fr) ensures that the features corresponding to the privacy-protected second sample data still contain the features of the preset network layer in the encoder or the retrained encoder, so that the features of the preset network layer in the encoder or the retrained encoder can be recovered, and L6(p, y) can be a binary classification sub-loss function, which is used to distinguish whether the privacy-protected second sample data is difficult sample data.

[0087] The processes of steps S202 to S220 can be completed by the server or the terminal device, and the specific process can be set according to the actual situation.

[0088] In step S222, the biological recognition request of the target user is obtained, and the biological recognition request of the target user includes the user biological information of the target user.

[0089] In the implementation, the server can obtain the biological recognition request of the target user from the terminal device of the target user, and the system architecture diagram can refer to Figure 3As shown, the terminal device of the target user can directly obtain the biometric recognition request triggered by the target user, the server can also directly obtain the biometric recognition request triggered by the target user, and the like, which can be set according to actual conditions.

[0090] In step S224, the user biological information is respectively input into the encoder for privacy protection of the user biological information, so as to perform privacy protection processing on the user biological information by the encoder, and obtain the user biological information after privacy protection.

[0091] In step S226, based on the pre-stored reference user biological information and the user biological information after privacy protection, the target user is subjected to biometric recognition processing, and a biometric recognition result of the target user is obtained.

[0092] In actual application, in addition to the above-mentioned manner based on the user biological information after privacy protection, the target user can also be subjected to biometric recognition processing in other various manners, and two optional processing manners are provided as follows, which can include manner one and manner two:

[0093] Manner one: for the case that the server obtains the biometric recognition request of the target user from the terminal device of the target user, as shown in Figure 4 The processing of steps S202-S220 can be completed by the server, then the server obtains the biometric recognition request of the target user from the terminal device of the target user, the server can perform the processing of steps S222-S226 to obtain the biometric recognition result of the target user, and the server can send the biometric recognition result to the terminal device of the target user.

[0094] Manner two: as shown in Figure 5 The processing of steps S202-S220 can be completed by the server, then the server can send the trained encoder to the terminal device, after the terminal device receives the encoder, the processing of steps S222-S224 can be performed to obtain the user biological information after privacy protection, then the terminal device can no longer perform the processing of step S226, but perform the processing of steps D2 and D4 as follows:

[0095] In step D2, the user biological information after privacy protection is sent to the server, and the user biological information after privacy protection is used to trigger the server to perform biometric recognition processing on the target user based on the pre-stored reference user biological information and the user biological information after privacy protection.

[0096] In step D4, the biometric recognition result of the target user sent by the server is received.

[0097] Or, the above steps S202 to S220 can be completed by the server, and then the server can send the trained encoder to the terminal device. After the terminal device receives the encoder, it can perform the above steps S222 to S226.

[0098] The embodiment of the present specification provides a kind of biological identification processing method, by obtaining the biological identification request of target user, the user biological information of target user is included in the biological identification request;The user biological information is input into the encoder for carrying out privacy protection to user biological information respectively, to obtain the user biological information after privacy protection by the privacy protection processing of encoder to user biological information, encoder is obtained by first model training mode model training and / or by second model training mode model training, first model training mode is the mode that model training is carried out by successively through the training sample data containing difficult sample data and through multiple sets of adversarial sample data, second model training mode is the mode that model training is carried out by pre-acquired multiple adversarial features;Target user is carried out biological identification processing based on the user biological information after privacy protection, in this way, on the one hand, known sample data is added with adversarial noise data, so that the robustness and security ability of encoder itself is stronger, on the other hand, increase adversarial feature sampling processing in feature space, dig out of distribution unknown data, so that the generalization ability of encoder to unknown data is obviously improved, so as to realize the comprehensive coverage of various different types of data, so that there is no longer long tail problem in data, avoid the emergence of algorithm discrimination problem.

[0099] Embodiment three

[0100] Based on the same idea, the embodiment of the present specification also provides a biological identification processing device, as shown in Figure 6 .

[0101] The biological identification processing device includes request module 601, first privacy protection module 602 and biological identification module 603, wherein:

[0102] Request module 601 obtains the biological identification request of target user, and the biological identification request includes the user biological information of the target user;

[0103] The first privacy protection module 602 inputs the user biological information into an encoder for privacy protection of user biological information respectively, to obtain privacy-protected user biological information by privacy protection processing of the user biological information by the encoder, and the encoder is obtained by model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a model training manner by sequentially passing through training sample data containing difficult sample data and model training by multiple sets of adversarial sample data, and the second model training manner is a model training manner by multiple pre-acquired adversarial features;

[0104] The biological recognition module 603 performs biological recognition processing on the target user based on the privacy-protected user biological information.

[0105] In the embodiments of the present specification, the biological recognition module 603 comprises:

[0106] The information sending unit sends the privacy-protected user biological information to the server, and the privacy-protected user biological information is used to trigger the server to perform biological recognition processing on the target user based on the pre-stored reference user biological information and the privacy-protected user biological information;

[0107] The result receiving module receives the biological recognition result of the server for biological recognition processing on the target user.

[0108] In the embodiments of the present specification, the biological recognition module 603 comprises:

[0109] The biological recognition unit performs biological recognition processing on the target user based on the pre-stored reference user biological information and the privacy-protected user biological information, to obtain a biological recognition result of the target user;

[0110] The result sending unit sends the biological recognition result to the terminal device of the target user.

[0111] In the embodiments of the present specification, the apparatus further comprises:

[0112] The training sample acquisition module acquires training sample data containing difficult sample data, and the training sample data includes user biological information of a user;

[0113] The initial training module jointly trains the encoder, the decoder and the difficult sample recognizer based on the training sample data and a preset first loss function, to obtain an initial trained encoder, an initial trained decoder and an initial trained difficult sample recognizer, the decoder is configured to restore the privacy-protected training sample data, and the difficult sample recognizer is configured to determine whether the training sample data processed by the encoder meets a condition corresponding to the difficult sample data;

[0114] The adversarial sample generation module generates a set of adversarial sample data and labels the adversarial sample data as difficult sample data.

[0115] The retraining module continues to train the encoder, the decoder and the difficult sample recognizer using the adversarial sample data, and determines whether the accuracy of the determination result output by the difficult sample recognizer meets a preset stability condition, if not, a set of adversarial sample data is generated again, and the encoder, the decoder and the difficult sample recognizer are trained using the re-generated adversarial sample data until the accuracy of the determination result output by the difficult sample recognizer meets the stability condition, to obtain a retrained encoder.

[0116] In the embodiments of the present specification, the initial training module comprises:

[0117] The encoding unit inputs the training sample data into the encoder to obtain privacy-protected training sample data.

[0118] The decoding unit inputs the privacy-protected training sample data into the decoder corresponding to the encoder to restore the privacy-protected training sample data by the decoder to obtain reconstructed training sample data.

[0119] The difficult sample recognition unit inputs the privacy-protected training sample data into the difficult sample recognizer to determine whether the privacy-protected training sample data meets a condition corresponding to the difficult sample data by the difficult sample recognizer to obtain a corresponding determination result.

[0120] The initial training unit determines whether the encoder, the decoder and the difficult sample recognizer converge based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, the determination result and a preset first loss function, if not, the training sample data containing the difficult sample data is obtained to continue training the encoder, the decoder and the difficult sample recognizer until the encoder, the decoder and the difficult sample recognizer converge, to obtain an initial trained encoder, an initial trained decoder and an initial trained difficult sample recognizer.

[0121] In the embodiments of the present specification, the first loss function is determined by a maximum value of a similarity between the privacy-protected training sample data and the training sample data, a feature of whether the privacy-protected training sample data includes the identity information of the user, and a preset classification sub-loss function.

[0122] In the embodiments of the present specification, the adversarial sample generation module comprises:

[0123] The first sample acquisition unit acquires first sample data.

[0124] The adversarial sample determination unit inputs the first sample data into the initially trained encoder, the output data of the initially trained encoder are input into the initially trained decoder and the initially trained hard sample discriminator respectively, and a preset optimization algorithm is used to maximize the first loss function and minimize a preset multi-norm constraint regularization term, to obtain adversarial sample data corresponding to the first sample data.

[0125] The adversarial sample generation unit generates a group of adversarial sample data based on the adversarial sample data corresponding to the first sample data and the first sample data.

[0126] In the embodiments of the present specification, the method further comprises:

[0127] The adversarial feature acquisition module acquires adversarial features corresponding to features contained in the second sample data, the adversarial features being determined based on the features contained in the second sample data and the first loss function.

[0128] The feature extraction module inputs the second sample data into the encoder or the retrained encoder, to obtain features of a preset network layer in the encoder or the retrained encoder.

[0129] The encoding module inputs the adversarial features into the preset network layer in the encoder or the retrained encoder, to obtain privacy-protected second sample data.

[0130] The decoding module inputs the privacy-protected second sample data into the decoder or the retrained decoder, to obtain reconstructed features of the preset network layer in the encoder or the retrained encoder.

[0131] The discrimination module inputs the privacy-protected second sample data into the hard sample discriminator, to obtain a corresponding judgment result.

[0132] The continuing training module determines, based on the second sample data, the privacy-protected second sample data, the features corresponding to the privacy-protected second sample data, the reconstructed features of the preset network layer in the encoder or the retrained encoder, the judgment result, and a preset second loss function, whether the encoder, the decoder, and the difficult sample discriminator converge. If not, the model training of the encoder, the decoder, and the difficult sample discriminator continues to be performed on the adversarial features corresponding to the features included in the third sample data until the encoder, the decoder, and the difficult sample discriminator converge, and a continuing trained encoder is obtained.

[0133] In an embodiment of the present specification, the second loss function is determined by the maximum value of the similarity between the privacy-protected second sample data and the second sample data, whether the features of the preset network layer in the encoder or the retrained encoder are included in the features corresponding to the privacy-protected second sample data, and a preset classification sub-loss function.

[0134] In an embodiment of the present specification, the adversarial feature acquisition module comprises:

[0135] The second sample acquisition unit acquires second sample data.

[0136] The noise adding unit collects target noise data from noise data satisfying a preset distribution, adds the target noise data to the features included in the second sample data to obtain noisy features.

[0137] The loss determination unit inputs the noisy features into the encoder, and inputs the data output by the encoder into the decoder and the difficult sample discriminator, respectively, to obtain the output data of the decoder and the output data of the difficult sample discriminator. Based on the output data of the decoder and the output data of the difficult sample discriminator, the loss value corresponding to the loss value calculated by the first loss function is calculated.

[0138] The adversarial feature acquisition unit determines the adversarial features corresponding to the features included in the second sample data based on the calculated loss value and the loss value corresponding to the features included in the second sample data.

[0139] The embodiment of the present specification provides a kind of biological recognition processing device, by obtaining the biological recognition request of target user, the user biological information of target user is included in the biological recognition request;The user biological information is input into the encoder for privacy protection to the user biological information respectively, to carry out privacy protection processing to the user biological information by encoder, obtain the user biological information after privacy protection, encoder is obtained by the first model training mode model training and / or by the second model training mode model training, the first model training mode is the way of model training by successively through the training sample data containing difficult sample data and through multiple sets of adversarial sample data, the second model training mode is the way of model training by pre-acquired multiple adversarial features;Target user is carried out biological recognition processing based on the user biological information after privacy protection, in this way, on the one hand, adversarial noise data is added on known sample data, so that the robustness and security capability of encoder itself is stronger, on the other hand, increase adversarial feature sampling processing in feature space, excavate the unknown data of outofdistribution simulation, so that the generalization ability of encoder to unknown data is obviously improved, so that the comprehensive coverage of various different types of data can be realized, so that there is no longer middle and long tail problem in data, avoid the emergence of algorithm discrimination problem.

[0140] Embodiment four

[0141] The biological recognition processing device provided by the embodiment of the present specification is based on the same idea, and the biological recognition processing device provided by the embodiment of the present specification is shown in the following Figure 7

[0142] The biological recognition processing device can be terminal device or server provided by the above-mentioned embodiment or the like.

[0143] The biological recognition processing device can have great difference due to different configurations or performances, and can include one or more processors 701 and memories 702, and one or more storage applications or data can be stored in the memories 702. The memory 702 can be temporary storage or persistent storage. The application stored in the memory 702 can include one or more modules (not shown in the figure), each module can include a series of computer executable instructions in the biological recognition processing device. Further, the processor 701 can be configured to communicate with the memory 702, and execute a series of computer executable instructions in the memory 702 on the biological recognition processing device. The biological recognition processing device can also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input / output interfaces 705, and one or more keyboards 706.

[0144] ​In particular in this embodiment, the biometric processing device comprises a memory, and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can comprise one or more modules, and each module can comprise a series of computer executable instructions in the biometric processing device, and the one or more programs configured to be executed by one or more processors include computer executable instructions for:

[0145] obtaining a biometric request of a target user, wherein the biometric request comprises user biometric information of the target user;

[0146] inputting the user biometric information into an encoder for privacy protection of user biometric information respectively, to obtain privacy-protected user biometric information by privacy protection processing of the user biometric information by the encoder, wherein the encoder is obtained by model training by a first model training manner and / or model training by a second model training manner, the first model training manner is a manner of sequentially performing model training by training sample data comprising difficult sample data and performing model training by multiple sets of adversarial sample data, and the second model training manner is a manner of performing model training by multiple pre-obtained adversarial features;

[0147] performing biometric processing on the target user based on the privacy-protected user biometric information.

[0148] In the embodiments of the present specification, the biometric processing on the target user based on the privacy-protected user biometric information comprises:

[0149] sending the privacy-protected user biometric information to a server, wherein the privacy-protected user biometric information is used to trigger the server to perform biometric processing on the target user based on pre-stored reference user biometric information and the privacy-protected user biometric information;

[0150] receiving a biometric result of biometric processing on the target user sent by the server.

[0151] In the embodiments of the present specification, the biometric processing on the target user based on the privacy-protected user biometric information comprises:

[0152] performing biometric processing on the target user based on pre-stored reference user biometric information and the privacy-protected user biometric information, to obtain a biometric result of biometric processing on the target user;

[0153] sending the biometric result to a terminal device of the target user.

[0154] In the embodiments of the present specification, the following are further included:

[0155] Obtaining training sample data containing difficult sample data, wherein the training sample data includes user biological information of a user;

[0156] Jointly training the encoder, the decoder and the difficult sample discriminator through the training sample data and a preset first loss function to obtain a first trained encoder, a first trained decoder and a first trained difficult sample discriminator, wherein the decoder is configured to perform restoration processing on the privacy-protected training sample data, and the difficult sample discriminator is configured to determine whether the training sample data processed by the encoder satisfies a condition corresponding to the difficult sample data;

[0157] Generating a set of adversarial sample data and marking the adversarial sample data as difficult sample data;

[0158] Continuing to train the encoder, the decoder and the difficult sample discriminator using the adversarial sample data, and determining whether the accuracy of the determination result output by the difficult sample discriminator satisfies a preset stability condition, if not, generating a set of adversarial sample data again, and continuing to train the encoder, the decoder and the difficult sample discriminator using the adversarially generated sample data again until the accuracy of the determination result output by the difficult sample discriminator satisfies the stability condition, and obtaining a second trained encoder.

[0159] In the embodiments of the present specification, the jointly training the encoder, the decoder and the difficult sample discriminator through the training sample data and a preset first loss function to obtain a first trained encoder, a first trained decoder and a first trained difficult sample discriminator comprises:

[0160] Inputting the training sample data into the encoder to obtain privacy-protected training sample data;

[0161] Inputting the privacy-protected training sample data into a decoder corresponding to the encoder to perform restoration processing on the privacy-protected training sample data by the decoder to obtain reconstructed training sample data;

[0162] Inputting the privacy-protected training sample data into a difficult sample discriminator to determine whether the privacy-protected training sample data satisfies a condition corresponding to the difficult sample data by the difficult sample discriminator to obtain a corresponding determination result;

[0163] determine whether the encoder, the decoder and the hard sample discriminator converge based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, the judgment result and a preset first loss function, if not, continue to train the encoder, the decoder and the hard sample discriminator by obtaining training sample data containing hard sample data until the encoder, the decoder and the hard sample discriminator converge, to obtain the initially trained encoder, the initially trained decoder and the initially trained hard sample discriminator.

[0164] In the embodiments of the present specification, the first loss function is determined by the maximum value of the similarity between the privacy-protected training sample data and the training sample data, the feature of whether the privacy-protected training sample data includes the identity information of the user, and a preset classification sub-loss function.

[0165] In the embodiments of the present specification, the generating a set of adversarial sample data comprises:

[0166] obtaining first sample data;

[0167] inputting the first sample data into the initially trained encoder, inputting the output data of the initially trained encoder into the initially trained decoder and the initially trained hard sample discriminator respectively, and using a preset optimization algorithm to maximize the first loss function and minimize a preset multi-norm constraint regularization term, to obtain adversarial sample data corresponding to the first sample data;

[0168] generating a set of adversarial sample data based on the adversarial sample data corresponding to the first sample data and the first sample data.

[0169] In the embodiments of the present specification, it also comprises:

[0170] obtaining adversarial features corresponding to features contained in second sample data, the adversarial features being determined based on the features contained in the second sample data and the first loss function;

[0171] inputting the second sample data into the encoder or the retrained encoder to obtain features of a preset network layer in the encoder or the retrained encoder;

[0172] inputting the adversarial features into the preset network layer in the encoder or the retrained encoder to obtain privacy-protected second sample data;

[0173] input the second sample data after privacy protection into the decoder or the decoder after retraining to obtain reconstructed features of a preset network layer in the encoder or the encoder after retraining;

[0174] input the second sample data after privacy protection into the hard sample discriminator to obtain a corresponding judgment result;

[0175] based on the second sample data, the second sample data after privacy protection, the features corresponding to the second sample data after privacy protection, the reconstructed features of the preset network layer in the encoder or the encoder after retraining, the judgment result and a preset second loss function, determine whether the encoder, the decoder and the hard sample discriminator converge, if not, continue to perform model training on the encoder, the decoder and the hard sample discriminator on the adversarial features corresponding to the features contained in the third sample data, until the encoder, the decoder and the hard sample discriminator converge, to obtain the encoder after retraining.

[0176] In the embodiments of the present specification, the second loss function is determined by the maximum value of the similarity between the second sample data after privacy protection and the second sample data, whether the features corresponding to the second sample data after privacy protection include the features of the preset network layer in the encoder or the encoder after retraining, and a preset classification sub-loss function.

[0177] In the embodiments of the present specification, the obtaining of the adversarial features corresponding to the features contained in the second sample data comprises:

[0178] obtaining second sample data;

[0179] collecting target noise data from noise data satisfying a preset distribution, adding the target noise data to the features contained in the second sample data to obtain noisy features;

[0180] input the noisy features into the encoder, and input the data output by the encoder into the decoder and the hard sample discriminator respectively to obtain output data of the decoder and output data of the hard sample discriminator, and calculate a corresponding loss value through the first loss function based on the output data of the decoder and the output data of the hard sample discriminator;

[0181] determine the adversarial features corresponding to the features contained in the second sample data based on the calculated loss value and the loss value corresponding to the features contained in the second sample data.

[0182] The embodiment of the present specification provides a kind of biological recognition processing equipment, by obtaining the biological recognition request of target user, the user biological information of target user is included in the biological recognition request;The user biological information is respectively input into the encoder for carrying out privacy protection to user biological information, to carry out privacy protection processing to the user biological information by encoder, obtain the user biological information after privacy protection, encoder is obtained by first model training mode model training and / or by second model training mode model training, first model training mode is the mode that model training is successively carried out through the training sample data containing difficult sample data and model training is carried out through multiple sets of adversarial sample data, second model training mode is the mode that model training is carried out through multiple adversarial features obtained in advance;The biological recognition processing of target user is carried out based on the user biological information after privacy protection, in this way, on the one hand, adversarial noise data is added on known sample data, so that the robustness and security capability of encoder itself is stronger, on the other hand, adversarial feature sampling processing is increased in feature space, unknown data that simulates outofdistribution is mined, so that the generalization ability of encoder to unknown data is obviously improved, so that comprehensive coverage to various different types of data can be realized, so that there is no longer long tail problem in data, and algorithm discrimination problem is avoided.

[0183] Embodiment five

[0184] Further, based on the above Figures 1 to 5 The one or more embodiments of the present specification also provide a storage medium for storing computer executable instruction information, in a specific embodiment, the storage medium can be U disk, optical disc, hard disk and the like, and the computer executable instruction information stored in the storage medium can realize the following process when being executed by processor:

[0185] Obtain the biological recognition request of target user, the user biological information of target user is included in the biological recognition request;

[0186] The user biological information is respectively input into the encoder for carrying out privacy protection to user biological information, to carry out privacy protection processing to the user biological information by encoder, obtain the user biological information after privacy protection, the encoder is obtained by first model training mode model training and / or by second model training mode model training, the first model training mode is the mode that model training is successively carried out through the training sample data containing difficult sample data and model training is carried out through multiple sets of adversarial sample data, the second model training mode is the mode that model training is carried out through multiple adversarial features obtained in advance;

[0187] The biological recognition processing of target user is carried out based on the user biological information after privacy protection.

[0188] In the embodiments of the present specification, the biological recognition processing of the target user based on the privacy-protected user biological information comprises:

[0189] The privacy-protected user biological information is sent to a server, and the privacy-protected user biological information is used to trigger the server to perform biological recognition processing on the target user based on pre-stored reference user biological information and the privacy-protected user biological information.

[0190] The biological recognition result of the biological recognition processing of the target user sent by the server is received.

[0191] In the embodiments of the present specification, the biological recognition processing of the target user based on the privacy-protected user biological information comprises:

[0192] Based on the pre-stored reference user biological information and the privacy-protected user biological information, biological recognition processing is performed on the target user to obtain a biological recognition result of the biological recognition processing of the target user.

[0193] The biological recognition result is sent to the terminal device of the target user.

[0194] In the embodiments of the present specification, it further comprises:

[0195] Obtain training sample data containing difficult sample data, wherein the training sample data includes user biological information of a user;

[0196] Jointly train the encoder, the decoder and the difficult sample recognizer through the training sample data and a preset first loss function to obtain a primary trained encoder, a primary trained decoder and a primary trained difficult sample recognizer, wherein the decoder is used for restoration processing on the privacy-protected training sample data, and the difficult sample recognizer is used for judging whether the training sample data processed by the encoder satisfies the condition corresponding to the difficult sample data.

[0197] Generate a set of adversarial sample data, and mark the adversarial sample data as difficult sample data.

[0198] continue training the encoder, the decoder and the hard sample discriminator using the generated set of adversarial sample data, and determine whether the accuracy of the determination result output by the hard sample discriminator meets a preset stability condition, if not, generate another set of adversarial sample data, and continue training the encoder, the decoder and the hard sample discriminator using the generated set of adversarial sample data, until the accuracy of the determination result output by the hard sample discriminator meets the stability condition, and obtain the retrained encoder.

[0199] In the embodiments of the present specification, the joint training of the encoder, the decoder and the hard sample discriminator through the training sample data and the preset first loss function to obtain the initially trained encoder, the initially trained decoder and the initially trained hard sample discriminator comprises:

[0200] inputting the training sample data into the encoder to obtain privacy-protected training sample data;

[0201] inputting the privacy-protected training sample data into the decoder corresponding to the encoder to restore the privacy-protected training sample data through the decoder to obtain reconstructed training sample data;

[0202] inputting the privacy-protected training sample data into the hard sample discriminator to determine whether the privacy-protected training sample data meets the condition corresponding to the hard sample data through the hard sample discriminator to obtain a corresponding determination result;

[0203] based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, the determination result and the preset first loss function, determining whether the encoder, the decoder and the hard sample discriminator converge, if not, obtaining training sample data containing hard sample data to continue training the encoder, the decoder and the hard sample discriminator, until the encoder, the decoder and the hard sample discriminator converge, and obtain the initially trained encoder, the initially trained decoder and the initially trained hard sample discriminator.

[0204] In the embodiments of the present specification, the first loss function is determined by the maximum value of the similarity between the privacy-protected training sample data and the training sample data, the feature of whether the privacy-protected training sample data includes the identity information of the user, and a preset classification sub-loss function.

[0205] In the embodiments of the present specification, the generating a set of adversarial sample data comprises:

[0206] obtaining first sample data;

[0207] inputting the first sample data into the pre-trained encoder, inputting output data of the pre-trained encoder into the pre-trained decoder and the pre-trained hard sample discriminator respectively, and maximizing the first loss function and minimizing a preset regularization term of a multi-norm constraint using a preset optimization algorithm to obtain adversarial sample data corresponding to the first sample data;

[0208] generating a group of adversarial sample data based on the adversarial sample data corresponding to the first sample data and the first sample data.

[0209] In the embodiments of the present specification, the following are further included:

[0210] obtaining adversarial features corresponding to features contained in second sample data, the adversarial features being determined based on the features contained in the second sample data and the features determined based on the first loss function;

[0211] inputting the second sample data into the encoder or the re-trained encoder to obtain features of a preset network layer in the encoder or the re-trained encoder;

[0212] inputting the adversarial features into the preset network layer in the encoder or the re-trained encoder to obtain privacy-protected second sample data;

[0213] inputting the privacy-protected second sample data into the decoder or the re-trained decoder to obtain reconstructed features of the preset network layer in the encoder or the re-trained encoder;

[0214] inputting the privacy-protected second sample data into the hard sample discriminator to obtain a corresponding judgment result;

[0215] determining whether the encoder, the decoder and the hard sample discriminator converge based on the second sample data, the privacy-protected second sample data, the features corresponding to the privacy-protected second sample data, the reconstructed features of the preset network layer in the encoder or the re-trained encoder, the judgment result and a preset second loss function, and if not, continuing to perform model training on the encoder, the decoder and the hard sample discriminator using adversarial features corresponding to features contained in third sample data until the encoder, the decoder and the hard sample discriminator converge to obtain a re-trained encoder.

[0216] In the embodiments of the present specification, the second loss function is determined by the maximum value of the similarity between the privacy-protected second sample data and the second sample data, whether the feature corresponding to the privacy-protected second sample data includes the feature of the preset network layer in the encoder or the retrained encoder, and a preset classification sub-loss function.

[0217] In the embodiments of the present specification, the obtaining of the adversarial feature corresponding to the feature included in the second sample data comprises:

[0218] Obtaining second sample data;

[0219] Collecting target noise data from noise data satisfying a preset distribution, adding the target noise data to the feature included in the second sample data to obtain a noisy feature;

[0220] Inputting the noisy feature into the encoder, and inputting the data output by the encoder into the decoder and the hard sample discriminator respectively to obtain output data of the decoder and output data of the hard sample discriminator, and calculating a corresponding loss value through the first loss function based on the output data of the decoder and the output data of the hard sample discriminator;

[0221] Based on the calculated loss value and the loss value corresponding to the feature included in the second sample data, the adversarial feature corresponding to the feature included in the second sample data is determined.

[0222] The embodiments of the present specification provide a storage medium, by obtaining a biological recognition request of a target user, the biological recognition request including user biological information of the target user; inputting the user biological information into an encoder for privacy protection of user biological information, to perform privacy protection processing on the user biological information through the encoder to obtain privacy-protected user biological information, the encoder being obtained by model training through a first model training manner and / or model training through a second model training manner, the first model training manner being a manner of sequentially performing model training through training sample data including hard sample data and performing model training through multiple sets of adversarial sample data, and the second model training manner being a manner of performing model training through multiple adversarial features obtained in advance; and performing biological recognition processing on the target user based on the privacy-protected user biological information. In this way, on the one hand, the adversarial noise data is added to the known sample data, so that the robustness and security capability of the encoder itself are stronger, and on the other hand, the adversarial feature sampling processing is increased in the feature space, the unknown data simulating outofdistribution is mined, so that the generalization capability of the encoder for unknown data is obviously improved, so as to realize comprehensive coverage of various types of data, so that there is no longer a medium and long tail problem in the data, and the algorithm discrimination problem is avoided.

[0223] The above describes particular embodiments of the present specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0224] In the 1990s, it was possible to distinguish whether an improvement in a technology was a hardware improvement (e.g., an improvement in the circuit structure of a diode, transistor, switch, etc.) or a software improvement (an improvement in a method flow). However, as technology has advanced, many improvements in method flows today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain a corresponding hardware circuit structure by programming the improved method flow into a hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented using "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in one of the above hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0225] The controller can be implemented in any suitable way, e.g. the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91 SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to being implemented in pure computer readable program code form, the controller can perfectly well be implemented by means of logic programmed into logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to perform the same functions. The controller can thus be considered as a hardware component, and the means comprised therein for performing various functions can be considered as structures within the hardware component. Alternatively, or even, the means for performing various functions can be considered as both a software module implementing a method and a structure within a hardware component.

[0226] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0227] For the sake of description, the above apparatuses are described in various units with functions respectively. Of course, the functions of each unit can be implemented in one or more software and / or hardware in implementing one or more embodiments of the present specification.

[0228] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0229] The embodiments of the present specification are described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable electronic devices to produce a machine, so that the instructions executed by the computer or other programmable electronic devices generate a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0230] These computer program instructions can also be stored in a computer readable memory capable of directing the computer or other programmable electronic devices to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0231] These computer program instructions can also be loaded into a computer or other programmable electronic devices, so that a series of operation steps are performed on the computer or other programmable electronic devices to produce a computer implemented process, so that the instructions executed on the computer or other programmable electronic devices provide steps for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0232] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0233] The memory can include non-persistent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of the computer readable medium.

[0234] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0235] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0236] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, one or more embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0237] One or more embodiments of the present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0238] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0239] The above only describes the embodiments of the specification and is not intended to limit the specification. The specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the specification shall be included in the scope of claims of the specification.

Claims

1. A biometric processing method, the method comprising: Obtain a biometric request from a target user, wherein the biometric request includes the target user's biometric information; The user's biometric information is input into an encoder for privacy protection of the user's biometric information. The encoder performs privacy protection processing on the user's biometric information to obtain privacy-protected user biometric information. The encoder is obtained by training a model using a first model training method, or by training a model using a first model training method and a second model training method. The first model training method is to train the model sequentially using training sample data containing hard sample data and training the model using multiple sets of adversarial sample data. The second model training method is to train the model using multiple pre-acquired adversarial features. Biometric processing is performed on the target user based on the privacy-protected user biometric information.

2. The method according to claim 1, wherein performing biometric processing on the target user based on the privacy-protected user biometric information includes: The privacy-protected user biometric information is sent to the server, and the privacy-protected user biometric information is used to trigger the server to perform biometric identification processing on the target user based on the pre-stored baseline user biometric information and the privacy-protected user biometric information; Receive the biometric results of the target user sent by the server.

3. The method according to claim 1, wherein performing biometric processing on the target user based on the privacy-protected user biometric information includes: Based on the pre-stored baseline user biometric information and the privacy-protected user biometric information, biometric processing is performed on the target user to obtain the biometric result of the target user's biometric processing. The biometric results are sent to the target user's terminal device.

4. The method according to claim 1, further comprising: Obtain training sample data containing hard sample data, wherein the training sample data includes the user's biometric information; The encoder, decoder, and hard sample distinguisher are jointly trained using the training sample data and a preset first loss function to obtain the encoder, decoder, and hard sample distinguisher after initial training. The decoder is used to restore the privacy-protected training sample data, and the hard sample distinguisher is used to determine whether the training sample data processed by the encoder meets the conditions corresponding to hard sample data. Generate a set of adversarial sample data and label the adversarial sample data as hard sample data; The encoder, decoder, and hard sample discriminator are trained using the adversarial sample data. The accuracy of the judgment result output by the hard sample discriminator is determined to meet the preset stability condition. If not, another set of adversarial sample data is generated. The encoder, decoder, and hard sample discriminator are trained using the re-generated adversarial sample data until the accuracy of the judgment result output by the hard sample discriminator meets the stability condition, thus obtaining the re-trained encoder.

5. The method according to claim 4, wherein the step of jointly training the encoder, decoder, and hard sample discriminator using the training sample data and a preset first loss function to obtain the initially trained encoder, initially trained decoder, and initially trained hard sample discriminator comprises: The training sample data is input into the encoder to obtain privacy-preserved training sample data; The privacy-protected training sample data is input into the decoder corresponding to the encoder, so that the privacy-protected training sample data is restored by the decoder to obtain the reconstructed training sample data. The privacy-protected training sample data is input into the hard sample discriminator, so that the hard sample discriminator can determine whether the privacy-protected training sample data meets the conditions corresponding to hard sample data, and obtain the corresponding judgment result. Based on the training sample data, the privacy-protected training sample data, the reconstructed training sample data, the judgment result, and the preset first loss function, it is determined whether the encoder, the decoder, and the hard sample discriminator have converged. If not, training sample data containing hard sample data is obtained to continue training the encoder, the decoder, and the hard sample discriminator until they converge, thus obtaining the encoder, the decoder, and the hard sample discriminator after initial training.

6. The method according to claim 4 or 5, wherein the first loss function is determined by the maximum similarity between the privacy-protected training sample data and the training sample data, whether the privacy-protected training sample data includes the user's identity information, and a preset classification sub-loss function.

7. The method according to claim 4, wherein generating a set of adversarial example data comprises: Obtain the first sample data; The first sample data is input into the encoder after initial training. The output data of the encoder after initial training is input into the decoder after initial training and the hard sample discriminator after initial training, respectively. The first loss function is maximized and the regularization term of the preset multi-norm constraint is minimized using a preset optimization algorithm to obtain the adversarial sample data corresponding to the first sample data. Based on the adversarial sample data corresponding to the first sample data and the first sample data, a set of adversarial sample data is generated.

8. The method according to claim 4, further comprising: Obtain adversarial features corresponding to the features contained in the second sample data, wherein the adversarial features are determined based on the features contained in the second sample data and the first loss function; The second sample data is input into the encoder or the retrained encoder to obtain the features of the preset network layer in the encoder or the retrained encoder; The adversarial features are input into a preset network layer in the encoder or the retrained encoder to obtain privacy-preserved second sample data; The privacy-protected second sample data is input into the decoder or the retrained decoder to obtain the reconstructed features of the preset network layer in the encoder or the retrained encoder; The privacy-protected second sample data is input into the hard sample distinguisher to obtain the corresponding judgment result; Based on the second sample data, the privacy-protected second sample data, the features corresponding to the privacy-protected second sample data, the reconstructed features of the preset network layer in the encoder or the retrained encoder, the judgment result, and the preset second loss function, it is determined whether the encoder, the decoder, and the hard sample discriminator have converged. If not, the adversarial features corresponding to the features contained in the third sample data continue to train the encoder, the decoder, and the hard sample discriminator until the encoder, the decoder, and the hard sample discriminator converge, thus obtaining the encoder after further training.

9. The method according to claim 8, wherein the second loss function is determined by the maximum similarity between the privacy-preserved second sample data and the second sample data, whether the features corresponding to the privacy-preserved second sample data include the features of the encoder or the preset network layer in the retrained encoder, and a preset sub-loss function.

10. The method according to claim 8 or 9, wherein obtaining the adversarial features corresponding to the features contained in the second sample data includes: Obtain the second sample data; Target noise data is collected from noise data that meets a preset distribution, and the target noise data is added to the features contained in the second sample data to obtain noisy features; The noisy features are input into the encoder, and the data output by the encoder is input into the decoder and the hard sample resolver respectively, to obtain the output data of the decoder and the output data of the hard sample resolver. Based on the output data of the decoder and the output data of the hard sample resolver, the corresponding loss value is calculated through the first loss function. Based on the calculated loss value and the loss value corresponding to the feature contained in the second sample data, the adversarial feature corresponding to the feature contained in the second sample data is determined.

11. A biometric processing device, the device comprising: The request module obtains a biometric request from a target user, wherein the biometric request includes the user's biometric information. The first privacy protection module inputs the user's biometric information into an encoder for privacy protection of the user's biometric information, so that the user's biometric information is processed by the encoder to obtain privacy-protected user biometric information. The encoder is obtained by training a model using a first model training method, or by training a model using a first model training method and a second model training method. The first model training method is to train the model sequentially using training sample data containing hard sample data and training the model using multiple sets of adversarial sample data. The second model training method is to train the model using multiple pre-acquired adversarial features. The biometric module performs biometric processing on the target user based on the privacy-protected user biometric information.

12. A biometric processing device, the biometric processing device comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Obtain a biometric request from a target user, wherein the biometric request includes the target user's biometric information; The user's biometric information is input into an encoder for privacy protection of the user's biometric information. The encoder performs privacy protection processing on the user's biometric information to obtain privacy-protected user biometric information. The encoder is obtained by training a model using a first model training method, or by training a model using a first model training method and a second model training method. The first model training method is to train the model sequentially using training sample data containing hard sample data and training the model using multiple sets of adversarial sample data. The second model training method is to train the model using multiple pre-acquired adversarial features. Biometric processing is performed on the target user based on the privacy-protected user biometric information.

13. A storage medium for storing computer-executable instructions, which, when executed by a processor, perform the following process: Obtain a biometric request from a target user, wherein the biometric request includes the target user's biometric information; The user's biometric information is input into an encoder for privacy protection of the user's biometric information. The encoder performs privacy protection processing on the user's biometric information to obtain privacy-protected user biometric information. The encoder is obtained by training a model using a first model training method, or by training a model using a first model training method and a second model training method. The first model training method is to train the model sequentially using training sample data containing hard sample data and training the model using multiple sets of adversarial sample data. The second model training method is to train the model using multiple pre-acquired adversarial features. Biometric processing is performed on the target user based on the privacy-protected user biometric information.

Images