Encryption identification method for biological feature template protection
The end-to-end biometric encryption identification framework built through the Pytorch deep learning framework solves the problems of user privacy leakage and security risks in biometric identity authentication, and realizes efficient encryption and identity recognition of biometric information, meeting irreversibility and performance requirements. .
Patent Information
- Application Number
- CN202411163691.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-22
AI Technical Summary
Biometric identity authentication is easily stolen during storage and transmission, resulting in user privacy leakage and illegal authentication. The same biological feature is easily tracked and cross-matched in multiple application systems, posing a security risk.
The end-to-end biometric encryption recognition framework is built using the Pytorch deep learning framework. It is trained through a combined loss function combining structural similarity loss and cross-entropy loss to generate irreversible encrypted images for encryption processing and identity recognition of biometric images.
It realizes efficient encryption of biometric information, prevents inverse operations from restoring the original image, ensures user privacy and security, meets irreversibility, revocability, unlinkability and performance requirements, and provides high-level privacy protection and identity recognition of biometric information. performance.
Smart Images

Figure CN119004511A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of biometric technology data information privacy protection, and in particular relates to an encryption recognition method for biometric template protection. Background Art
[0002] With the rapid development of artificial intelligence, biometric-based identity authentication technology has gradually provided convenient and secure identity authentication for the intelligent era. Compared with the traditional method based on passwords or identity cards, it is more efficient and secure, and has been widely used in the market. However, with the growth of commercial applications, biometric identity authentication faces security risks that cannot be ignored. Biometrics are permanent and unique, and digitized features need to be stored as templates in applications. Once these templates are stolen and the original biometrics are obtained, the user's privacy information will be leaked, which will be used by criminals to impersonate legitimate users for illegal authentication, resulting in the failure of the biometrics permanently associated with the user's identity in the system.
[0003] In addition, if the same biometric feature is stored in multiple application systems without protection, it is easy for criminals to track and cross-match it. Therefore, when promoting biometric identity authentication applications, how to effectively protect biometric information and user privacy has become an important issue in the research field at home and abroad. Summary of the invention
[0004] In order to solve the above problems, the purpose of the present invention is to provide an encryption identification method for biometric template protection.
[0005] In order to achieve the above object, the encryption identification method for biometric template protection provided by the present invention comprises the following steps performed in sequence:
[0006] S1. Collect a public biometric dataset as a biometric image encryption recognition dataset, and divide it into a training set, a validation set, and a test set in proportion. Then preprocess the training set to generate a biometric image encryption recognition training set.
[0007] S2. Use the Pytorch deep learning framework to build an end-to-end biometric encryption recognition framework;
[0008] S3, input the biometric image encryption recognition training set obtained in S1 into the end-to-end biometric encryption recognition framework constructed by S2 for training, use an adaptive gradient descent optimizer, adopt a combination loss SSIM-CE that combines structural similarity loss and cross entropy loss as the loss function, and use a validation set to verify the effect of each round of training, and obtain a trained end-to-end biometric encryption recognition framework;
[0009] S4. Resize the biometric image in the test set or the biometric image to be detected obtained in S1 to a size of 224×224 pixels, copy it into a three-channel image, and input it into the above-mentioned trained end-to-end biometric encryption recognition framework. The trained end-to-end biometric encryption recognition framework simultaneously encrypts and identifies the biometric image, and outputs the final encrypted image result and identity recognition prediction result.
[0010] In S1, the public biometric data set is collected as a biometric image encryption recognition data set, and is divided into a training set, a validation set, and a test set in proportion, and then the training set is preprocessed to generate a biometric image encryption recognition training set. The specific method is as follows:
[0011] Public biometric datasets consisting of single-channel biometric images are collected as biometric image encryption recognition datasets, and divided into training set, validation set and test set in a ratio of 4:1:1. The biometric images in the training set are then unified into a size of 224×224 pixels, and then copied into three-channel images. They are then rotated, horizontally flipped, and the brightness and contrast are adjusted. Finally, they are normalized to obtain the original image I after data enhancement. All the original images I constitute the biometric image encryption recognition training set.
[0012] The biometric image includes a finger vein image, a face image, and a fingerprint image.
[0013] In S2, the end-to-end biometric encryption recognition framework includes an image encryption model and a recognition model; the image encryption model includes an encryption module, a shallow feature extraction network 1, and a feature expansion network; wherein the encryption module includes a noise expansion network, a shallow feature extraction network 2, and a spatial channel attention network;
[0014] The shallow feature extraction network 1 is composed of a stack of convolutional layers, normalization layers, and activation layers; the feature expansion network is composed of a stack of transposed convolutional layers, normalization layers, and activation layers, and then a convolutional layer is superimposed;
[0015] The noise expansion network is composed of a stack of transposed convolutional layers, normalization layers, and activation layers; the shallow feature extraction network 2 has the same structure as the shallow feature extraction network 1, but does not share parameters; the spatial channel attention network is used to recalibrate the importance of features, and has three paths: channel attention path, spatial attention path, and feature path;
[0016] The recognition model adopts a pyramid-structured visual graph neural network ViG, including a Stem module, a graph convolution module GCNBlock, and a fully connected layer;
[0017] The parameter settings in the pyramid architecture visual graph neural network ViG are as follows: the number of neighbor nodes for each node is set to 9, and the maximum graph convolution is used to aggregate and update the operations of graph node features; the pyramid architecture visual graph neural network ViG contains 4 trunks, and the number of graph convolution modules GCNBlock contained in each trunk is 2, 2, 6, and 2 respectively. The input features are downsampled at the input of each trunk to form a pyramid architecture.
[0018] In S3, the biometric image encryption recognition training set obtained in S1 is input into the end-to-end biometric encryption recognition framework constructed in S2 for training, an adaptive gradient descent optimizer is used, a combined loss SSIM-CE combining structural similarity loss and cross entropy loss is adopted as a loss function, and a validation set is used to verify the effect of each round of training. The specific method for obtaining the trained end-to-end biometric encryption recognition framework is as follows:
[0019] S3.1. First, the original image I with a pixel size of 3×224×224 and the random noise U with a pixel size of 224×1×1 are input into the image encryption model in the end-to-end biometric encryption recognition framework. After the original image I and the random noise U enter the encryption module, the noise expansion network is used to expand the dimension of the random noise U to obtain the noise feature map U1. The shallow feature extraction network 2 is used to extract the shallow image feature map F2 carrying the identity information from the original image I. Then, the noise feature map U1 and the shallow image feature map F2 are spliced along the channel to obtain the feature map K with a pixel size of 224×32×32. And set A hyperparameter r is used to adjust the number of channels of the noise feature map U1 and the shallow image feature map F2, thereby adjusting the proportion of noise in the feature map K; the dimension of the noise feature map U1 is 224*r×32×32, and the dimension of the shallow image feature map F2 is 224*(1-r)×32×32; then the channel attention path in the spatial channel attention network uses a linear layer to learn the interdependence between channels, and the spatial attention path uses a convolutional layer to learn the importance of spatial features, and the feature path introduces the original features; then the outputs of all three paths are fused through element-by-element multiplication to recalibrate the feature importance and obtain the encrypted feature map K e , improves the security of encryption processing;
[0020] At the same time, the shallow feature extraction network 1 is used to extract the shallow image feature map F1 to be encrypted from the original image I;
[0021] Then the encrypted feature map K e Perform a row multiplication operation with the shallow image feature map F1 to be encrypted to obtain an encrypted image feature map F with a pixel size of 224×32×32. eFinally, the input features are expanded and fine-tuned through the feature expansion network, thereby converting the encrypted image feature map F e Transformed into a human-readable encrypted image I with a pixel size of 3×224×224 e ;
[0022] S3.2, the encrypted image I obtained in S3.1 e The input is sent to the recognition model in the end-to-end biometric encryption recognition framework, and the encrypted image I is extracted through the Stem module in the pyramid architecture visual graph neural network ViG. e The local content features of the image are converted into a graph representation; then, the graph node features are aggregated and updated through the graph convolution module GCNBlock in series to obtain a more reasonable graph structure; finally, the identity recognition prediction result is output through a fully connected layer;
[0023] After each round of training, the validation set is used to verify the recognition effect of the current parameter weights. When the number of training times reaches the set threshold, or the value of the loss function reaches the set range, or the recognition result of the validation set reaches the expected setting, the training is considered completed, the parameter weights are saved, and the final trained end-to-end biometric encryption recognition framework is obtained.
[0024] The encryption recognition method for biometric template protection provided by the present invention has the following advantages: ① an end-to-end biometric encryption recognition framework is provided that can simultaneously complete the encryption task and recognition task of the biometric image; ② the method uses the biometric image feature information learned by the deep neural network to complete the encryption processing, and is not a biometric encryption based on some mathematical encryption or image transformation processing, so the original image cannot be obtained through inverse operation or inverse transformation; ③ the designed encryption module can effectively fuse the noise and the identity information carried by the biometrics, ensuring that the encrypted image generated by the image encryption model can be used for identity recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 Flowchart of the encrypted identification method for biometric template protection provided by the present invention.
[0026] Figure 2 This is a structural diagram of the end-to-end biometric encryption recognition framework provided by the present invention.
[0027] Figure 3 This is a structural diagram of the encryption module in the end-to-end biometric encryption recognition framework provided by the present invention. DETAILED DESCRIPTION
[0028] The following describes in detail the encryption identification method for biometric template protection provided by the present invention in conjunction with the accompanying drawings and specific embodiments.
[0029] like Figure 1 As shown, the encryption identification method for biometric template protection provided by the present invention includes the following steps performed in sequence:
[0030] S1. Collect a public biometric dataset as a biometric image encryption recognition dataset, and divide it into a training set, a validation set, and a test set in proportion. Then preprocess the training set to generate a biometric image encryption recognition training set.
[0031] The specific method is as follows:
[0032] Public biometric datasets consisting of single-channel biometric images are collected as biometric image encryption recognition datasets, and divided into training set, validation set and test set in a ratio of 4:1:1. The biometric images in the training set are then unified into a size of 224×224 pixels, and then copied into three-channel images. They are then rotated, horizontally flipped, and the brightness and contrast are adjusted. Finally, they are normalized to obtain the original image I after data enhancement. All the original images I constitute the biometric image encryption recognition training set.
[0033] The biometric image includes a finger vein image, a face image, and a fingerprint image;
[0034] In the present invention, finger vein images are used as one of the biometric images for experiments, and the public datasets used are the Finger Vein Dataset of Universiti Teknologi Malaysia (FV-USM for short) and the Finger Vein Dataset of Shandong University (SDUMLA for short).
[0035] There are 5904 finger vein images in the FV-USM, 492 categories, 12 samples in each category, and the resolution of each finger vein image is 100 × 300. In the experiment, 8 finger vein images of each category are randomly selected to form a training set, 2 to form a validation set, and the remaining 2 to form a test set.
[0036] A total of 600 finger vein images from 100 categories in SDUMLA were used, with 6 samples in each category and a resolution of 120×280. In the experiment, 4 finger vein images from each category were randomly selected to form a training set, 1 to form a validation set, and the remaining 1 to form a test set.
[0037] S2. Use the Pytorch deep learning framework to build an end-to-end biometric encryption recognition framework;
[0038] like Figure 2 , Figure 3As shown, the end-to-end biometric encryption recognition framework includes an image encryption model and a recognition model; the image encryption model includes an encryption module, a shallow feature extraction network 1, and a feature expansion network; wherein the encryption module includes a noise expansion network, a shallow feature extraction network 2, and a spatial channel attention network;
[0039] The shallow feature extraction network 1 is composed of a stack of convolutional layers (Conv), normalization layers (BN) and activation layers (ReLU); the feature expansion network is composed of a stack of transposed convolutional layers, normalization layers and activation layers, and then a convolutional layer is superimposed; the input features are first expanded by stacking transposed convolutional layers, normalization layers and activation layers, and then the features are fine-tuned by a convolutional layer;
[0040] The noise expansion network is composed of a stack of transposed convolutional layers, normalization layers, and activation layers; the shallow feature extraction network 2 has the same structure as the shallow feature extraction network 1, but the parameters are not shared; the spatial channel attention network is used for feature depth conditioning, and there are three paths in total: channel attention path, spatial attention path, and feature path;
[0041] The recognition model adopts a pyramid-structured visual graph neural network ViG, including a Stem module, a graph convolution module GCNBlock, and a fully connected layer;
[0042] The parameter settings in the pyramid architecture visual graph neural network ViG are as follows: the number of neighbor nodes for each node is set to 9, and the maximum graph convolution is used to aggregate and update the operations of graph node features; the pyramid architecture visual graph neural network ViG contains 4 trunks, and the number of graph convolution modules GCNBlock contained in each trunk is 2, 2, 6, and 2 respectively. The input features are downsampled at the input of each trunk to form a pyramid architecture.
[0043] S3, input the biometric image encryption recognition training set obtained in S1 into the end-to-end biometric encryption recognition framework constructed by S2 for training, use an adaptive gradient descent optimizer, adopt a combination loss SSIM-CE that combines structural similarity loss and cross entropy loss as the loss function, and use a validation set to verify the effect of each round of training, and obtain a trained end-to-end biometric encryption recognition framework;
[0044] The specific method is as follows:
[0045] S3.1, if Figure 2As shown, first, the original image I with a pixel size of 3×224×224 and the random noise U with a pixel size of 224×1×1 are input into the image encryption model in the end-to-end biometric encryption recognition framework, and the type and parameters of the random noise U can be arbitrarily changed, thereby serving as a method of replacing the protection template; after the original image I and the random noise U enter the encryption module, the dimension of the random noise U is expanded by using the noise expansion network therein to obtain the noise feature map U1, and the shallow feature extraction network 2 is used to extract the shallow image feature map F2 carrying the identity information from the original image I, and then the noise feature map U1 and the shallow image feature map F2 are spliced along the channel to obtain the feature map K with a pixel size of 224×32×32; A hyperparameter r is set to adjust the number of channels of the noise feature map U1 and the shallow image feature map F2, thereby adjusting the proportion of noise in the feature map K; the dimension of the noise feature map U1 is 224*r×32×32, and the dimension of the shallow image feature map F2 is 224*(1-r)×32×32; changing the proportion of noise can be used as a method to replace the protection template; then the channel attention path in the spatial channel attention network uses the linear layer to learn the interdependence between channels, the spatial attention path uses the convolutional layer to learn the importance of spatial features, and the feature path introduces the original features; then the outputs of all three paths are fused through element-by-element multiplication to recalibrate the feature importance and obtain the encrypted feature map K e , improves the security of encryption processing;
[0046] At the same time, the shallow feature extraction network 1 is used to extract the shallow image feature map F1 to be encrypted from the original image I;
[0047] Then the encrypted feature map K e Perform a row multiplication operation with the shallow image feature map F1 to be encrypted to obtain an encrypted image feature map F with a pixel size of 224×32×32. e Finally, the input features are expanded and fine-tuned through the feature expansion network, thereby converting the encrypted image feature map F e Transformed into a human-readable encrypted image I with a pixel size of 3×224×224 e , that is, a protected template is obtained. This method of generating an encrypted image is not based on a mathematical encryption algorithm or image transformation processing, so the original image I cannot be obtained through inverse operation or inverse transformation.
[0048] S3.2, the encrypted image I obtained in S3.1 e The input is sent to the recognition model in the end-to-end biometric encryption recognition framework, and the encrypted image I is extracted through the Stem module in the pyramid architecture visual graph neural network ViG. eThe local content features of the image are converted into a graph representation; then, the graph node features are aggregated and updated through the graph convolution module GCNBlock in series to obtain a more reasonable graph structure; finally, the identity recognition prediction result is output through a fully connected layer.
[0049] The present invention provides a SSIM-CE loss function (L SSIM-CE ) to calculate the structural similarity loss (L SSIM ), and the loss between the framework prediction result and the true label (L CE ), thereby optimizing the entire end-to-end biometric encryption recognition framework. The formula of the SSIM-CE loss function is:
[0050]
[0051] Among them, x is the pixel information of the original image I, and y is the encrypted image I e , M is the number of blocks used to calculate SSIM, j is the jth block in the block processing; p is the output of the end-to-end biometric encryption recognition framework, which is a vector, i is the i-th value, and class is the actual label; μ x and μ y They are the original image I and the encrypted image I e The mean value, σ x and σ y They are the original image I and the encrypted image I e The variance, σ xy The original image I and the encrypted image I e The covariance of , C1 and C2 are constants used to avoid the denominator being zero.
[0052] After each round of training, the validation set is used to verify the recognition effect of the current parameter weights. When the number of training times reaches the set threshold, or the value of the loss function reaches the set range, or the recognition result of the validation set reaches the expected setting, the training is considered completed, the parameter weights are saved, and the final trained end-to-end biometric encryption recognition framework is obtained.
[0053] S4. Resize the biometric image in the test set obtained in S1 or the biometric image to be detected to a size of 224×224 pixels, copy it into a three-channel image, and input it into the above-mentioned trained end-to-end biometric encryption recognition framework. The trained end-to-end biometric encryption recognition framework simultaneously encrypts and identifies the biometric image, and outputs the final encrypted image result and identity recognition prediction result.
[0054] The end-to-end biometric encryption recognition framework proposed by the method of the present invention needs to meet the four requirements of the biometric template protection task: irreversibility, revocability, unlinkability, and performance requirements. The method of the present invention is not based on a fixed encryption operation to generate an encrypted image. An attacker cannot obtain the original image by performing an inverse operation on the encrypted image, thus satisfying irreversibility. The following experiments are used to prove that the method of the present invention meets the requirements of revocability, unlinkability, and performance. In order to test the effectiveness and superiority of the method of the present invention, the inventors used the test sets in the FV-USM finger vein dataset and the SDUMLA finger vein dataset for experimental testing, and compared them with other methods.
[0055] The present invention demonstrates the experimental results of the proposed method under three different input random noises (normal distribution, uniform distribution, exponential distribution). The effectiveness of the method of the present invention is demonstrated by demonstrating the experimental results of different input noises, different noise parameters, and different noise ratios.
[0056] Table 1 shows the experimental results of different types of input noise distribution and different noise parameters, and Table 2 shows the experimental results of changing the noise input ratio (i.e., changing the value of the hyperparameter r). Among them: ACC is the recognition accuracy, that is, the ratio of the number of correctly classified samples to the total number of samples in the test set; EER is the equal error rate, that is, when the false positive rate (FPR) and the false negative rate (FNR) of the biometric system are equal, its value is EER. The higher the ACC and the lower the EER, the better the recognition performance of the method. It can be seen that the method of the present invention can obtain high recognition accuracy whether it is facing noise inputs of different categories and parameters or facing different noise proportions, which shows that the method of the present invention has good robustness and meets the requirements of revocability and identity recognition performance.
[0057] Table 1 Experimental results of different input noise distribution types and different noise parameters
[0058]
[0059] Table 2 Experimental results of different noise ratios
[0060]
[0061]
[0062] In order to verify that the end-to-end biometric encryption recognition framework proposed by the method of the present invention meets the unlinkability, the test sets of the FV-USM dataset and the SDUMLA dataset are used to obtain the encrypted test set through the framework with exponential input noise {3}, and then the obtained encrypted test set is input into the framework with the protection template replaced for experimental testing. That is, the attacker is simulated to get the leaked template to attack the recognition system with the protection template replaced. The lower the ACC, the higher the EER, indicating that the framework can better meet the unlinkability.
[0063] It can be seen from the experimental results in Table 3 that the method of the present invention meets the unlinkability. Due to the poor image quality of the SDUMLA dataset, the experimental results fluctuate widely, but the ACC is still low and the EER is large. When the FV-USM dataset with better image quality is used, the experimental results are better and more stable.
[0064] Table 3 Unlinkability verification experimental results
[0065]
[0066] In order to prove the superiority of the method of the present invention, it is compared with five typical finger vein recognition methods, and the experimental results are shown in Table 4. It can be seen that even compared with the finger vein recognition method that does not provide template protection, the recognition performance of the method of the present invention is still superior, and only the ACC on the SDUMLA dataset is lower than that of the CNN-framework method. Therefore, the method of the present invention can provide users with a high level of biometric information privacy protection and identity recognition, which is a huge advantage.
[0067] Table 4 Comparative experimental results of finger vein recognition with different methods
[0068]
[0069]
[0070] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.
Claims
1. An encryption recognition method for biometric template protection, characterized by: The encryption identification method for biometric template protection comprises the following steps performed in sequence: S1. Collect a public biometric dataset as a biometric image encryption recognition dataset, and divide it into a training set, a validation set, and a test set in proportion. Then preprocess the training set to generate a biometric image encryption recognition training set. S2. Use the Pytorch deep learning framework to build an end-to-end biometric encryption recognition framework; S3, input the biometric image encryption recognition training set obtained in S1 into the end-to-end biometric encryption recognition framework constructed by S2 for training, use an adaptive gradient descent optimizer, adopt a combination loss SSIM-CE that combines structural similarity loss and cross entropy loss as the loss function, and use a validation set to verify the effect of each round of training, and obtain a trained end-to-end biometric encryption recognition framework; S4. Resize the biometric image in the test set or the biometric image to be detected obtained in S1 to a size of 224×224 pixels, copy it into a three-channel image, and input it into the above-mentioned trained end-to-end biometric encryption recognition framework. The trained end-to-end biometric encryption recognition framework simultaneously encrypts and identifies the biometric image, and outputs the final encrypted image result and identity recognition prediction result.
2. The encryption identification method for biometric template protection according to claim 1 is characterized in that: In S1, the public biometric data set is collected as a biometric image encryption recognition data set, and is divided into a training set, a validation set, and a test set in proportion, and then the training set is preprocessed to generate a biometric image encryption recognition training set. The specific method is as follows: Public biometric datasets consisting of single-channel biometric images are collected as biometric image encryption recognition datasets, and divided into training set, validation set and test set in a ratio of 4:1:
1. The biometric images in the training set are then unified into a size of 224×224 pixels, and then copied into three-channel images. They are then rotated, horizontally flipped, and the brightness and contrast are adjusted. Finally, they are normalized to obtain the original image I after data enhancement. All the original images I constitute the biometric image encryption recognition training set. The biometric image includes a finger vein image, a face image, and a fingerprint image.
3. The encryption identification method for biometric template protection according to claim 1 is characterized in that: In S2, the end-to-end biometric encryption recognition framework includes an image encryption model and a recognition model; the image encryption model includes an encryption module, a shallow feature extraction network 1, and a feature expansion network; The encryption module includes noise expansion network, shallow feature extraction network 2, and spatial channel attention network; The shallow feature extraction network 1 is composed of a stack of convolutional layers, normalization layers, and activation layers; the feature expansion network is composed of a stack of transposed convolutional layers, normalization layers, and activation layers, and then a convolutional layer is superimposed; the input features are first expanded by stacking transposed convolutional layers, normalization layers, and activation layers, and then the features are fine-tuned by a convolutional layer; The noise expansion network is composed of a stack of transposed convolutional layers, normalization layers, and activation layers; The shallow feature extraction network 2 has the same structure as the shallow feature extraction network 1, but the parameters are not shared; the spatial channel attention network is used for feature depth conditioning, and there are three paths: channel attention path, spatial attention path and feature path; The recognition model adopts a pyramid-structured visual graph neural network ViG, including a Stem module, a graph convolution module GCNBlock, and a fully connected layer; The parameter settings in the pyramid architecture visual graph neural network ViG are as follows: the number of neighbor nodes for each node is set to 9, and the maximum graph convolution is used to aggregate and update the operations of graph node features; the pyramid architecture visual graph neural network ViG contains 4 trunks, and the number of graph convolution modules GCNBlock contained in each trunk is 2, 2, 6, and 2 respectively. The input features are downsampled at the input of each trunk to form a pyramid architecture.
4. The encryption identification method for biometric template protection according to claim 1, characterized in that: In S3, the biometric image encryption recognition training set obtained in S1 is input into the end-to-end biometric encryption recognition framework constructed in S2 for training, an adaptive gradient descent optimizer is used, a combined loss SSIM-CE combining structural similarity loss and cross entropy loss is adopted as a loss function, and a validation set is used to verify the effect of each round of training. The specific method for obtaining the trained end-to-end biometric encryption recognition framework is as follows: S3.
1. First, the original image I with a pixel size of 3×224×224 and the random noise U with a pixel size of 224×1×1 are input into the image encryption model in the end-to-end biometric encryption recognition framework. After the original image I and the random noise U enter the encryption module, the noise expansion network is used to expand the dimension of the random noise U to obtain the noise feature map U1. The shallow feature extraction network 2 is used to extract the shallow image feature map F2 carrying the identity information from the original image I. Then, the noise feature map U1 and the shallow image feature map F2 are spliced along the channel to obtain the image. The feature map K is 224×32×32 pixels; and a hyperparameter r is set to adjust the number of channels of the noise feature map U1 and the shallow image feature map F2, so as to adjust the proportion of noise in the feature map K; the dimension of the noise feature map U1 is 224*r×32×32, and the dimension of the shallow image feature map F2 is 224*(1-r)×32×32; then the channel attention path in the spatial channel attention network uses the linear layer to learn the interdependence between channels, and the spatial attention path uses the convolutional layer to learn the importance of spatial features, and the feature path introduces the original features; The outputs of all three paths are then fused by element-wise multiplication to recalibrate the feature importance and obtain the encrypted feature map K e , improves the security of encryption processing; At the same time, the shallow feature extraction network 1 is used to extract the shallow image feature map F1 to be encrypted from the original image I; Then the encrypted feature map K e Perform a row multiplication operation with the shallow image feature map F1 to be encrypted to obtain an encrypted image feature map F with a pixel size of 224×32×32. e Finally, the input features are expanded and fine-tuned through the feature expansion network, thereby converting the encrypted image feature map F e Transformed into a human-readable encrypted image I with a pixel size of 3×224×224 e ; S3.2, the encrypted image I obtained in S3.1 e The input is sent to the recognition model in the end-to-end biometric encryption recognition framework, and the encrypted image I is extracted through the Stem module in the pyramid architecture visual graph neural network ViG. e The local content features of the image are converted into a graph representation; then, the graph node features are aggregated and updated through the graph convolution module GCNBlock in series to obtain a more reasonable graph structure; finally, the identity recognition prediction result is output through a fully connected layer; After each round of training, the validation set is used to verify the recognition effect of the current parameter weights. When the number of training times reaches the set threshold, or the value of the loss function reaches the set range, or the recognition result of the validation set reaches the expected setting, the training is considered completed, the parameter weights are saved, and the final trained end-to-end biometric encryption recognition framework is obtained.
