Face recognition method based on deep learning and image encryption

Abstract

The invention discloses a face recognition method based on deep learning and image encryption, and belongs to the field of face image recognition. According to the method, the problems of poor image encryption effect, low face recognition efficiency and low accuracy of an existing method are solved. The standard face template image and the to-be-recognized face image are sent to the cloud, the to-be-recognized face image is recognized at the cloud, and in order to avoid privacy disclosure in the image transmission process, an image encryption algorithm is designed to improve the face image encryption effect. According to the invention, clustering processing is carried out on the standard face template image in the database, and after clustering processing, only similarity calculation is needed to be carried out on the face image to be recognized and the images in one category, so that the recognition efficiency is greatly improved. The feature vector of the image is extracted by adopting the feature extraction model designed by the invention, and the recognition accuracy can be greatly improved by performing similarity calculation based on the extracted feature vector. The method can be applied to face image recognition.

Description

Technical Field

[0001] This invention belongs to the field of face image recognition, specifically relating to a face recognition method based on deep learning and image encryption. Background Technology

[0002] With the continuous development of society, economy, and technology, people are paying increasing attention to information security. Traditional identification methods, including chip card identification, are increasingly unable to meet the needs of today's society for personal and information security due to their drawbacks such as easy loss, easy forgetting, easy theft, and complex processes. In recent years, with the vigorous development of computer science and biological science technologies, facial recognition technology has emerged. Because facial features are unique to each individual organism, and because it is convenient, interactive, and does not require physical contact with the detection and recognition equipment, it has received widespread attention from industry and academia.

[0003] Security is a crucial area where facial recognition technology is applied. From individual residences to shopping malls and residential communities, and from large corporations and factories, widespread deployment of surveillance cameras is used to establish public safety. Facial recognition technology primarily involves capturing images containing facial features, extracting the facial features to be identified, and comparing these features with a database of processed facial features to determine the identity of the person based on the similarity score. However, facial recognition systems require storing a large number of images in their databases, and the subsequent similarity calculations are computationally intensive. Therefore, it is necessary to send facial template images captured at various edge devices to the cloud for the facial recognition process.

[0004] However, directly transmitting unprocessed or unprotected facial images from the edge to the cloud significantly increases the risk of user privacy leaks and violations. Illegal collection of these images could have serious consequences. Therefore, in situations where the security and integrity of facial image information during transmission cannot be guaranteed, the facial template images collected at the edge are encrypted before being sent to the cloud. The encrypted images are then decrypted in the cloud, and finally, a database is created using the decrypted images for facial recognition. However, traditional image encryption methods still have limited effectiveness and remain vulnerable to cracking.

[0005] Meanwhile, existing similarity-based face recognition algorithms use deep learning models to extract facial features from images and then obtain the final face recognition result based on the similarity of these features. However, limited by the performance of existing deep learning models, the face recognition accuracy of current methods still needs further improvement. Furthermore, it requires matching the features of the face image to be recognized with the features of each face image in the database separately, thus the recognition efficiency of existing face recognition methods remains relatively low.

[0006] In summary, to address the problems of poor image encryption and low efficiency and accuracy in face recognition using existing methods, a new face recognition method is proposed, which is a pressing issue that needs to be resolved. Summary of the Invention

[0007] The purpose of this invention is to solve the problems of poor image encryption effect, low efficiency and low accuracy of face recognition in existing methods, and to propose a face recognition method based on deep learning and image encryption.

[0008] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0009] A face recognition method based on deep learning and image encryption, the method specifically includes the following steps:

[0010] Step 1: Each edge device encrypts each collected face template image and then sends the encrypted face images to the cloud.

[0011] The cloud decrypts each received face image and uses the decrypted face images to form a face image database.

[0012] Step 2: Use a feature extraction model to extract the feature vector of each face template image in the database, and then cluster each face template image according to the extracted feature vector to obtain the face template images contained in each category.

[0013] Step 3: After encrypting the face image to be identified, send the encrypted face image to the cloud. The cloud decrypts the encrypted image and uses a feature extraction model to extract the feature vector of the face image to be identified.

[0014] Step 4: Calculate the distance between the center of each category in the clustering results of Step 2 and the feature vector of the face image to be identified. Select the category center with the smallest distance to the feature vector of the face image to be identified, and take the cluster corresponding to the selected category center as the cluster corresponding to the face image to be identified.

[0015] Then, calculate the similarity between the face image to be identified and each image in the corresponding cluster of the face image to be identified, and obtain the recognition result of the face image to be identified based on the face template image corresponding to the highest similarity.

[0016] The beneficial effects of this invention are:

[0017] This invention sends standard face template images and the face image to be identified to the cloud for recognition. To prevent privacy leaks during image transmission, this invention designs an image encryption algorithm to improve the encryption effect of the face image. Simultaneously, to improve the recognition efficiency, this invention performs clustering processing on the standard face template images in the database. After clustering, it is only necessary to calculate the similarity between the face image to be identified and images within the same category, greatly accelerating the recognition process. Furthermore, the feature extraction model designed in this invention is used to extract feature vectors from the image. Similarity calculation based on the extracted feature vectors can significantly improve the recognition accuracy. Attached Figure Description

[0018] Figure 1 This is a flowchart of a face recognition method based on deep learning and image encryption according to the present invention. Detailed Implementation

[0019] Specific implementation method one: Combining Figure 1 This embodiment describes a face recognition method based on deep learning and image encryption, which specifically includes the following steps:

[0020] Step 1: Each edge device encrypts the collected standard face template images with known identities (converting the collected images to grayscale before encryption), and then sends the encrypted face images to the cloud.

[0021] The cloud decrypts each received face image and uses the decrypted face images to form a face image database.

[0022] Step 2: Use a feature extraction model to extract the feature vector of each face template image in the database, and then cluster each face template image according to the extracted feature vector to obtain the face template images contained in each category.

[0023] Step 3: After encrypting the face image to be identified, send the encrypted face image to the cloud. The cloud decrypts the encrypted image and uses a feature extraction model to extract the feature vector of the face image to be identified.

[0024] It should be noted that before encrypting the face image to be recognized, an affine transformation is required to obtain a corrected image in order to improve the accuracy of face recognition.

[0025] Step 4: Calculate the distance between the center of each category in the clustering results of Step 2 and the feature vector of the face image to be identified. Select the category center with the smallest distance to the feature vector of the face image to be identified, and take the cluster corresponding to the selected category center as the cluster corresponding to the face image to be identified.

[0026] Next, calculate the similarity between the face image to be identified and each image in the corresponding cluster (calculate the Euclidean distance based on the image feature vector, and use the calculated Euclidean distance as the similarity). The recognition result of the face image to be identified is obtained based on the face template image corresponding to the highest similarity.

[0027] Due to the limited computing power at the edge, this invention executes the facial recognition process in the cloud after obtaining the image of the face to be identified. This requires transmitting the facial template images acquired at each edge to the cloud. After extracting the feature vectors of each facial template image and the face image to be identified in the cloud, the facial template image with the highest similarity to the face image to be identified can be obtained based on the feature vectors. In special circumstances, suspicious facial images can be extracted from the surveillance video of the monitored area and then sent to the cloud for matching to determine the identity of the suspicious person, which can also be used for tracking suspicious persons. Furthermore, the method of this invention can also be used for system login management and authentication within large chain enterprises or organizations. Standard facial images of all employees are recorded at various edge terminals and transmitted to the cloud. During actual login verification, the facial image of the user initiating the login request, actually collected at the edge terminal, is sent to the cloud. The cloud finds the template image that best matches the user. If the similarity between the user's facial image and the best-matching template image is greater than a set threshold, the verification is successful, and the user is allowed to log in. Otherwise, the user is not allowed to log in, thus preventing unauthorized intrusion into the system and the potential for malicious tampering or illegal acquisition of system information. Moreover, each edge terminal can use different vectors to encrypt the acquired images, and the cloud can decrypt the images using the encryption method shared by the edge terminals.

[0028] Specific Implementation Method Two: This implementation method further defines Specific Implementation Method One. Specifically, the encryption processing of each collected face template image involves:

[0029] For any sheet of size Face template image :

[0030] Step 11: Initialize the number of iterations ;

[0031] Steps 1 and 2: Generate a length of row vectors , ,satisfy And vector No two elements in the array are equal;

[0032] According to the vector For images The image is obtained by reordering the pixels in each column. Arrange the vectors in ascending order. Sort the elements in the image, and then process the image according to the order of the elements in the sorting result. The pixels in each column are reordered to obtain the image. ;

[0033] For example, in vectors If the value of the 3rd element is the smallest and the value of the 7th element is the second smallest, then the image is... The third column element is used as the first column after reordering, and the image is... The 7th column element becomes the 2nd column after reordering, and so on;

[0034] Step 13, to Randomly sort the vectors to obtain the vectors. , Then the image The Column pixels move upwards in a loop Bits, to obtain the image For example, if it is necessary to transfer the first If the column pixels are cyclically shifted upwards by 2 positions, then the original column number... The third pixel in the column becomes the first pixel;

[0035] in: ;

[0036] Step 14: Generate a length of column vectors , ,satisfy And vector No two elements in the array are equal;

[0037] According to the vector For images The image is obtained by reordering the pixels in each row. Arrange the vectors in ascending order. Sort the elements in the image, and then process the image according to the order of the elements in the sorting result. The pixels in each row are reordered to obtain the image. ;

[0038] For example, in vectors If the value of the second element is the smallest and the value of the tenth element is the second smallest, then the image is... The second row of pixels is used as the first row after reordering the image. The 10th row of pixels becomes the 2nd row after reordering, and so on;

[0039] Step 15, to Randomly sort the vectors to obtain the vectors. , Then the image The Row pixels move to the left in a loop Bits, to obtain the image ;

[0040] Step 16: Image Noise is added to obtain the image after noise processing. ;

[0041] Step 17: Determine if the iteration stopping condition is met. If the stopping condition is met, use the image with added noise obtained from the last iteration as the final encrypted image. If the stopping condition is not met, then let... For images Return to steps one and two (i.e., the next iteration changes the image). (pixel order in the image), that is, continuing with the image Encryption processing is performed.

[0042] The other steps and parameters are the same as in Specific Implementation Method 1.

[0043] Before sending the image from the edge to the cloud, this invention requires encrypting the image at the edge to ensure security during image transmission and prevent privacy leaks.

[0044] Specific Implementation Method Three: This implementation method is a further limitation of Specific Implementation Method Two. The specific process of step one six is ​​as follows:

[0045] Step 161: Based on the image Calculate intermediate variables :

[0046]

[0047] in, Representing an image The Middle Line 1 The pixel values ​​of the column;

[0048] Give image Add a mean of 0 and a variance of Gaussian noise was used to obtain the image. ;in, It is a constant;

[0049] Step 162: Randomly generate a sparse matrix Using sparse matrices and images Generating matrix :

[0050]

[0051] In the formula Representing an image The corresponding matrix; the superscript T indicates the transpose of the matrix;

[0052] Step 163, for the matrix Summing all elements in the matrix and taking the average, the integer part of the average is denoted as . ;

[0053] Step 164, for the image For any pixel in the array, select the 8 neighboring pixels centered at that pixel (i.e., pixels within a 3×3 neighborhood centered at that pixel). The result will be greater than or equal to... The neighboring pixels are marked as 1, and those smaller than 1 are marked as 1. The neighboring pixels are marked as 0. Then, the values ​​of the 8 neighboring pixels are combined in a clockwise direction to form a sequence containing an 8-bit binary number. For example, starting from the top left corner of a 3×3 neighborhood, the resulting sequence may be 00011000. The sequence is converted to obtain the new pixel value. Specifically, the last four bits of the sequence 1000 are converted to the hexadecimal number 4, and the first four bits of the sequence 0001 are converted to the hexadecimal number 2. The new pixel value obtained by the sequence conversion is 24. The maximum new pixel value cannot exceed 255.

[0054] Traversing the image After selecting the pixels, the image with added noise is obtained. It should be noted that only pixels with 8 neighboring pixels need to be processed here; boundary pixels do not need to be processed.

[0055] The other steps and parameters are the same as in Specific Implementation Method Two.

[0056] Specific Implementation Method Four: This implementation method further defines Specific Implementation Method Three. In step one seven, determining whether the stopping condition of the iteration is met specifically involves:

[0057] (1) Calculate the image The Middle row pixels and the first Correlation of row pixels :

[0058]

[0059] in, , Representing an image The Middle Line 1 The pixel values ​​of the columns of pixels. Representing an image The Middle The average pixel value of the row of pixels. Representing an image The Middle Line 1 The pixel values ​​of the columns of pixels. Representing an image The Middle The average pixel value of the row of pixels;

[0060]

[0061] like If it is less than or equal to the set horizontal correlation threshold, then the first... row pixels and the first The row pixels meet the correlation requirements;

[0062] like If the correlation is greater than the set horizontal correlation threshold, then the first... row pixels and the first The row pixels do not meet the correlation requirements;

[0063] (2) Similarly, the horizontal correlation calculation method is used to calculate the correlation in the column direction;

[0064] If any two adjacent rows of pixels satisfy the correlation requirement and any two adjacent columns of pixels satisfy the correlation requirement, then the image The stopping condition for iteration is met; otherwise, the image... The stopping condition for iteration is not met.

[0065] The other steps and parameters are the same as in Specific Implementation Method 3.

[0066] The horizontal and vertical correlation thresholds of this invention can be set according to actual needs, and can be set according to the encryption requirements.

[0067] Specific Implementation Method Five: This implementation method further defines Specific Implementation Method One. The clustering of each face template image based on the extracted feature vectors is as follows:

[0068] Step 21, place the first The feature vector of the face template image is denoted as . , , respectively, feature vectors The first, second, ..., the first One element, For feature vectors The dimension;

[0069] Step 22: Calculate the first... Intermediate variables corresponding to the Zhang face template image :

[0070]

[0071] in, Indicates the first The first feature vector of the face template image One element, Indicates calculation and European distance, This indicates the set distance threshold. Represents a collection of images in a database;

[0072] make ,like ,but ,like ,but ;

[0073] Then, the intermediate variables corresponding to each face template image Sort the features from largest to smallest and select the top K feature vectors.

[0074] Steps two and three: The one ranked... The eigenvector of a bit is denoted as , ,Will , As the first set of initial cluster centers, then based on the first set of initial cluster centers, we obtain... The new initial cluster centers are obtained, i.e., the total number of clusters is 1. Initial cluster centers;

[0075] The specific process of steps two and three is as follows:

[0076] Step 231: Remove the face template images corresponding to the first set of initial cluster centers from the face template image database, and denote the set of the remaining face template images as follows: ;

[0077] Step 232: Initialization ;

[0078] Steps two and three: From the set Select the feature vector and nearest image Using images eigenvectors Replace the first set of initial cluster centers This yields a new set of initial cluster centers;

[0079] Steps two, three, and four: Determine if the conditions are met. :

[0080] If satisfied Then you will get Form new initial cluster centers;

[0081] If not satisfied Then let Return to step two or three;

[0082] Step 24: Determine the final cluster centers based on the initial cluster centers obtained in Step 23;

[0083] Step 25: Based on the final cluster centers obtained in Step 24, cluster each face template image in the database to obtain the face template images contained in each category.

[0084] The other steps and parameters are the same as in Specific Implementation Method 1.

[0085] This implementation uses clustering to group face template images with similar feature vectors into one category. Based on the final clustering results, the final cluster center for each category can be obtained. Thus, when matching face images to be identified, the distance between the feature vector of the face image to be identified and each final cluster center can be calculated to determine the category corresponding to the minimum distance. Then, by calculating the similarity between the feature vector of the face image to be identified and the feature vector of each image in this category, the face template image with the highest similarity to the face image to be identified can be obtained.

[0086] Specific Implementation Method Six: This implementation method is a further limitation of Specific Implementation Method Five. The specific process of step two and four is as follows:

[0087] Step 241: Initialize each initial cluster center obtained in Step 23 as a gray wolf individual in the first iteration of the gray wolf optimization algorithm;

[0088] Step 242: Initialize the number of iterations ;

[0089] Steps two, four, and three: Calculate the first, fourth, and third steps respectively. The objective function value corresponding to each individual gray wolf in each iteration;

[0090] Step 244, from the first In the next iteration, the top three gray wolf individuals with the largest objective function values ​​are selected, and the others... Each individual gray wolf is updated based on the three selected gray wolf individuals;

[0091] The three selected gray wolf individuals and the updated The first gray wolf individual as the first Each individual gray wolf in the next iteration;

[0092] Steps two, four, and five: Determine if the conditions are met. ; The set threshold;

[0093] If satisfied Then from the first After the next iteration Select the gray wolf with the largest objective function value from the gray wolf individuals, and then execute steps two, four, and six.

[0094] If not satisfied Then let Return to steps two, four, and three;

[0095] Steps 2, 4, and 6: Take each feature vector of the gray wolf individuals selected in Steps 2, 4, and 5 as a cluster center.

[0096] The other steps and parameters are the same as in Specific Implementation Method 5.

[0097] Specific Implementation Method Seven: This implementation method further defines Specific Implementation Method Six. The method for calculating the objective function value is as follows:

[0098] For any individual gray wolf:

[0099] Calculate the first The distance between the feature vector of the face template image and each feature vector of the individual gray wolf is then calculated. The feature vector of the Zhang face template image belongs to the cluster center corresponding to the individual gray wolf with the minimum distance;

[0100] Similarly, the cluster center corresponding to the feature vector of each face template image in the individual gray wolf is calculated.

[0101] Then the objective function value of this individual gray wolf for:

[0102]

[0103] in, Indicates belonging to the cluster center A collection of images.

[0104] The other steps and parameters are the same as in Specific Implementation Method Six.

[0105] Specific Implementation Method Eight: This implementation method is a further limitation of Specific Implementation Method Seven. The specific process of step two five is as follows:

[0106] Step 251: Denote the cluster centers obtained in Step 24 as follows: , , ;

[0107] Step 252: Initialize the number of iterations ;

[0108] Steps 2-53: Calculate the... Feature vector of a face template image The distance to each cluster center, then the first The feature vector of the face template image belongs to the cluster center corresponding to the minimum distance. After traversing all face template images in the database, the first one is obtained. The clustering results of the next iteration;

[0109] Step 254, according to the... The clustering results of the next iteration update the cluster centers of each cluster, and the updated cluster centers of each cluster are denoted as follows: ;

[0110] Step 255: Determine the cluster centers and Compared to whether there has been a change, ;

[0111] like If all cluster centers remain unchanged, then the iteration stops to obtain the final clustering result, i.e., the final clustering result. A cluster;

[0112] Otherwise, let Return to step two, five, and three.

[0113] The other steps and parameters are the same as in Specific Implementation Method Seven.

[0114] Specific Implementation Method Nine: This implementation method is a further limitation of Specific Implementation Method One. The working process of the feature extraction model is as follows:

[0115] The face region image is used as the input to the feature extraction model. In the feature extraction model, the input face region image first passes through the first convolutional layer with a kernel size of 3×3. Then, the output of the first convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e obtained by the division passes through the second convolutional layer with a kernel size of 3×3. The output of the second convolutional layer then passes through the first BN layer, the first ReLU activation function layer, the third convolutional layer with a kernel size of 3×3, and the second BN layer in sequence.

[0116] Then, add the first half e obtained from the division to the output of the second BN layer, and use the sum as the input of the second ReLU activation function layer;

[0117] The resulting second half f passes through a first improved multi-head self-attention mechanism layer. The output of the first improved multi-head self-attention mechanism layer is used as the input of the first fully connected layer. The output of the first fully connected layer then passes through a first depthwise separable convolutional layer and a first GeLU activation function layer in sequence. The output of the first fully connected layer is added to the output of the first GeLU activation function layer, and the result of the addition is used as the input of the second fully connected layer.

[0118] The output of the second ReLU activation function layer and the output of the second fully connected layer are used as the input of the first feature fusion unit, and the output of the first feature fusion unit is passed through a fourth convolutional layer with a kernel size of 3×3.

[0119] Then, the output of the fourth convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e' is passed through the fifth convolutional layer with a kernel size of 3×3. The output of the fifth convolutional layer then passes through the third BN layer, the third ReLU activation function layer, the sixth convolutional layer with a kernel size of 3×3, and the fourth BN layer in sequence.

[0120] Then, add the first half e' obtained from the partitioning to the output of the fourth BN layer, and use the result as the input of the fourth ReLU activation function layer;

[0121] The resulting second half f' passes through the second improved multi-head self-attention mechanism layer. The output of the second improved multi-head self-attention mechanism layer is used as the input of the third fully connected layer. The output of the third fully connected layer then passes through the second depthwise separable convolutional layer and the second GeLU activation function layer in sequence. The output of the third fully connected layer is added to the output of the second GeLU activation function layer, and the result of the addition is used as the input of the fourth fully connected layer.

[0122] The outputs of the fourth ReLU activation function layer and the fourth fully connected layer are used as the inputs to the second feature fusion unit;

[0123] The output of the second feature fusion unit is passed through a seventh convolutional layer with a kernel size of 3×3;

[0124] Then, the output of the seventh convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e'' is passed through the eighth convolutional layer with a kernel size of 3×3. The output of the eighth convolutional layer then passes through the fifth BN layer, the fifth ReLU activation function layer, the ninth convolutional layer with a kernel size of 3×3, and the sixth BN layer in sequence.

[0125] Then, add the first half e'' obtained from the partitioning to the output of the sixth BN layer, and use the result as the input of the sixth ReLU activation function layer;

[0126] The resulting second half f'' passes through the third improved multi-head self-attention mechanism layer. The output of the third improved multi-head self-attention mechanism layer is used as the input of the fifth fully connected layer. The output of the fifth fully connected layer then passes through the third depthwise separable convolutional layer and the third GeLU activation function layer in sequence. The output of the fifth fully connected layer is added to the output of the third GeLU activation function layer, and the result of the addition is used as the input of the sixth fully connected layer.

[0127] The output of the sixth ReLU activation function layer is passed through the first max pooling layer and the first average pooling layer respectively. Then the outputs of the first max pooling layer and the first average pooling layer are added together, and the result is denoted as g.

[0128] The output of the sixth fully connected layer is passed through the second max pooling layer and the second average pooling layer respectively. Then the outputs of the second max pooling layer and the second average pooling layer are added together, and the sum is recorded as h.

[0129] Finally, g and h are weighted and fused, and the weighted fusion result is flattened into a one-dimensional vector to obtain the features of the input image.

[0130] The other steps and parameters are the same as in Specific Implementation Method 1.

[0131] The working process of the first improved multi-head self-attention mechanism layer is as follows:

[0132]

[0133]

[0134]

[0135] in, Indicates average pooling;

[0136] Then the first The output of each attention head is:

[0137]

[0138] in, Features In terms of dimensions, the output of the first improved multi-head self-attention mechanism layer is: .

[0139] Specific Implementation Method Ten: This implementation method is a further limitation of Specific Implementation Method Nine. The working process of the first feature fusion unit is as follows:

[0140] The output of the second ReLU activation function layer is flattened, and the result of the flattening is denoted as... Let the output of the second fully connected layer be denoted as Then query vector Key vector Sum value vector They are respectively:

[0141]

[0142]

[0143]

[0144] in, , and This is the weight matrix;

[0145] Query vector Key vector Sum value vector They are respectively:

[0146]

[0147]

[0148]

[0149] The output of the first feature fusion unit for:

[0150]

[0151]

[0152]

[0153] in, Indicates feature reshaping (reshaping the output) (The dimension is the same as the output dimension of the first convolutional layer). express Dimensions express Dimensions.

[0154] The other steps and parameters are the same as in Specific Implementation Method Nine.

[0155] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A face recognition method based on deep learning and image encryption, characterized in that, The method specifically includes the following steps: Step 1: Each edge device encrypts each collected face template image and then sends the encrypted face images to the cloud. The cloud decrypts each received face image and uses the decrypted face images to form a face image database. Step 2: Use a feature extraction model to extract the feature vector of each face template image in the database, and then cluster each face template image according to the extracted feature vector to obtain the face template images contained in each category. Step 3: After encrypting the face image to be identified, send the encrypted face image to the cloud. The cloud decrypts the encrypted image and uses a feature extraction model to extract the feature vector of the face image to be identified. Step 4: Calculate the distance between the center of each category in the clustering results of Step 2 and the feature vector of the face image to be identified. Select the category center with the smallest distance to the feature vector of the face image to be identified, and take the cluster corresponding to the selected category center as the cluster corresponding to the face image to be identified. Then, calculate the similarity between the face image to be identified and each image in the corresponding cluster of the face image to be identified, and obtain the recognition result of the face image to be identified based on the face template image corresponding to the highest similarity.

2. The face recognition method based on deep learning and image encryption according to claim 1, characterized in that, The encryption process for each collected face template image is specifically as follows: For any sheet of size Face template image : Step 11: Initialize the number of iterations ; Steps 1 and 2: Generate a length of row vectors , ,satisfy And vector No two elements in the array are equal; According to the vector For images The image is obtained by reordering the pixels in each column. Arrange the vectors in ascending order. Sort the elements in the image, and then process the image according to the order of the elements in the sorting result. The pixels in each column are reordered to obtain the image. ; Step 13, to Randomly sort the vectors to obtain the vectors. , Then the image The Column pixels move upwards in a loop Bits, to obtain the image ; in: ; Step 14: Generate a length of column vectors , ,satisfy And vector No two elements in the array are equal; According to the vector For images The image is obtained by reordering the pixels in each row. Arrange the vectors in ascending order. Sort the elements in the image, and then process the image according to the order of the elements in the sorting result. The pixels in each row are reordered to obtain the image. ; Step 15, to Randomly sort the vectors to obtain the vectors. , Then the image The Row pixels move to the left in a loop Bits, to obtain the image ; Step 16: Image Noise is added to obtain the image after noise processing. ; Step 17: Determine if the iteration stopping condition is met. If the stopping condition is met, use the image with added noise obtained from the last iteration as the final encrypted image. If the stopping condition is not met, then let... For images Return to steps one and two, that is, continue processing the image. Encryption processing is performed.

3. The face recognition method based on deep learning and image encryption according to claim 2, characterized in that, The specific process of step one six is ​​as follows: Step 161: Based on the image Calculate intermediate variables : in, Representing an image The Middle Line 1 The pixel values ​​of the column; Give image Add a mean of 0 and a variance of Gaussian noise was used to obtain the image. ;in, It is a constant; Step 162: Randomly generate a sparse matrix Using sparse matrices and images Generating matrix : Step 163, for the matrix Summing all elements in the matrix and taking the average, the integer part of the average is denoted as . ; Step 164, for the image For any pixel in the array, select the 8 neighboring pixels centered at that pixel, and then select the pixels that are greater than or equal to... The neighboring pixels are marked as 1, and those smaller than 1 are marked as 1. The neighboring pixels are marked as 0, and then the values ​​of 8 neighboring pixels are combined in a clockwise direction to form a sequence containing an 8-bit binary number; Traversing the image After selecting the pixels, the image with added noise is obtained. .

4. The face recognition method based on deep learning and image encryption according to claim 3, characterized in that, In step 17, determining whether the stopping condition for the iteration is met specifically involves: (1) Calculate the image The Middle row pixels and the first Correlation of row pixels : in, , Representing an image The Middle Line 1 The pixel values ​​of the columns of pixels. Representing an image The Middle The average pixel value of the row of pixels. Representing an image The Middle Line 1 The pixel values ​​of the columns of pixels. Representing an image The Middle The average pixel value of the row of pixels; like If it is less than or equal to the set horizontal correlation threshold, then the first... row pixels and the first The row pixels meet the correlation requirements; like If the correlation is greater than the set horizontal correlation threshold, then the first... row pixels and the first The row pixels do not meet the correlation requirements; (2) Similarly, the horizontal correlation calculation method is used to calculate the correlation in the column direction; If any two adjacent rows of pixels satisfy the correlation requirement and any two adjacent columns of pixels satisfy the correlation requirement, then the image The stopping condition for iteration is met; otherwise, the image... The stopping condition for iteration is not met.

5. The face recognition method based on deep learning and image encryption according to claim 1, characterized in that, The step of clustering each face template image based on the extracted feature vectors is as follows: Step 21, place the first The feature vector of the face template image is denoted as . , , They are the feature vectors The first, second, ..., the first One element, For feature vectors The dimension; Step 22, Calculate the first Intermediate variables corresponding to the Zhang face template image : in, Indicates the first The first feature vector of the face template image One element, Indicates calculation and European distance, This indicates the set distance threshold. Represents a collection of images in a database; make ,like ,but ,like ,but ; Then, the intermediate variables corresponding to each face template image Sort the features from largest to smallest and select the top K feature vectors. Steps two and three: The one ranked... The eigenvector of a bit is denoted as , ,Will , As the first set of initial cluster centers, then based on the first set of initial cluster centers, we obtain... The new initial cluster centers are obtained, i.e., the total number of clusters is 1. Initial cluster centers; The specific process of steps two and three is as follows: Step 231: Remove the face template images corresponding to the first set of initial cluster centers from the face template image database, and denote the set of the remaining face template images as follows: ; Step 232: Initialization ; Steps two and three: From the set Select the feature vector and nearest image Using images eigenvectors Replace the first set of initial cluster centers This yields a new set of initial cluster centers; Steps two, three, and four: Determine if the conditions are met. : If satisfied Then you will get Form new initial cluster centers; If not satisfied Then let Return to step two or three; Step 24: Determine the final cluster centers based on the initial cluster centers obtained in Step 23; Step 25: Based on the final cluster centers obtained in Step 24, cluster each face template image in the database to obtain the face template images contained in each category.

6. The face recognition method based on deep learning and image encryption according to claim 5, characterized in that, The specific process of step two or four is as follows: Step 241: Initialize each initial cluster center obtained in Step 23 as a gray wolf individual in the first iteration of the gray wolf optimization algorithm; Step 242: Initialize the number of iterations ; Steps two, four, and three: Calculate the first, fourth, and third steps respectively. The objective function value corresponding to each individual gray wolf in each iteration; Step 244, from the first In the next iteration, the top three gray wolf individuals with the largest objective function values ​​are selected, and the others... Each individual gray wolf is updated based on the three selected gray wolf individuals; The three selected gray wolf individuals and the updated The first gray wolf individual as the first Each individual gray wolf in the next iteration; Steps two, four, and five: Determine if the conditions are met. ; The set threshold; If satisfied Then from the first After the next iteration Select the gray wolf with the largest objective function value from the gray wolf individuals, and then execute steps two, four, and six. If not satisfied Then let Return to steps two, four, and three; Steps 2, 4, and 6: Take each feature vector of the gray wolf individuals selected in Steps 2, 4, and 5 as a cluster center.

7. A face recognition method based on deep learning and image encryption according to claim 6, characterized in that, The method for calculating the objective function value is as follows: For any individual gray wolf: Calculate the first The distance between the feature vector of the face template image and each feature vector of the individual gray wolf is then calculated. The feature vector of the Zhang face template image belongs to the cluster center corresponding to the individual gray wolf with the minimum distance; Similarly, the cluster center corresponding to the feature vector of each face template image in the individual gray wolf is calculated. Then the objective function value of this individual gray wolf for: in, Indicates belonging to the cluster center A collection of images.

8. A face recognition method based on deep learning and image encryption according to claim 7, characterized in that, The specific process of step two five is as follows: Step 251: Denote the cluster centers obtained in Step 24 as follows: , , ; Step 252: Initialize the number of iterations ; Steps 2-53: Calculate the... Feature vector of a face template image The distance to each cluster center, then the first The feature vector of the face template image belongs to the cluster center corresponding to the minimum distance. After traversing all face template images in the database, the first one is obtained. The clustering results of the next iteration; Step 254, according to the... The clustering results of the next iteration update the cluster centers of each cluster, and the updated cluster centers of each cluster are denoted as follows: ; Step 255: Determine the cluster centers and Compared to whether there has been a change, ; like If all cluster centers remain unchanged, then the iteration stops to obtain the final clustering result, i.e., the final clustering result. A cluster; Otherwise, let Return to step two, five, and three.

9. A face recognition method based on deep learning and image encryption according to claim 1, characterized in that, The working process of the feature extraction model is as follows: The face region image is used as the input to the feature extraction model. In the feature extraction model, the input face region image first passes through the first convolutional layer with a kernel size of 3×3. Then, the output of the first convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e obtained by the division passes through the second convolutional layer with a kernel size of 3×3. The output of the second convolutional layer then passes through the first BN layer, the first ReLU activation function layer, the third convolutional layer with a kernel size of 3×3, and the second BN layer in sequence. Then, add the first half e obtained from the division to the output of the second BN layer, and use the sum as the input of the second ReLU activation function layer; The resulting second half f passes through a first improved multi-head self-attention mechanism layer. The output of the first improved multi-head self-attention mechanism layer is used as the input of the first fully connected layer. The output of the first fully connected layer then passes through a first depthwise separable convolutional layer and a first GeLU activation function layer in sequence. The output of the first fully connected layer is added to the output of the first GeLU activation function layer, and the result of the addition is used as the input of the second fully connected layer. The output of the second ReLU activation function layer and the output of the second fully connected layer are used as the input of the first feature fusion unit, and the output of the first feature fusion unit is passed through a fourth convolutional layer with a kernel size of 3×3. Then, the output of the fourth convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e' is passed through the fifth convolutional layer with a kernel size of 3×3. The output of the fifth convolutional layer then passes through the third BN layer, the third ReLU activation function layer, the sixth convolutional layer with a kernel size of 3×3, and the fourth BN layer in sequence. Then, add the first half e' obtained from the partitioning to the output of the fourth BN layer, and use the result as the input of the fourth ReLU activation function layer; The resulting second half f' passes through the second improved multi-head self-attention mechanism layer. The output of the second improved multi-head self-attention mechanism layer is used as the input of the third fully connected layer. The output of the third fully connected layer then passes through the second depthwise separable convolutional layer and the second GeLU activation function layer in sequence. The output of the third fully connected layer is added to the output of the second GeLU activation function layer, and the result of the addition is used as the input of the fourth fully connected layer. The outputs of the fourth ReLU activation function layer and the fourth fully connected layer are used as the inputs to the second feature fusion unit; The output of the second feature fusion unit is passed through a seventh convolutional layer with a kernel size of 3×3; Then, the output of the seventh convolutional layer is integrated and recombined along the channel dimension through pointwise convolution. The recombined result is divided into two parts along the channel dimension. The first half e'' is passed through the eighth convolutional layer with a kernel size of 3×3. The output of the eighth convolutional layer then passes through the fifth BN layer, the fifth ReLU activation function layer, the ninth convolutional layer with a kernel size of 3×3, and the sixth BN layer in sequence. Then, add the first half e'' obtained from the partitioning to the output of the sixth BN layer, and use the result as the input of the sixth ReLU activation function layer; The resulting second half f'' passes through the third improved multi-head self-attention mechanism layer. The output of the third improved multi-head self-attention mechanism layer is used as the input of the fifth fully connected layer. The output of the fifth fully connected layer then passes through the third depthwise separable convolutional layer and the third GeLU activation function layer in sequence. The output of the fifth fully connected layer is added to the output of the third GeLU activation function layer, and the result of the addition is used as the input of the sixth fully connected layer. The output of the sixth ReLU activation function layer is passed through the first max pooling layer and the first average pooling layer respectively. Then the outputs of the first max pooling layer and the first average pooling layer are added together, and the result is denoted as g. The output of the sixth fully connected layer is passed through the second max pooling layer and the second average pooling layer respectively. Then the outputs of the second max pooling layer and the second average pooling layer are added together, and the sum is recorded as h. Finally, g and h are weighted and fused, and the weighted fusion result is flattened into a one-dimensional vector to obtain the features of the input image.

10. A face recognition method based on deep learning and image encryption according to claim 9, characterized in that, The working process of the first feature fusion unit is as follows: The output of the second ReLU activation function layer is flattened, and the result of the flattening is denoted as... Let the output of the second fully connected layer be denoted as Then query vector Key vector Sum value vector They are respectively: in, , and This is the weight matrix; Query vector Key vector Sum value vector They are respectively: The output of the first feature fusion unit for: in, Indicates feature reshaping, express Dimensions express Dimensions.

Images