A privacy-preserving face recognition method based on MindSpore

Abstract

The present invention discloses a privacy-preserving face recognition method based on MindSpore. The method comprises the following steps: firstly, a face image is read and cut into uniform width and height, a horizontal and vertical 2DPCA is used to obtain an optimal projection matrix of a data set and normalize the data set, and then Laplace noise is added to the normalized optimal projection matrix to obtain a disturbed projection matrix, and finally a randomized reconstructed image is obtained. The method of the present invention is used in face recognition technology to protect user face information and prevent privacy leakage, and differential privacy and a horizontal and vertical 2DPCA method are introduced into face recognition, thereby improving the security of user face image information and improving face recognition efficiency.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and data security, and in particular to a privacy-preserving face recognition method based on MindSpore. Background Art

[0002] In today's highly information-based society, the rich perception and computing capabilities of smart devices have led to the continuous generation of data related to user privacy, and face recognition products and applications are emerging in an endless stream. Face is a sensitive biological information. Third-party servers may be maliciously accessed, resulting in the leakage of privacy data such as biometric information, which has created a major hidden danger of user privacy data leakage. Digital images often contain rich personal privacy information, such as pictures containing facial information, which are complex and easy to share. With the introduction of the "Personal Information Protection Law", the protection of personal privacy information has become increasingly valued. Untrustworthy third parties may pose a threat to personal privacy security. Therefore, the release of privacy-protected images has received widespread attention.

[0003] When the attacker has specific background knowledge, anonymization techniques such as k-same anonymization proposed by Newton and Gross et al. will leak the user's privacy information to a certain extent. Sadegh et al. proposed homomorphic encryption of grayscale images to protect image privacy, but its computational complexity is high. Differential privacy technology does not care how much background knowledge the attacker has, and achieves the effect of privacy protection by adding appropriate noise to the query or analysis results. Publishing technology based on differential privacy has involved a variety of data types. Although the PCA feature face method is widely used in facial feature extraction, the 2DPCA method is faster and more effective. How to use differential privacy protection technology to publish facial images and maintain a balance between accuracy and privacy is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a privacy-preserving face recognition method based on MindSpore to maintain a balance between face recognition accuracy and image privacy.

[0005] The technical solution to achieve the purpose of the present invention is: a privacy-preserving face recognition method based on MindSpore, built on the MindSpore deep learning framework, where data is transmitted from the client to the server, including the following steps:

[0006] Step 1: Get the face image: crop the face image into a uniform size, with a width of w and a height of h;

[0007] Step 2: Get the projection matrix and feature matrix: According to the two-dimensional matrix A of the face image in the data set i , A i ∈R h×w , generate the covariance matrix G based on the horizontal and vertical directions respectively t and H t , and further obtain the optimal projection matrix U and feature matrix B in the horizontal direction, as well as the optimal projection matrix V and feature matrix C in the vertical direction;

[0008] Step 3: Normalize the projection matrix based on the horizontal direction: normalize the generated feature matrix U data based on the horizontal direction to [0,1];

[0009] Step 4, generating a disturbed projection matrix based on the horizontal direction: adding Laplace noise to the projection matrix based on the horizontal direction to generate a disturbed projection matrix U′;

[0010] Step 5: Reconstruct the face image: Generate the reconstructed face image F based on the projection matrix U obtained by the perturbed horizontal 2DPCA method and the projection matrix V obtained by the vertical 2DPCA method. new ;

[0011] Step 6: Train the reconstructed face image dataset: Use the multi-layer perceptron MLP to train the reconstructed face image dataset for classification.

[0012] Furthermore, in step 2, the projection matrix and feature matrix are generated. Specifically, 2DPCA is implemented based on the MindSpore open source computing framework and the projection matrix is ​​generated. The steps are as follows:

[0013] Step 2.1, get the data set: Assume there are N samples, the pictures that constitute the training set need to be taken under the same lighting conditions, and the eyes and mouths of all images need to be aligned. The width of the image sample is w, the height is h, and it is represented as a two-dimensional matrix A i ∈R h ×w , data set S = A1,…,A n , i=1,…,N;

[0014] Step 2.2: Center the image: Each image matrix needs to be subtracted from the mean matrix according to Get the centralized image CA i ;

[0015] Step 2.3: Obtain the covariance matrix based on the 2DPCA method in the horizontal direction G t ∈R n×n ;

[0016] Step 2.4: According to G t u i =λ i u iPerform eigenvalue decomposition, λ i represents the eigenvalue, u i represents the feature vector;

[0017] Step 2.5: Sort the n eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form the horizontal transformation matrix U = [e1, e2, …, e k ],U∈R n×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space U, and the image feature matrix B = AU;

[0018] Step 2.6: Obtain the covariance matrix based on the 2DPCA method in the vertical direction H t ∈R m ×m ;

[0019] Step 2.7: According to H t v i =λ i v i Perform eigenvalue decomposition, λ i represents the eigenvalue, v i represents the feature vector;

[0020] Step 2.8: Sort the m eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form a vertical transformation matrix V = [e1, e2, …, e k ] T , V∈R m×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space V, and the image feature matrix C = V T A.

[0021] Furthermore, the perturbed horizontal projection matrix generated in step 4 is as follows:

[0022] Based on the normalized horizontal projection matrix U, the index position of each column of the projection matrix Add Laplacian noise Lap(1 / ε) with a sensitivity of 1 and a noise scale of 1 / ε to generate the perturbed projection matrix U′, where ε represents the privacy budget.

[0023] Furthermore, the facial image is reconstructed in step 5, specifically as follows:

[0024] According to F new =VV TAU′U′ T Get the reconstructed face image F new , where V represents the projection matrix based on the vertical direction, and U′ represents the projection matrix based on the horizontal direction after adding noise;

[0025] Reconstruct the face image F according to the noisy projection matrix U′ new , the client uploads the reconstructed face image to the server.

[0026] Furthermore, the training and reconstruction of the face image dataset in step 6 is performed in the following specific steps:

[0027] Take 70% of the reconstructed face image dataset as the training set and 30% as the test set, send the images and labels to the network for training and save the model, and perform face recognition through the multi-layer perceptron MLP training dataset;

[0028] Multilayer Perceptron MLP is a feedforward neural network. Multilayer Perceptron MLP is a neural network composed of fully connected layers with at least one hidden layer. The neural network consists of two fully connected layers and one activation function. The first fully connected layer FC1 has w*h nodes at the input and 512 nodes at the output. The number of input nodes of the activated fully connected layer is 512, and the number of output nodes is the number of categories of the face set.

[0029] Further, in step 2.4, according to G t u i =λ i u i Perform eigenvalue decomposition as follows:

[0030] The covariance matrix is ​​a high-dimensional matrix C∈R n×n , when n is much larger than m, first perform A T The eigenvalue decomposition of A, A T A∈R m×m , get A T Av i =β i v i , assuming that β1,β2,…,β m Yes A T The eigenvalues ​​of A, v1, v2, …, v m is the eigenvector corresponding to the eigenvalue, multiplying both sides by A to get AA T Av i =β i Av i , i.e. CAv i =β i Av i ,β1,β2,…,β m Also AA TThe eigenvalues ​​of Av1, Av2, …, Av m is the eigenvector corresponding to the eigenvalue. In fact, AA T There should be n eigenvalues ​​and n linearly independent eigenvectors, and A T The m eigenvalues ​​and eigenvectors of A correspond to AA T The first m largest eigenvalues ​​and the eigenvectors corresponding to the eigenvalues; Through this transformation, AA is obtained T The eigenvector u i =Av i ,i=1,2,...,m.

[0031] Compared with the prior art, the present invention has the following significant advantages: (1) In 2DPCA, the covariance matrix of the image is directly constructed by using the original image matrix, which is much smaller in dimension than the covariance matrix constructed by PCA, thus reducing the time required to calculate the eigenvector; (2) the data is disturbed before being sent to a third party, which better ensures the privacy of the data; (3) under standard privacy settings, the classification accuracy of the algorithm is 82%-91%. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is the overall framework diagram of the privacy-preserving face recognition method based on MindSpore of the present invention.

[0033] Figure 2 This is a flowchart for generating a projection matrix in the present invention.

[0034] Figure 3 This is a flow chart of the perturbation projection matrix in the present invention. DETAILED DESCRIPTION

[0035] This paper proposes a privacy-preserving face recognition method based on MindSpore, which is built on the MindSpore deep learning framework. Data is transmitted from the edge to the server, and 2DPCA and differential privacy are introduced. Figure 1 , the main steps are as follows:

[0036] Step 1: Get the face image: crop the face image into a uniform size, with a width of w and a height of h;

[0037] Step 2: Get the projection matrix and feature matrix: According to the two-dimensional matrix A of the face image in the data set i , A i ∈R h ×w , generate the covariance matrix G based on the horizontal and vertical directions respectively t and H t, and further obtain the optimal projection matrix U and feature matrix B in the horizontal direction, as well as the optimal projection matrix V and feature matrix C in the vertical direction;

[0038] Step 3: Normalize the projection matrix based on the horizontal direction: normalize the generated feature matrix U data based on the horizontal direction to [0,1];

[0039] Step 4, generating a disturbed projection matrix based on the horizontal direction: adding Laplace noise to the projection matrix based on the horizontal direction to generate a disturbed projection matrix U′;

[0040] Step 5: Reconstruct the face image: Generate the reconstructed face image F based on the projection matrix U obtained by the perturbed horizontal 2DPCA method and the projection matrix V obtained by the vertical 2DPCA method. new ;

[0041] Step 6: Train the reconstructed face image dataset: Use the multi-layer perceptron MLP to train the reconstructed face image dataset for classification.

[0042] Furthermore, before the user sends the face image to the server, the image will be randomized and the face image will be regenerated through the perturbed projection matrix, thereby protecting the image privacy information.

[0043] Furthermore, its server receives the disturbed facial image, and an untrusted third party cannot obtain the facial information contained in the image.

[0044] Furthermore, its server performs face recognition on the reconstructed face dataset and maintains a balance between face recognition accuracy and image privacy by adjusting different parameters such as privacy budgets.

[0045] The face image privacy protection method of the present invention is based on MindSpore, and the 2DPCA method described in step 2 is implemented based on the MindSpore open source computing framework to generate the projection matrix and the feature matrix. Figure 2 The basic steps are as follows:

[0046] Step 2.1, get the data set: Assume there are N samples, the pictures that constitute the training set need to be taken under the same lighting conditions, and the eyes and mouths of all images need to be aligned. The width of the image sample is w, the height is h, and it is represented as a two-dimensional matrix A i ∈R h ×w , data set S = A1,…,A N , i=1,…,N;

[0047] Step 2.2: Center the image: Each image matrix needs to be subtracted from the mean matrix according to Get the centralized image CA i .

[0048] Step 2.3: Obtain the covariance matrix based on the 2DPCA method in the horizontal direction G t ∈R n×n ;

[0049] Step 2.4: According to G t u i =λ i u i Perform eigenvalue decomposition, λ i represents the eigenvalue, u i represents the feature vector;

[0050] Step 2.5: Sort the n eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form the horizontal transformation matrix U = [e1, e2, …, e k ],U∈R n×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space U, and the image feature matrix B = AU;

[0051] Step 2.6: Obtain the covariance matrix based on the 2DPCA method in the vertical direction H t ∈R m ×m ;

[0052] Step 2.7: According to H t v i =λ i v i Perform eigenvalue decomposition, λ i represents the eigenvalue, v i represents the feature vector;

[0053] Step 2.8: Sort the m eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form a vertical transformation matrix V = [e1, e2, …, e k ] T , V∈R m×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space V, and the image feature matrix C = V T A.

[0054] Furthermore, the specific contents of the eigenvalue decomposition method when n is much larger than m are as follows:

[0055] The covariance matrix is ​​a high-dimensional matrix C∈R n×n If we directly perform feature decomposition, it will not only consume resources but also be time-consuming when n is much larger than m. Therefore, when n is much larger than m, we first perform A T The eigenvalue decomposition of A is T A∈R m×m , the dimension is much smaller at this time, so we can get A T Av i =β i v i , assuming that β1,β2,…,β m Yes A T The eigenvalues ​​of A, v1, v2, …, v m is the eigenvector corresponding to the eigenvalue. Multiplying A on both sides gives AA T Av i =β i Av i , i.e. CAv i =β i Av i , we can see that β1,β2,…,β m Also AA T The eigenvalues ​​of Av1, Av2, …, Av m is the eigenvector corresponding to the eigenvalue. In fact, AA T There should be n eigenvalues ​​(considering repeated roots), and n linearly independent eigenvectors, and A T The m eigenvalues ​​and eigenvectors of A correspond to AA T The first m largest eigenvalues ​​and the eigenvectors corresponding to the eigenvalues. Therefore, through this transformation, AA can be quickly obtained T The eigenvector u i =Av i ,i=1,2,...,m.

[0056] Furthermore, the method for generating a disturbed horizontal projection matrix is ​​combined with Figure 3 , the specific contents are as follows:

[0057] Based on the normalized horizontal projection matrix U, the index position of each column of the projection matrix Add Laplacian noise Lap(1 / ε) with a sensitivity of 1 and a noise scale of 1 / ε to generate the perturbed projection matrix U′, where ε represents the privacy budget.

[0058] Furthermore, the method for reconstructing a face image is specifically described as follows:

[0059] According to F new =VV T AU′U′T Get the reconstructed face image F new , where V represents the projection matrix based on the vertical direction, and U′ represents the projection matrix based on the horizontal direction after adding noise. Reconstruct the face image F according to the projection matrix U′ after adding noise new , the client uploads the reconstructed face image to the server.

[0060] Furthermore, a neural network is used for face recognition under the MindSpore open source framework. The training and reconstruction of the face image described in step 6 are as follows:

[0061] Take 70% of the reconstructed face image dataset as the training set, and 30% of it as the test set. Send the pictures and labels to the network for training and save the model. Perform face recognition on the multi-layer perceptron (MLP) training dataset. The multi-layer perceptron is a feedforward neural network that contains at least one hidden layer composed of fully connected layers. The network consists of two fully connected layers and one activation function. The first fully connected layer FC1 has w*h nodes at the input and 512 nodes at the output. The number of input nodes of the activated fully connected layer is 512, and the number of output nodes is the number of categories of the face set.

[0062] Furthermore, the before and after comparison of the projection matrix perturbation is as follows:

[0063] 2DPCA improves the PCA method. It does not need to convert the image into a one-dimensional vector in advance. Instead, it directly generates a general scatter matrix based on the two-dimensional matrix of the image, and then generates a projection matrix based on the eigenvectors corresponding to the k eigenvalues ​​with the largest eigenvalues. The data in the projection matrix is ​​normalized. Sensitivity is used as a parameter of the noise amount, indicating the maximum impact of deleting a record in the data set on the query result. Therefore, its sensitivity is 1. Add Laplace noise to the projection matrix, where ε represents the privacy budget, represents the position of the normalized projection matrix, It represents the perturbation calculated according to the normalized matrix. When adding the perturbation projection matrix to reconstruct the data, the privacy information of the image is protected.

[0064] Furthermore, the inventors used the LFW dataset in the privacy-preserving face recognition method of MindSpore of the present invention, and selected 1,140 face images from 5 people with more than 100 face images, and used 70% of the data as a training set and 30% of the data as a test set. In actual operation, the face recognition rate will be greatly reduced due to different lighting conditions and imaging angles. When using the projection matrix, it is necessary to limit the image to be recognized using the frontal image under uniform lighting conditions.

[0065] Furthermore, the inventor uses a multi-layer perceptron to classify face data sets in the privacy-preserving face recognition method of MindSpore of the present invention. The multi-layer perceptron is a feedforward neural network composed of two fully connected layers and one activation function, which overcomes the weakness of the perceptron that it cannot recognize linearly inseparable data.

[0066] Furthermore, the inventor adjusts parameters in the privacy-preserving face recognition method based on MindSpore of the present invention to maintain a balance between face recognition accuracy and image privacy. As the privacy budget ε increases, the classification accuracy gradually increases and the privacy gradually decreases. Therefore, keeping ε at (0,9] is conducive to maintaining a balance between accuracy and image privacy; and as the number of principal components increases, the more face images each person has, the higher the classification accuracy. Perturbing the data before sending it to a third party better ensures the privacy of the data, resolves the privacy crisis encountered by consumers when using data-driven products or services, and promotes the development of the data industry.

[0067] The present invention will be further described in detail below with reference to examples.

[0068] Example

[0069] This paper proposes a privacy-preserving face recognition method based on MindSpore, which is built on the MindSpore deep learning framework. The data set uses the LFW data set. The data is transmitted from the edge to the server. 2DPCA and differential privacy are introduced. Figure 1 The main steps are as follows:

[0070] Step 1: Get the face image: crop the face image into a uniform size, with a width of w and a height of h;

[0071] Step 2: Get the projection matrix and feature matrix: According to the two-dimensional matrix A of the face image in the data set i , A i ∈R h ×w , generate the covariance matrix G based on the horizontal and vertical directions respectively t and H t , and further obtain the optimal projection matrix U and feature matrix B in the horizontal direction, as well as the optimal projection matrix V and feature matrix C in the vertical direction;

[0072] Step 3: Normalize the projection matrix based on the horizontal direction: normalize the generated feature matrix U data based on the horizontal direction to [0,1];

[0073] Step 4, generating a disturbed projection matrix based on the horizontal direction: adding Laplace noise to the projection matrix based on the horizontal direction to generate a disturbed projection matrix U′;

[0074] Step 5: Reconstruct the face image: Generate the reconstructed face image F based on the projection matrix U obtained by the perturbed horizontal 2DPCA method and the projection matrix V obtained by the vertical 2DPCA method. new ;

[0075] Step 6: Train the reconstructed face image dataset: Use the multi-layer perceptron MLP to train the reconstructed face image dataset for classification.

[0076] Furthermore, before the user sends the face image to the server, the image will be randomized and the face image will be regenerated through the perturbed projection matrix, thereby protecting the image privacy information.

[0077] Furthermore, its server receives the disturbed facial image, and an untrusted third party cannot obtain the facial information contained in the image.

[0078] Furthermore, its server performs face recognition on the reconstructed face dataset and maintains a balance between face recognition accuracy and image privacy by adjusting different parameters such as privacy budgets.

[0079] Table 1 Symbol Description

[0080]

[0081] The present invention proposes a method for protecting the privacy of facial images by perturbing the projection matrix, which is characterized in that the 2DPCA method described in step 2 is implemented based on the MindSpore open source computing framework to generate the projection matrix and the feature matrix, and Figure 2 The basic steps are as follows:

[0082] The 2DPCA method described herein, in combination with Table 1, Figure 2 The basic steps are as follows:

[0083] Step 2.1, get the data set: Assume there are N samples, the pictures that constitute the training set need to be taken under the same lighting conditions, and the eyes and mouths of all images need to be aligned. The width of the image sample is w, the height is h, and it is represented as a two-dimensional matrix A i ∈R h ×w , data set S = A1,…,A N , i=1,…,N;

[0084] Step 2.2: Center the image: Each image matrix needs to be subtracted from the mean matrix according to Get the centralized image CA i .

[0085] Step 2.3: Obtain the covariance matrix based on the 2DPCA method in the horizontal direction G t ∈R n×n ;

[0086] Step 2.4: According to G t u i =λ i u i Perform eigenvalue decomposition, λ i represents the eigenvalue, u i represents the feature vector;

[0087] Step 2.5: Sort the n eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form the horizontal transformation matrix U = [e1, e2, …, e k ],U∈R n×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space U, and the image feature matrix B = AU;

[0088] Step 2.6: Obtain the covariance matrix based on the 2DPCA method in the vertical direction H t ∈R m ×m ;

[0089] Step 2.7: According to H t v i =λ i v i Perform eigenvalue decomposition, λ i represents the eigenvalue, v i represents the feature vector;

[0090] Step 2.8: Sort the m eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form a vertical transformation matrix V = [e1, e2, …, e k ] T , V∈R m×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors form the projection space V, and the image feature matrix C = V T A;

[0091] Furthermore, the specific contents of the eigenvalue decomposition method when n is much larger than m are as follows:

[0092] The covariance matrix is ​​a high-dimensional matrix C∈R n×nIf we directly perform feature decomposition, it will not only consume resources but also be time-consuming when n is much larger than m. Therefore, when n is much larger than m, we first perform A T The eigenvalue decomposition of A is T A∈R m×m , the dimension is much smaller at this time, so we can get A T Av i =β i v i , assuming that β1,β2,…,β m Yes A T The eigenvalues ​​of A, v1, v2, …, v m is the eigenvector corresponding to the eigenvalue. Multiplying A on both sides gives AA T Av i =β i Av i , i.e. CAv i =β i Av i , we can see that β1,β2,…,β m Also AA T The eigenvalues ​​of Av1, Av2, …, Av m is the eigenvector corresponding to the eigenvalue. In fact, AA T There should be n eigenvalues ​​(considering repeated roots), and n linearly independent eigenvectors, and A T The m eigenvalues ​​and eigenvectors of A correspond to AA T The first m largest eigenvalues ​​and the eigenvectors corresponding to the eigenvalues. Therefore, through this transformation, AA can be quickly obtained T The eigenvector u i =Av i ,i=1,2,...,m.

[0093] Furthermore, the method for generating a disturbed horizontal projection matrix is ​​combined with Figure 3 , the specific contents are as follows:

[0094] Based on the normalized horizontal projection matrix U, the index position of each column of the projection matrix Add Laplacian noise Lap(1 / ε) with a sensitivity of 1 and a noise scale of 1 / ε to generate the perturbed projection matrix U′, where ε represents the privacy budget.

[0095] Furthermore, the method for reconstructing a face image is specifically described as follows:

[0096] According to F new =VV T AU′U′ T Get the reconstructed face image F new, where V represents the projection matrix based on the vertical direction, and U′ represents the projection matrix based on the horizontal direction after adding noise. Reconstruct the face image F according to the projection matrix U′ after adding noise new , the client uploads the reconstructed face image to the server.

[0097] Furthermore, a neural network is used for face recognition under the MindSpore open source framework. The training and reconstruction of the face image described in step 6 are as follows:

[0098] Take 70% of the reconstructed face image dataset as the training set, and 30% of it as the test set. Send the pictures and labels to the network for training and save the model. Perform face recognition on the multi-layer perceptron (MLP) training dataset. The multi-layer perceptron is a feedforward neural network that contains at least one hidden layer composed of fully connected layers. The network consists of two fully connected layers and one activation function. The first fully connected layer FC1 has w*h nodes at the input and 512 nodes at the output. The number of input nodes of the activated fully connected layer is 512, and the number of output nodes is the number of categories of the face set.

[0099] Furthermore, the before and after comparison of the projection matrix perturbation is as follows:

[0100] 2DPCA improves the PCA method. It does not need to convert the image into a one-dimensional vector in advance. Instead, it directly generates a general scatter matrix based on the two-dimensional matrix of the image, and then generates a projection matrix based on the eigenvectors corresponding to the k eigenvalues ​​with the largest eigenvalues. The data in the projection matrix is ​​normalized. Sensitivity is used as a parameter of the noise amount, indicating the maximum impact of deleting a record in the data set on the query result. Therefore, its sensitivity is 1. Add Laplace noise to the projection matrix, where ε represents the privacy budget, represents the position of the normalized projection matrix, It represents the perturbation calculated according to the normalized matrix. When adding the perturbation projection matrix to reconstruct the data, the privacy information of the image is protected.

[0101] Furthermore, this embodiment is experimented on a Windows platform with a 2.30 GHz Intel Core i5-6300 processor and 8.00 GB RAM, and the experimental environment is: Python version is 3.7.5, and MindSpore version is 1.3.0.

[0102] Furthermore, the inventors used the LFW dataset in the privacy-preserving face recognition method of MindSpore of the present invention, and selected 1,140 face images from five people with more than 100 face images, namely Colin Powell, Donald Rumsfeld, George W Bush, Gerhard Schroeder and Tony Blair, and used 70% of the data as a training set and 30% of the data as a test set. In actual operation, the face recognition rate will be greatly reduced based on different lighting conditions and imaging angles. When using the projection matrix, it is necessary to limit the image to be recognized using the front image under uniform lighting conditions.

[0103] Furthermore, the inventor uses a multi-layer perceptron to classify face data sets in the privacy-preserving face recognition method of MindSpore of the present invention. The multi-layer perceptron is a feedforward neural network composed of two fully connected layers and one activation function, which overcomes the weakness of the perceptron that it cannot recognize linearly inseparable data.

[0104] Furthermore, the inventor adjusts parameters in the privacy-preserving face recognition method based on MindSpore of the present invention to maintain a balance between face recognition accuracy and image privacy. As the privacy budget ε continues to increase, the classification accuracy gradually increases and the privacy gradually decreases. Therefore, keeping ε at (0,9] is conducive to maintaining a balance between accuracy and image privacy; and as the number of principal components increases, the more face images each person has, the higher the classification accuracy. The data is disturbed before being sent to a third party, and disturbances are added to personal images to effectively cover up the original features of the image, which better guarantees the privacy of the data. This method is sufficiently efficient and lightweight, ensuring the normal use of product functions, resolving the privacy crisis encountered by consumers when using data-driven products or services, and promoting the development of the data industry.

Claims

1. A privacy-preserving face recognition method based on MindSpore, characterized in that: Based on the MindSpore deep learning framework, data is transmitted from the client to the server, including the following steps: Step 1: Get the face image: crop the face image into a uniform size, with a width of w and a height of h; Step 2: Get the projection matrix: According to the two-dimensional matrix A of the face image in the data set i , A i ∈R h×w , generate the covariance matrix G based on the horizontal and vertical directions respectively t and H t , and further obtain the optimal projection matrix U in the horizontal direction and the optimal projection matrix V in the vertical direction, specifically, implementing 2DPCA based on the MindSpore open source computing framework and generating the projection matrix; Step 3: Normalize the projection matrix based on the horizontal direction: normalize the projection matrix U data based on the horizontal direction to [0,1]; Step 4, generating a disturbed projection matrix based on the horizontal direction: adding Laplace noise to the projection matrix based on the horizontal direction to generate a disturbed projection matrix U′; Step 5: Reconstruct the face image: Generate the reconstructed face image F based on the projection matrix U′ obtained by the perturbed horizontal 2DPCA method and the projection matrix V obtained by the vertical 2DPCA method. new ; Step 6: Train the reconstructed face image dataset: Use the multi-layer perceptron MLP to train the reconstructed face image dataset for classification.

2. The privacy-preserving face recognition method based on MindSpore according to claim 1, characterized in that: In step 2, the projection matrix is ​​generated. Specifically, 2DPCA is implemented based on the MindSpore open source computing framework and the projection matrix is ​​generated. The steps are as follows: Step 2.1, get the data set: Assume there are N samples, the pictures that constitute the training set need to be taken under the same lighting conditions, and the eyes and mouths of all images need to be aligned. The width of the image sample is w, the height is h, and it is represented as a two-dimensional matrix A i ∈R h×w , data set S = A1,…,A N , i=1,…,N; Step 2.2: Center the image: Each image matrix needs to be subtracted from the mean matrix according to Get the centralized image CA i ; Step 2.3: Obtain the covariance matrix based on the 2DPCA method in the horizontal direction G t ∈R n×n ; step 2.4 According to G t u i =λ i u i Perform eigenvalue decomposition, λ i represents the eigenvalue, u i represents the feature vector; Step 2.5: Sort the n eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form the horizontal transformation matrix U = [e1, e2, …, e k ],U∈R n×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors constitute the projection space; Step 2.6: Obtain the covariance matrix based on the 2DPCA method in the vertical direction H t ∈R m×m ; Step 2.7: According to H t v i =λ i v i Perform eigenvalue decomposition, λ i represents the eigenvalue, v i represents the feature vector; Step 2.8: Sort the m eigenvalues ​​from large to small, extract the first k eigenvalues ​​and their corresponding eigenvectors, and form a vertical transformation matrix V = [e1, e2, …, e k ] T , V∈R m×k , e i is the eigenvector e i ∈R n×1 , these eigenvectors constitute the projection space.

3. The privacy-preserving face recognition method based on MindSpore according to claim 1, characterized in that: The perturbed horizontal projection matrix generated in step 4 is as follows: Based on the normalized horizontal projection matrix U, the index position of each column of the projection matrix Add Laplacian noise Lap(1 / ε) with a sensitivity of 1 and a noise scale of 1 / ε to generate the perturbed projection matrix U′, where ε represents the privacy budget.

4. The privacy-preserving face recognition method based on MindSpore according to claim 1, characterized in that: The facial image reconstruction described in step 5 is as follows: According to F new =VV T AU′U′ T Get the reconstructed face image F new ; Reconstruct the face image F according to the noisy projection matrix U′ new , the client uploads the reconstructed face image to the server.

5. The privacy-preserving face recognition method based on MindSpore according to claim 1, characterized in that: Step 6 is to train and reconstruct the face image dataset, and the specific steps are as follows: Take 70% of the reconstructed face image dataset as the training set and 30% as the test set, send the images and labels to the network for training and save the model, and perform face recognition through the multi-layer perceptron MLP training dataset; Multilayer Perceptron MLP is a feedforward neural network. Multilayer Perceptron MLP is a neural network composed of fully connected layers with at least one hidden layer. The neural network consists of two fully connected layers and one activation function. The first fully connected layer FC1 has w*h nodes at the input and 512 nodes at the output. The number of input nodes of the activated fully connected layer is 512, and the number of output nodes is the number of categories of the face set.

6. The privacy-preserving face recognition method based on MindSpore according to claim 1, characterized in that: Step 2.4 was described in G t u i =λ i u i Perform eigenvalue decomposition as follows: The covariance matrix is ​​a high-dimensional matrix C∈R n×n , when n is much larger than m, first perform A T The eigenvalue decomposition of A, A T A∈R m ×m , get A T Av i =β i v i , assuming that β1,β2,…,β m Yes A T The eigenvalues ​​of A, v1, v2, …, v m is the eigenvector corresponding to the eigenvalue, multiplying both sides by A to get AA T Av i =β i Av i , i.e. CAv i =β i Av i ,β1,β2,…,β m Also AA T The eigenvalues ​​of Av1, Av2, …, Av m is the eigenvector corresponding to the eigenvalue. In fact, AA T There should be n eigenvalues ​​and n linearly independent eigenvectors, and A T The m eigenvalues ​​and eigenvectors of A correspond to AA T The first m largest eigenvalues ​​and the eigenvectors corresponding to the eigenvalues; Through this transformation, AA is obtained T The eigenvector u i =Av i ,i=1,2,...,m.

Images