A privacy-protected personal privacy image data recognition method and system

Abstract

The present invention relates to the field of privacy protection technology, and in particular to a method and system for identifying personal privacy image data with privacy protection, comprising: S1, obtaining personal privacy image data to be identified; S2, inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection; S3, using the trained neural network model to identify the personal privacy image data to be identified, and outputting the identification result. The present invention can realize the identification of personal privacy image data with privacy protection.

Description

Technical Field

[0001] The present invention relates to the field of privacy protection technology, and in particular to a method and system for identifying personal privacy image data with privacy protection. Background Art

[0002] There is a situation where a company or a department needs to identify uploaded personal privacy image data. For example, it is necessary to use a neural network model to identify which employee's privacy image data belongs to which type of privacy image data, and the entire process needs to be privacy protected to prevent the leakage of personal privacy image data. However, the training and recognition processes of existing neural network models rarely perform privacy protection. Therefore, a privacy-protected personal privacy image data recognition method is urgently needed. Summary of the invention

[0003] The present invention provides a privacy-protected personal privacy image data recognition method and system, which are used to realize the privacy-protected personal privacy image data recognition. The technical solution is as follows:

[0004] On the one hand, a privacy-protected personal privacy image data recognition method is provided, comprising:

[0005] S1. Obtaining personal privacy image data to be identified;

[0006] S2. Inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection;

[0007] S3. Use the trained neural network model to identify the personal privacy image data to be identified, and output the identification result.

[0008] Optionally, the distributed training objective of the neural network model is:

[0009]

[0010] Where x is the training value output by distributed training node i, l i (x,ξ i ) is the local loss function of the distributed training node i inputting a training sample, f i (x) is the loss function of all training samples input to distributed training node i, F(x) is the total loss function of all distributed training nodes, and denote the set of all real numbers and d-dimensional Euclidean space respectively, Represents expectations, A training set representing personal privacy image data;

[0011] Formula (1) can be written as the following minimization problem:

[0012]

[0013] Optionally, the distributed training process of the neural network model is to achieve the distributed training goal and solve the minimization problem (2), making the following assumptions:

[0014] Assumption 1: Communication diagram of each training node during the distributed training process of the neural network model It is connected and undirected, and the adjacency matrix between training nodes is It is double random. 1 of them n represents an n-dimensional column vector whose elements are all 1, and T represents transpose;

[0015] Assumption 2: The loss function of the training node during the distributed training of the neural network model satisfies the following conditions:

[0016] (i) For any training node i, f i (x) having a Lipschitz continuous gradient;

[0017] (ii) The loss function of each training node and its sum start from the lower bound,

[0018] (iii) For For any training node i sampled uniformly, there exists a random first-order oracle that returns f i The sampling gradient g of (x) i (x,ζ i ), there exists a constant σ greater than 0 g , so that each sampled gradient g i (x,ζ i )satisfy represents f i The gradient of (x);

[0019] Assumption 3: The quantizer Q(x) in the distributed training process of the neural network model is unbiased and has a bounded variance, which means that there is a constant Δ greater than 0 such that and Quantizer Q(x) with the following probability distribution:

[0020]

[0021] in represents the probability, is the floor rounding symbol;

[0022] Assumption 4: During the distributed training of the neural network model, for any training time K ≥ 1, 0 ≤ k ≤ K, the training step size Sample size and the privacy noise parameter σ k =(k+1) σ The following conditions are met:

[0023]

[0024]

[0025]

[0026]

[0027] in Represents the adjacency matrix of the training node The Laplace matrix of is a matrix The smallest positive eigenvalue of .

[0028] Optionally, the distributed training process of the neural network model specifically includes:

[0029] S21, initialization is performed;

[0030] Set the initial training node according to assumption 1 and set the initial training value of the training node Initialize the weight matrix between training nodes (a ij ) 1≤i,j≤n , by properly choosing the training step size Differential privacy parameter δ k , privacy noise parameter σ k and quantizer, satisfying assumptions 2-4;

[0031] S22. For training time k=0, 1, 2, ..., K, perform the following steps:

[0032] S221, training node i will make noise n i,k Add to training value x i,k , and then passed to the neighbor training node, and then the quantizer is used to calculate the quantization information in the form of assumption (3) I d represents the identity matrix of d dimensions;

[0033] S222, training node i to train neighboring nodes Send z i,k ;

[0034] S223, training node i receives data from neighboring training nodes z j,k , and from Zhongtong took out without putting back Data samples Returns the sampled gradient

[0035]

[0036] S224, training node i updates its training value by the following method

[0037]

[0038] Optionally, the method further comprises:

[0039] A symmetric binary relationship called adjacency is defined by sampling the change of gradients: For any training node i, let For two sets of personal privacy image data samples, if for a given constant C>0 and any There is at most one pair of personal privacy image data samples in Make

[0040]

[0041] Then it is called and is the adjacent personal privacy image data sample in the distributed training of the neural network model, denoted as

[0042] Optionally, the method further comprises:

[0043] Introduce the concept of (ε,δ)-differential privacy, the (ε,δ)-differential privacy:

[0044] Given ε,δ>0 and algorithm If any private image data subset is satisfy Algorithm for Satisfies (ε,δ)-differential privacy.

[0045] Optionally, the method is private:

[0046] For any training time K ≥ 1, 0 ≤ k ≤ K in the distributed training process of the neural network model, let the training step length Number of personal privacy image sampling samples The privacy noise added during training is chosen as σk =(k+1) σ , the differential privacy parameter is chosen as The other constants are selected as a1, a3>0, 0<a2<1, α, β, σ>0, After adding noise to the training values ​​of the original neural network model distributed training, for two adjacent personal privacy image data samples in the training process The neural network model satisfies - Differential privacy, where

[0047] Optionally, the method has convergence property:

[0048] If the neural network model satisfies assumptions 1-4, then during the training process, for any training node i and training time K≥1, we obtain:

[0049]

[0050] Wherein O() represents an infinitesimal of the same order, indicating that after a certain period of training, the loss function of each training node in the distributed training of the neural network model can be minimized, thereby achieving the distributed training goal, so that the trained model can successfully identify the input personal privacy image data to be identified.

[0051] Optionally, as the number of training times k increases, the training and test accuracy increases accordingly. The smaller Δ and σ are, the higher the accuracy of the neural network model after training. Δ determines the choice of quantizer, and σ determines the situation of adding noise to the model. The smaller Δ and σ are, the smaller the quantization error and noise variance are, and the faster the distributed training converges.

[0052] On the other hand, a privacy-protected personal privacy image data recognition system is provided, comprising:

[0053] An acquisition module, used to acquire the personal privacy image data to be identified;

[0054] An input module, used for inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection;

[0055] The recognition module is used to use the trained neural network model to recognize the personal privacy image data to be recognized and output the recognition result.

[0056] On the other hand, an electronic device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned personal privacy image data recognition method with privacy protection.

[0057] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement the above-mentioned privacy-protected personal privacy image data recognition method.

[0058] Compared with the prior art, the above technical solution has at least the following beneficial effects:

[0059] The present invention can realize the recognition of personal privacy image data with privacy protection, while taking into account training accuracy, differential privacy and communication efficiency. It can successfully protect the privacy security in the distributed training process of the neural network model and can obtain very high training accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 A flowchart of a privacy-protected personal privacy image data identification method provided by an embodiment of the present invention;

[0062] Figure 2 A distributed training topology diagram of a neural network model provided in an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the accuracy of the method provided in the embodiment of the present invention when Δ is 1, 5, and 10 respectively;

[0064] Figure 4 A schematic diagram of the accuracy of the method provided in the embodiment of the present invention when σ is -0.1, 0.1, and 0.2 respectively;

[0065] Figure 5 A schematic diagram showing a comparison of the accuracy of the method provided in the embodiment of the present invention and other methods;

[0066] Figure 6 A schematic diagram comparing the degree of differential privacy between the method provided in an embodiment of the present invention and other methods;

[0067] Figure 7 A block diagram of a privacy-protected personal privacy image data recognition system provided by an embodiment of the present invention;

[0068] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] like Figure 1 As shown, an embodiment of the present invention provides a privacy-protected personal privacy image data identification method, comprising:

[0071] S1. Obtaining personal privacy image data to be identified;

[0072] Collect personal privacy image data through the camera, such as taking pictures of books that individuals often read and uploading them to the company's intranet.

[0073] S2. Inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection;

[0074] The embodiment of the present invention uses privacy-protected distributed training for training the neural network model, such as Figure 2 As shown, five distributed training nodes are used, and each distributed training node i can be a computer. While each computer is training the neural network model, it will also transmit data to each other to influence each other. Weights are used to represent the degree of influence on each other. The personal privacy image data samples in the training set are divided into five parts, which are respectively trained by five distributed training nodes i. Each distributed training node i inputs several samples in each personal privacy image data sample at a time for training. The output of the neural network model at each moment is used as the original value of the training value of each distributed training node i at each moment. However, in order to protect privacy, the embodiment of the present invention adds noise to the original value of the training value of each distributed training node i at each moment, sends the original value with added noise to the neighboring node, receives the original value with added noise sent by the neighboring node, and uses the formula to update to obtain the training value at the next moment. The specific process is described later.

[0075] Optionally, the distributed training objective of the neural network model is:

[0076]

[0077] Where x is the training value output by distributed training node i, l i (x,ξ i) is the local loss function of the distributed training node i inputting a training sample, f i (x) is the loss function of all training samples input to distributed training node i, F(x) is the total loss function of all distributed training nodes, and denote the set of all real numbers and d-dimensional Euclidean space respectively, Represents expectations, A training set representing personal privacy image data;

[0078] Formula (1) can be written as the following minimization problem:

[0079]

[0080] Optionally, the distributed training process of the neural network model is to achieve the distributed training goal and solve the minimization problem (2), making the following assumptions:

[0081] Assumption 1: Communication diagram of each training node during the distributed training process of the neural network model It is connected and undirected, and the adjacency matrix between training nodes is It is double random. 1 of them n represents an n-dimensional column vector whose elements are all 1, and T represents transpose;

[0082] Assumption 2: The loss function of the training node during the distributed training of the neural network model satisfies the following conditions:

[0083] (i) For any training node i, f i (x) having a Lipschitz continuous gradient;

[0084] (ii) The loss function of each training node and its sum start from the lower bound,

[0085] (iii) For For any training node i sampled uniformly, there exists a random first-order oracle that returns f i The sampling gradient g of (x) i (x,ζ i ), there exists a constant σ greater than 0 g , so that each sampled gradient g i (x,ζ i )satisfy represents f i The gradient of (x);

[0086] Assumption 3: The quantizer Q(x) in the distributed training process of the neural network model is unbiased and has a bounded variance, which means that there is a constant Δ greater than 0 such that and Quantizer Q(x) with the following probability distribution:

[0087]

[0088] in represents the probability, is the floor rounding symbol;

[0089] Assumption 4: During the distributed training of the neural network model, for any training time K ≥ 1, 0 ≤ k ≤ K, the training step size Sample size and the privacy noise parameter σ k =(k+1) σ The following conditions are met:

[0090]

[0091]

[0092]

[0093]

[0094] in represents the Laplacian matrix of the adjacency matrix A of the training nodes, is a matrix The smallest positive eigenvalue of .

[0095] Optionally, the distributed training process of the neural network model specifically includes:

[0096] S21, initialization is performed;

[0097] Set the initial training node according to assumption 1 and set the initial training value of the training node Initialize the weight matrix between training nodes (a ij ) 1≤i,j≤n , by properly choosing the training step size Differential privacy parameter δ k , privacy noise parameter σ k and quantizer, satisfying assumptions 2-4;

[0098] S22. For training time k=0, 1, 2, ..., K, perform the following steps:

[0099] S221, training node i will make noise n i,kAdd to training value x i,k , and then passed to the neighbor training node, and then the quantizer is used to calculate the quantization information in the form of assumption (3) I d represents the identity matrix of d dimensions;

[0100] S222, training node i to train neighboring nodes Send z i,k ;

[0101] S223, training node i receives data from neighboring training nodes z j,k , and from Zhongtong took out without returning Data samples Returns the sampled gradient

[0102]

[0103] S224, training node i updates its training value by the following method

[0104]

[0105] During the distributed training of the neural network model, the random first-order oracle requires personal privacy image data samples to return the sampled gradient. Therefore, an attacker can infer sensitive information of the personal privacy image data samples during the training process from the changes in the sampled gradient. In order to provide privacy protection for personal privacy image data samples during the training process, the embodiment of the present invention defines a symmetric binary relationship called an adjacency relationship through the changes in the sampled gradient.

[0106] Optionally, the method further comprises:

[0107] A symmetric binary relationship called adjacency is defined by sampling the change of gradients: For any training node i, let For two sets of personal privacy image data samples, if for a given constant C>0 and any There is at most one pair of personal privacy image data samples in Make

[0108]

[0109] Then it is called and is the adjacent personal privacy image data sample in the distributed training of the neural network model, denoted as

[0110] Optionally, the method further comprises:

[0111] The concept of (ε,δ)-differential privacy is introduced (in order to reflect the level of privacy protection in the distributed training process of the neural network model). The (ε,δ)-differential privacy:

[0112] Given ε,δ>0 and algorithm If any private image data subset is satisfy Algorithm for Satisfies (ε,δ)-differential privacy.

[0113] Optionally, the method is private:

[0114] For any training time K ≥ 1, 0 ≤ k ≤ K in the distributed training process of the neural network model, let the training step length Number of personal privacy image sampling samples The privacy noise added during training is chosen as σ k =(k+1) σ , the differential privacy parameter is chosen as The other constants are selected as a1, a3>0, 0<a2<1, α, β, σ>0, After adding noise to the training values ​​of the original neural network model distributed training, for two adjacent personal privacy image data samples in the training process The neural network model satisfies - Differential privacy, where

[0115] Optionally, the method has convergence property:

[0116] If the neural network model satisfies assumptions 1-4, then during the training process, for any training node i and training time K≥1, we obtain:

[0117]

[0118] Wherein O() represents an infinitesimal of the same order, indicating that after a certain period of training, the loss function of each training node in the distributed training of the neural network model can be minimized, thereby achieving the distributed training goal, so that the trained model can successfully identify the input personal privacy image data to be identified.

[0119] Optionally, as the number of training times k increases, the training and test accuracy increases accordingly. The smaller Δ and σ are, the higher the accuracy of the neural network model after training. Δ determines the choice of quantizer, and σ determines the situation of adding noise to the model. The smaller Δ and σ are, the smaller the quantization error and noise variance are, and the faster the distributed training converges.

[0120] In order to further verify the correctness of the method, the embodiment of the present invention uses the "MNIST" data set to simulate the distributed training process of the neural network model, and divides the data set into two sub-data sets, which are used as the training set and the test set respectively. The training set is uniformly divided into 5 sub-data sets, consisting of 12,000 binary images. Each sub-data set can only be accessed by one training node to update its model parameters. After applying the privacy protection method of the embodiment of the present invention to the distributed training of the neural network model (the convolutional neural network CNN used in the implementation of the present invention), after a certain period of training, the training and test accuracy on the "MNIST" data set is as follows: Figure 3 and Figure 4 As shown in the figure, it can be seen that as the number of training times k increases, the training and test accuracy increases accordingly. More importantly, the smaller Δ and σ are, the higher the accuracy of the neural network model after training. This is because Δ determines the choice of quantizer and σ determines the situation of adding noise in the system. The smaller Δ and σ are, the smaller the quantization error and noise variance are, and the faster the distributed training of the neural network model converges.

[0121] Next, in the distributed training of the neural network model, the embodiment of the present invention fixes Δ and σ to Δ=1, σ=1. Then, the method of the embodiment of the present invention is compared with other methods. The accuracy comparison after comparison is as follows: Figure 5 As shown, in order to ensure fair comparison, the same step size as the method of the embodiment of the present invention is set in other methods. Figure 5 It can be seen that after using the method of the embodiment of the present invention, the neural network model achieves better accuracy than other methods. k and the differential privacy parameter δ k Next, the differential privacy budget Comparison of Figure 6 As shown, this means that after using the method of the embodiment of the present invention, the differential privacy budget of the neural network model is limited by a finite constant in infinite iterations, while the differential privacy budget in other methods tends to infinity in infinite iterations. Therefore, the privacy protection method of the embodiment of the present invention can not only meet the training requirements of distributed training convergence of the neural network model, but also provide a better differential privacy level.

[0122] S3. Use the trained neural network model to identify the personal privacy image data to be identified, and output the identification result.

[0123] After training, the neural network model of the embodiment of the present invention can be used to identify personal privacy image data to be identified and output the identification result.

[0124] like Figure 7 As shown, an embodiment of the present invention provides a privacy-protected personal privacy image data recognition system, comprising:

[0125] An acquisition module 710 is used to acquire the personal privacy image data to be identified;

[0126] An input module 720, configured to input the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection;

[0127] The recognition module 730 is used to use the trained neural network model to recognize the personal privacy image data to be recognized and output a recognition result.

[0128] An embodiment of the present invention provides a privacy-protected personal privacy image data recognition system, whose functional structure corresponds to a privacy-protected personal privacy image data recognition method provided by an embodiment of the present invention, and will not be repeated here.

[0129] Figure 8 It is a structural schematic diagram of an electronic device 800 provided in an embodiment of the present invention. The electronic device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 801 and one or more memories 802, wherein at least one instruction is stored in the memory 802, and the at least one instruction is loaded and executed by the processor 801 to implement the steps of the above-mentioned flotation foam image segmentation method based on multimodal data fusion.

[0130] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the above-mentioned flotation foam image segmentation method based on multimodal data fusion. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0131] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A privacy-protected personal privacy image data recognition method, characterized in that: include: S1. Obtaining personal privacy image data to be identified; S2. Inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection; S3, using the trained neural network model to identify the personal privacy image data to be identified, and outputting the identification result; The distributed training objectives of the neural network model are: Where x is the training value output by distributed training node i, l i (x,ξ i ) is the local loss function of the distributed training node i inputting a training sample, f i (x) is the loss function of all training samples input to distributed training node i, F(x) is the total loss function of all distributed training nodes, and denote the set of all real numbers and d-dimensional Euclidean space respectively, Represents expectations, A training set representing personal privacy image data; Formula (1) can be written as the following minimization problem: The distributed training process of the neural network model is to achieve the distributed training goal and solve the minimization problem (2), making the following assumptions: Assumption 1: Communication diagram of each training node during the distributed training process of the neural network model It is connected and undirected, and the adjacency matrix between training nodes is It is double random. 1 of them n represents an n-dimensional column vector whose elements are all 1, and T represents transpose; Assumption 2: The loss function of the training node during the distributed training of the neural network model satisfies the following conditions: (i) For any training node i, f i (x) having a Lipschitz continuous gradient; (ii) The loss function of each training node and its sum start from the lower bound, (iii) For For any uniformly sampled training node i, there exists a random first-order oracle that returns f i The sampling gradient g of (x) i (x,ζ i ), there exists a constant σ greater than 0 g , so that each sampled gradient g i (x,ζ i )satisfy represents f i The gradient of (x); Assumption 3: The quantizer Q(x) in the distributed training process of the neural network model is unbiased and has a bounded variance, which means that there is a constant Δ greater than 0 such that and Quantizer Q(x) with the following probability distribution: in represents the probability, is the floor rounding symbol; Assumption 4: During the distributed training of the neural network model, for any training time K ≥ 1, 0 ≤ k ≤ K, the training step size Sample size and the privacy noise parameter σ k =(k+1) σ The following conditions are met: in Represents the adjacency matrix of the training node The Laplace matrix of is a matrix The smallest positive eigenvalue of .

2. The method according to claim 1, characterized in that The distributed training process of the neural network model specifically includes: S21, initialization is performed; Set the initial training node according to assumption 1 and set the initial training value of the training node Initialize the weight matrix between training nodes (a ij ) 1≤i,j≤n , by properly choosing the training step size Differential privacy parameter δ k , privacy noise parameter σ k and quantizer, satisfying assumptions 2-4; S22. For training time k=0, 1, 2, ..., K, perform the following steps: S221, training node i will make noise n i,k Add to training value x i,k , and then passed to the neighbor training node, and then the quantizer is used to calculate the quantization information in the form of assumption (3) I d represents the identity matrix of d dimensions; S222, training node i to train neighboring nodes Send z i,k ; S223, training node i receives data from neighboring training nodes z j,k , and from Zhongtong took out without putting back Data samples Returns the sampled gradient S224, training node i updates its training value by the following method 3. The method according to claim 2, characterized in that The method further comprises: A symmetric binary relationship called adjacency is defined by sampling the change of gradients: For any training node i, let For two sets of personal privacy image data samples, if for a given constant C>0 and any There is at most one pair of personal privacy image data samples in Make Then it is called and is the adjacent personal privacy image data sample in the distributed training of the neural network model, denoted as 4. The method according to claim 3, characterized in that: The method further comprises: Introduce the concept of (ε,δ)-differential privacy, the (ε,δ)-differential privacy: Given ε,δ>0 and algorithm If any private image data subset is satisfy Algorithm for Satisfies (ε,δ)-differential privacy.

5. The method according to claim 4, characterized in that The method described is private: For any training time K ≥ 1, 0 ≤ k ≤ K in the distributed training process of the neural network model, let the training step length Number of personal privacy image sampling samples The privacy noise added during training is chosen as σ k =(k+1) σ , the differential privacy parameter is chosen as The other constants are selected as a1, a3>0, 0<a2<1, α, β, σ>0, After adding noise to the training values ​​of the original neural network model distributed training, for two adjacent personal privacy image data samples in the training process The neural network model satisfies - Differential privacy, where 6. The method according to claim 5, characterized in that The method described is convergent: If the neural network model satisfies assumptions 1-4, then during the training process, for any training node i and training time K≥1, we obtain: Wherein O() represents an infinitesimal of the same order, indicating that after a certain period of training, the loss function of each training node in the distributed training of the neural network model can be minimized, thereby achieving the distributed training goal, so that the trained model can successfully identify the input personal privacy image data to be identified.

7. The method according to claim 6, characterized in that As the number of training times k increases, the training and test accuracy increases accordingly. The smaller Δ and σ are, the higher the accuracy of the neural network model after training. Δ determines the choice of quantizer, and σ determines the situation of adding noise to the model. The smaller Δ and σ are, the smaller the quantization error and noise variance are, and the faster the distributed training converges.

8. A privacy-protected personal privacy image data recognition system, characterized in that: include: An acquisition module, used to acquire the personal privacy image data to be identified; An input module, used for inputting the personal privacy image data to be identified into a trained neural network model, wherein the training of the neural network model uses distributed training with privacy protection; A recognition module, used to use the trained neural network model to recognize the personal privacy image data to be recognized and output a recognition result; The distributed training objectives of the neural network model are: Where x is the training value output by distributed training node i, l i (x,ξ i ) is the local loss function of the distributed training node i inputting a training sample, f i (x) is the loss function of all training samples input to distributed training node i, F(x) is the total loss function of all distributed training nodes, and denote the set of all real numbers and d-dimensional Euclidean space respectively, Represents expectations, A training set representing personal privacy image data; Formula (1) can be written as the following minimization problem: The distributed training process of the neural network model is to achieve the distributed training goal and solve the minimization problem (2), making the following assumptions: Assumption 1: Communication diagram of each training node during the distributed training process of the neural network model It is connected and undirected, and the adjacency matrix between training nodes is It is double random. 1 of them n represents an n-dimensional column vector whose elements are all 1, and T represents transpose; Assumption 2: The loss function of the training node during the distributed training of the neural network model satisfies the following conditions: (i) For any training node i, f i (x) having a Lipschitz continuous gradient; (ii) The loss function of each training node and its sum start from the lower bound, (iii) For For any uniformly sampled training node i, there exists a random first-order oracle that returns f i The sampling gradient g of (x) i (x,ζ i ), there exists a constant σ greater than 0 g , so that each sampled gradient g i (x,ζ i )satisfy represents f i The gradient of (x); Assumption 3: The quantizer Q(x) in the distributed training process of the neural network model is unbiased and has a bounded variance, which means that there is a constant Δ greater than 0 such that and Quantizer Q(x) with the following probability distribution: in represents the probability, is the floor rounding symbol; Assumption 4: During the distributed training of the neural network model, for any training time K ≥ 1, 0 ≤ k ≤ K, the training step size Sample size and the privacy noise parameter σ k =(k+1) σ The following conditions are met: in represents the Laplacian matrix of the adjacency matrix A of the training nodes, is a matrix The smallest positive eigenvalue of .

Images