A training method for a neural network model

Abstract

The embodiment of the present application discloses a training method for a neural network model, which belongs to the field of computer networks. The method includes: obtaining original user data, and determining the node data and a first adjacency matrix in the original user data, the first adjacency matrix being used to characterize the relationship between the node data; performing differential privacy processing on the adjacency matrix based on a Laplace noise mechanism to obtain a second adjacency matrix; performing differential privacy processing on the first node feature matrix of the node data based on a Gaussian noise mechanism to obtain a second node feature matrix; constructing a graph convolutional neural network prediction model based on the second adjacency matrix and the second node feature matrix, and training the graph convolutional neural network prediction model. The perturbation data output by this scheme exhibits the characteristics of irregularity and high robustness against attacks, which can improve the efficiency of privacy protection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a training method for a neural network model. Background Art

[0002] With the in-depth development of graph neural network theory in social network relationships and recommendation systems, people have gradually begun to pay attention to the privacy protection of node information. With the support of big data, it is very easy to obtain user privacy information through the node information of user data.

[0003] In related technologies, the original user data is mainly disturbed by the differential privacy algorithm, and noise is added to confuse data thieves, so that data thieves cannot tell whether the stolen data contains private information.

[0004] However, in the existing technology, since data thieves can infer sensitive information of data through noise filtering, big data and machine learning, as well as statistical measurement and analysis methods, it is easy to obtain user privacy after the collector obtains a large amount of raw user data. Summary of the Invention

[0005] The present invention provides a method for training a neural network model, which solves the problem that the existing technology cannot effectively protect the privacy of original user data and prevent the theft of user privacy after data leakage. The technical solution is as follows:

[0006] Obtaining original user data, and determining node data and a first adjacency matrix in the original user data, where the first adjacency matrix is ​​used to represent the relationship between the node data;

[0007] Performing differential privacy processing on the adjacency matrix based on a Laplace noise mechanism to obtain a second adjacency matrix, where the second adjacency matrix is ​​the adjacency matrix after adding Laplace noise perturbation;

[0008] Performing differential privacy processing on the first node feature matrix of the node data based on a Gaussian noise mechanism to obtain a second node feature matrix, where the second node feature matrix is ​​the node feature matrix after adding Gaussian noise perturbation;

[0009] A graph convolutional neural network prediction model is constructed based on the second adjacency matrix and the second node feature matrix, and model training is performed on the graph convolutional neural network prediction model.

[0010] Specifically, the method further includes: calculating query results of at least two adjacent data sets through a query function, and determining the sensitivity of the data sets based on the maximum norm distance of the query results;

[0011] Among them, the sensitivity is used to characterize the variation range of the query results of two adjacent data sets, and the two adjacent data sets only differ by one element. The data sets include the first adjacency matrix and the first node feature matrix. The first adjacency matrix is ​​an N×N matrix, and the first node feature matrix is ​​an N×M matrix. N is the number of node data in the original user data, and M is the maximum number of features in the node.

[0012] The performing differential privacy processing on the adjacency matrix based on the Laplace noise mechanism to obtain a second adjacency matrix includes:

[0013] Determining a first privacy allocation ratio under Laplace noise perturbation according to the data set, the differential privacy noise addition algorithm, and the privacy loss function, wherein the first privacy allocation ratio is used to determine the differential privacy weight of the Laplace noise mechanism;

[0014] The first privacy loss under the Laplace noise mechanism is less than the second privacy budget, the second privacy budget is the product of the first privacy allocation ratio and the first privacy budget, and the first privacy budget provides a degree of differential privacy loss;

[0015] The second adjacency matrix is ​​determined according to the sum of the first adjacency matrix and Laplace noise disturbance.

[0016] Specifically, performing differential privacy processing on the first node feature matrix of the node data based on the Gaussian noise mechanism to obtain the second node feature matrix includes:

[0017] Determining a second privacy allocation ratio and a third privacy budget based on the first privacy allocation ratio, where the third privacy budget is the product of the second privacy allocation ratio and the first privacy budget; the second privacy allocation ratio is used to determine the differential privacy weight of the Gaussian noise mechanism; and the sum of the first privacy allocation ratio and the second privacy allocation ratio is 1;

[0018] Determining a parameter estimate of a Gaussian distribution standard deviation based on the third privacy budget ratio and the second privacy loss under the Gaussian noise mechanism, wherein the parameter estimate is used to determine a variance distribution condition of the Gaussian noise mechanism;

[0019] The second node feature matrix is ​​determined according to the sum of the first node feature matrix and Gaussian noise disturbance.

[0020] Specifically, constructing a graph convolutional neural network prediction model based on the second adjacency matrix and the second node feature matrix, and performing model training on the graph convolutional neural network prediction model includes:

[0021] Constructing the graph convolutional neural network prediction model based on the second adjacency matrix, the second node feature matrix and the data set;

[0022] The graph convolutional neural network prediction model is trained using the original user data as input samples and the node classification information matrix as supervision. The node classification information matrix is ​​used to aggregate the information of neighboring nodes and its own information in a semi-supervised node classification task to predict node labels.

[0023] Specifically, the original user data is used as an input sample and the node classification information matrix is ​​used as supervision to train the graph convolutional neural network prediction model, including:

[0024] Determining a corresponding degree matrix based on the number of sample nodes of the original user data and the second adjacency matrix of the data set;

[0025] Based on the degree matrix, the second adjacency matrix, the second node feature matrix and the number of graph neural network layers, the second node feature matrix is ​​iteratively trained, and the node classification information matrix is ​​obtained by normalization function processing.

[0026] Specifically, when the query function is used for querying under the Laplace noise perturbation, the scale parameter consumed each time is the ratio of sensitivity to the second privacy budget; when the query function is used for querying under the Gaussian noise perturbation, the scale parameter consumed each time is the ratio of sensitivity to the third privacy budget.

[0027] Specifically, the method further includes:

[0028] The original user data is differentially privacy processed based on the graph convolutional neural network prediction model to obtain user disturbance data.

[0029] The beneficial effects brought about by the above technical solution include at least: protecting node relationship information by perturbing the first adjacency matrix in the original user data using a Laplace noise mechanism, and protecting node feature information by perturbing the first node feature matrix in the original user data using a Gaussian noise mechanism. Moreover, by iteratively training the established graph convolutional neural network prediction model, it is ensured that the perturbed data processed by the graph convolutional neural network prediction model for differential privacy is not easily parsed, exhibiting irregularities and high robustness against attacks. Even when the data is uploaded to the blockchain for sharing, collectors cannot accurately extract user privacy data through query functions and machine learning, thereby improving the efficiency of privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1is a schematic diagram of differential privacy processing provided by an exemplary embodiment of the present application;

[0031] Figure 2 This is a flow chart of a training method for a neural network model provided in an embodiment of the present application;

[0032] Figure 3a This is a verification comparison diagram of the iterative training process using the Gaussian mechanism to protect node feature information provided by the embodiment of the present application;

[0033] Figure 3b This is a comparison chart of the verification of the iterative training process using the Laplace mechanism to protect node relationship information;

[0034] Figure 4 This is a verification comparison chart of the iterative training process using a dual privacy protection mechanism. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0036] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0037] In related technologies, original user data is processed with a single differential privacy algorithm to obtain perturbed data. However, once the collector steals the perturbed data, they can use various means to analyze the perturbed data and thus steal user privacy. Figure 1 As shown in the figure, assume that the original user data consists of data A, data B, data C, data D, data E, and data F. After privacy processing and leakage, data A and data B are obtained by collector A; data C is obtained by collector B; data D and data E are obtained by collector C; and data F is obtained by collector D. Due to the ease of uploading and sharing of user data in blockchain and big data, data between any blocks in the blockchain is shared. Therefore, both Path 2 and Path 3 can obtain the content of data A through E. Due to the low security of a single differential privacy algorithm, label prediction can be performed on user data through behavioral learning and top-k queries, thereby stealing user privacy. For collector D, who has not uploaded data to the blockchain, he can only use Path 1 to perform statistics and prediction on data F.

[0038] In the embodiment of the present application, a dual differential privacy processing approach is adopted in the privacy processing phase. The Laplace noise mechanism is used to perturb the first adjacency matrix in the original user data to protect node relationship information. At the same time, a Gaussian noise mechanism is used to perturb the first node feature matrix in the original user data to protect node feature information. Furthermore, by iteratively training the established graph convolutional neural network prediction model, the perturbed data subjected to differential privacy processing by the graph convolutional neural network prediction model is ensured to be difficult to parse. Even if the data is uploaded to the blockchain for sharing, collectors cannot accurately extract user privacy data through query functions and machine learning.

[0039] Figure 2 This is a flow chart of a training method for a neural network model provided in an embodiment of the present application, which is used for a computer device or terminal to process raw user data, specifically comprising the following steps:

[0040] Step 201 : obtaining original user data, and determining node data and a first adjacency matrix in the original user data. The first adjacency matrix is ​​used to represent the relationship between the node data.

[0041] Original user data can be data generated by users through devices such as terminals, computer devices, servers, and workstations. Original user data contains node data and the first adjacency matrix. Node data is used to form an undirected graph. Through undirected graph analysis and label prediction, the information of multiple nodes can be aggregated to predict node labels, and then the user's private information can be extracted based on the node labels.

[0042] Step 202 : Calculate query results of at least two adjacent data sets through a query function, and determine the sensitivity of the data set according to the maximum norm distance of the query results.

[0043] Before performing differential privacy processing on raw user data, it's necessary to determine the datasets that make up the node data. These datasets consist of a first node feature matrix and a first adjacency matrix, with two adjacent datasets differing by only one element. The first adjacency matrix is ​​an N×N matrix, where N represents the number of node data in the raw user data; the first node feature matrix is ​​an N×M matrix, where M represents the maximum number of features in a node.

[0044] During differential privacy processing, it is necessary to add noise perturbations to the original user data. However, before adding noise perturbations, it is necessary to first determine the sensitivity of each data set. This can be represented by the maximum norm distance between the corresponding query functions of two adjacent data sets, which is specifically defined as:

[0045]

[0046] Where Δfp represents sensitivity, f is the query function, which represents the mapping of a data set to the d-dimensional real space, that is, D→R d , d represents the number of query results. D and D′ represent two adjacent datasets. The norm value distance between adjacent datasets through the query function is represented by Lp, as follows:

[0047]

[0048] The query function may be a query method based on any query algorithm, such as a Top-k query method, etc., which is not limited in this embodiment.

[0049] Step 203: Determine a first privacy allocation ratio under Laplace noise perturbation based on the data set, the differential privacy noise addition algorithm, and the privacy loss function.

[0050] For any dataset D, the noise perturbation mechanism adds uncertain random noise to the query results. The purpose of adding noise perturbation is to make it difficult for attackers to distinguish whether the query results contain sensitive information, or to distinguish the differences between query results of adjacent datasets. This means that attackers cannot determine whether a user's data is included in the queried statistics, and therefore cannot obtain sensitive information through differential inference by performing more than two queries. The noise distribution mechanism is expressed as follows:

[0051]

[0052] Among them, r represents the random noise added, and the output function This application introduces Laplace noise to ensure that for good data sets D and D′ that differ by at least one data or element, the data collector cannot distinguish the output and The difference between them makes it impossible to conduct subsequent data statistics.

[0053] When introducing Laplace noise perturbation, we must also ensure the privacy loss caused by the perturbation. For a random algorithm M, the privacy loss can be expressed as follows:

[0054]

[0055] Among them, the algorithm M Under the condition of (∈, 0)-differential privacy, Pr represents the value of the joint probability. and Theoretically, the ratio of the probability densities of these two possible outcomes represented by statistical distributions is less than e ε, through the definition of privacy loss, we know that Loss(o) is less than ∈. ε represents the first privacy budget. -Differential privacy, for any two datasets D and The value space of the output result is a set Moreover, for any noise algorithm, the following formula holds:

[0056]

[0057] The first privacy budget provides a degree of differential privacy, and the degree of privacy protection increases as the first privacy budget decreases, but the availability of training results decreases as the first privacy budget decreases. δ is a relaxation term added, indicating the distance that can be tolerated that violates differential privacy. In particular, when δ = 0, it indicates that no distance that violates differential privacy can be tolerated, which is called (∈, 0)-differential privacy or strict differential privacy.

[0058] The use of double differential privacy requires the pre-determination of the allocation of the first privacy budget, that is, the first privacy budget with a limited total amount should be reasonably allocated to the Laplace noise mechanism and the Gaussian noise mechanism. The output function of the Laplace noise mechanism can be expressed as:

[0059]

[0060] in, represents the output after adding Laplace noise perturbation. λ1 represents the first privacy allocation ratio under Laplace noise perturbation, which is used to determine the differential privacy weight of the Laplace noise mechanism. Since the specific value of λ1 is not known when designing the Laplace mechanism, it must be determined first before determining its weight in the double differential perturbation.

[0061] From the Laplace probability density function, we can see that the ratio of the probability density function obtained by querying under the condition of (λ1∈,0)-differential privacy without relaxation term does not exceed The first privacy loss under the Laplace mechanism is smaller than the second privacy budget. The second privacy budget is the product of the first privacy allocation ratio and the first privacy budget, that is, Loss L (o)≤λ1∈. The proof process is as follows:

[0062]

[0063] Through the above proof process, we can know that under the premise of known data set, differential privacy noise algorithm and privacy loss function, according to Loss L The corresponding relationship between (o) and λ1∈ can determine the value of λ1.

[0064] Step 204: Determine a second adjacency matrix according to the sum of the first adjacency matrix and the Laplace noise disturbance.

[0065] After determining λ1, the second adjacency matrix can be determined by the first adjacency matrix and the Laplace noise perturbation. The second adjacency matrix is ​​the adjacency matrix after adding the Laplace noise perturbation, which is expressed as follows:

[0066]

[0067] Among them, A represents the first adjacency matrix, I N represents the N-order identity matrix, represents the second adjacency matrix after adding Laplace noise perturbation. Since the second adjacency matrix is ​​the adjacency matrix after adding noise perturbation, the node relationship information can be protected.

[0068] Step 205: Determine a second privacy allocation ratio and a third privacy budget based on the first privacy allocation ratio.

[0069] Since the first privacy allocation ratio λ1 and the second privacy allocation ratio λ2 satisfy the formula: λ1+λ2=1, after determining the first privacy allocation ratio, the second privacy allocation ratio can be determined, and the third privacy budget λ2∈ is also determined accordingly.

[0070] Step 206 : Determine a parameter estimate of the standard deviation of the Gaussian distribution based on the third privacy budget and the second privacy loss under the Gaussian noise mechanism.

[0071] The premise of using the Gaussian mechanism is to verify the second privacy loss Loss after introducing Gaussian noise G (o) is less than the third privacy budget λ2∈, i.e., Loss G (o)≤λ2∈. The proof process is as follows:

[0072]

[0073] in represents the output function using Gaussian noise, r i =1,…,k, represents the query results output under the i-th Gaussian mechanism, and they are all random variables that obey the Gaussian distribution and satisfy From the definition of sensitivity, we can see that the sensitivity under the Gaussian mechanism meets the condition Therefore, the above formula can be further converted to:

[0074]

[0075] In particular, when and only when the number of queries is 1, that is, k = 1, in order to make the second privacy loss under the Gaussian mechanism be controlled within the third privacy budget, that is, Loss G(o)≤λ2∈, the value of r needs to satisfy the following formula:

[0076]

[0077] Derivation under the Gaussian mechanism confirms that this mechanism cannot definitively satisfy the (∈, 0)-differential privacy condition, so it is necessary to introduce (∈, δ)-differential privacy with a relaxation term. In the Gaussian mechanism, δ represents the tolerable distance that does not satisfy differential privacy. Based on the above formula and the definition of the relaxation term, the following formula can be derived:

[0078]

[0079] According to Chernoff bound theory, a Gaussian random variable with zero mean is The upper bound of the bilateral long tail is:

[0080]

[0081] When the value of α is When , we can get:

[0082]

[0083] Let the value of the relaxation term δ be equal to the upper bound of the bilateral long tail, and we can get:

[0084]

[0085] Then the condition for the Gaussian distribution variance that satisfies (∈,δ)-differential privacy can be obtained as follows:

[0086]

[0087] After sorting, the parameter estimate of the standard deviation of the Gaussian distribution can be obtained as follows:

[0088]

[0089] Step 207: Determine a second node characteristic matrix according to the sum of the first node characteristic matrix and the Gaussian noise disturbance.

[0090] By determining the parameter estimate of the standard deviation of the Gaussian distribution, the Gaussian noise mechanism can be designed. The second node feature matrix is ​​obtained by combining the first node feature matrix and the added Gaussian noise perturbation. The second node feature matrix is ​​expressed as follows:

[0091]

[0092] Where X0 represents the first node feature matrix, and X1 represents the second node feature matrix after adding Gaussian noise perturbation.

[0093] Step 208: construct a graph convolutional neural network prediction model based on the second adjacency matrix, the second node feature matrix, and the data set.

[0094] After determining the second node feature matrix, a graph convolutional neural network prediction model can be constructed based on the second adjacency matrix, the second node feature matrix, and the dataset. This approach considers both the structural information and node information of the graph model. A Laplace mechanism is used to differentially protect structural information, while a Gaussian mechanism is used to protect the privacy of node feature information, achieving comprehensive data protection.

[0095] In step 209, the graph convolutional neural network prediction model is trained using the original user data as input samples and the node classification information matrix as supervision. The node classification information matrix is ​​used to aggregate the information of neighboring nodes and the node itself in the semi-supervised node classification task to predict the node label.

[0096] For a given privacy budget ∈, the first adjacency matrix and the first node feature are both used as input datasets When iteratively training the graph convolutional neural network prediction model, it is necessary to design the number of graph neural network layers L of the model. max and a nonlinear activation function σ.

[0097] After determining the second privacy budget of the Laplace noise mechanism, the scale parameter consumed each time when querying with the query function under Laplace noise perturbation is calculated based on the sensitivity of the dataset, and is expressed as the ratio of sensitivity to the second privacy budget. Similarly, the scale parameter consumed each time when querying with the query function under Gaussian noise perturbation is calculated based on the sensitivity of the dataset, and is expressed as the ratio of sensitivity to the third privacy budget. They are expressed as:

[0098] and The second adjacency matrix is ​​obtained according to the Laplace noise perturbation Finally, it is necessary to further determine the corresponding degree matrix based on the number of sample nodes of the original user data and the second adjacency matrix of the data set. The degree matrix is ​​expressed as follows:

[0099]

[0100] Where i is a positive integer ranging from 1 to N, where N represents the number of sample nodes in the input original user data. The purpose of the certainty matrix is ​​to subsequently obtain the node classification information matrix.

[0101] When iteratively training the graph convolutional neural network prediction model based on the degree matrix, the second adjacency matrix, and the second node feature matrix, L max –1 iteration training. The iteration formula is as follows:

[0102]

[0103] X l+1 ←σ(Z l )

[0104] Among them, Z l Represents the i-th node classification information matrix, and the value of l ranges from 1 to L max -1, indicating L max –1 iterative training process, X l represents the second node feature matrix obtained by Gaussian noise perturbation (after l noise perturbations), Θ represents the trainable parameter matrix, and σ is the activation function. max After 1 iteration of training, the node classification information matrix output by the output layer is expressed as:

[0105]

[0106] Among them, softmax is the classification function, which is used for the final output of the classification task. Noisy adjacency matrix with self-loop It is still an aggregation operator. Adding self-loops is equivalent to involving the features of the current node and neighboring nodes in the convolution operation at the same time, which can enhance the aggregation effect of the adjacency matrix on the node's own information. In the usual semi-supervised node classification task, the information of neighboring nodes and the node's own information can be aggregated to predict the node's label.

[0107] After the graph convolutional neural network prediction model is trained, the original user data is differentially privacy processed based on the graph convolutional neural network prediction model to obtain the output user perturbation data.

[0108] In the embodiment of the present application, the first adjacency matrix in the original user data is perturbed using a Laplace noise mechanism to protect node relationship information. A Gaussian noise mechanism is also used to perturb the first node feature matrix in the original user data to protect node feature information. Furthermore, by iteratively training the established graph convolutional neural network prediction model, the perturbed data processed by the graph convolutional neural network prediction model for differential privacy is ensured to be difficult to parse, exhibiting irregularities and high robustness against attacks. Even when the data is uploaded to the blockchain for sharing, collectors cannot accurately extract user privacy data through query functions and machine learning, thereby improving the efficiency of privacy protection.

[0109] Figure 3a This is a verification comparison chart of the iterative training process using the Gaussian mechanism to protect node feature information. Figure 3bThis is a verification comparison chart of the iterative training process using the Laplace mechanism to protect node relationship information.

[0110] The training was performed through 200 iterations on the validation set and the dataset, and the accuracy was recorded. Because the elements in the first adjacency matrix of the original user data only take values ​​within the extremely small sample space {0,1}, they are vulnerable to differential attacks based on statistical inference, thus requiring a stronger privacy protection mechanism. The Laplace mechanism is more likely to take extreme values, which can better protect the security of the relationship information between nodes. In comparison, the Gaussian mechanism has more concentrated values ​​and a lower probability of taking extreme values, resulting in higher data availability and meeting the privacy protection requirements of node characteristics.

[0111] from Figure 3a It can be seen that when only the Gaussian mechanism is used to protect node feature information, the accuracy shows a rapid improvement in the first 50 training rounds. There is a rapid drop in the accuracy in the subsequent training rounds, but it quickly returns to normal levels in the 75th training round. Whether it is the verification set or the test set, the verification accuracy is stable at more than 80%, which makes it relatively easy to steal privacy.

[0112] from Figure 3b It can be seen that when only the Laplace mechanism is used to protect node relationship information, the model training process is relatively smooth, and there is no significant difference from the training process without the privacy protection mechanism. The accuracy of the training set and validation set quickly increases to around 100% and 80% and then remains stable, which cannot effectively prevent privacy theft.

[0113] Figure 4 This is a verification comparison chart of the iterative training process using a dual privacy protection mechanism.

[0114] The graph convolutional neural network model used in this experiment has two convolutional layers. The first convolutional layer has 16 neurons and uses ReLU as the nonlinear activation function. The second convolutional layer has 7 neurons and does not use a nonlinear activation function. Instead, it uses the Softmax value calculated from the convolution operation as the basis for classification. The total number of training cycles was set to 200.

[0115] In terms of training stability, the accuracy curve fluctuated significantly throughout the training process and was unable to maintain stability. The accuracy of both the test and validation sets rapidly dropped to 20% by the 60th training epoch, and continued to fluctuate and decline by the 160th epoch. In terms of training convergence speed, the training process under the double differential privacy mechanism converged more slowly. Even after a long period of stability between epochs 125 and 150, the accuracy dropped below 40% around epoch 160. The accuracy curve for the training set had a larger standard deviation than that for the validation set. This is primarily because the two different noise perturbations made the loss function less smooth, impacting the stability of the gradient descent training process. Consequently, more misleading information was introduced during training, making it more difficult for the gradient to reach a lower loss function value. The resulting curve did not steadily improve, but instead experienced multiple sharp rises and falls.

[0116] comprehensive Figure 3a 、 Figure 3b as well as Figure 4 It can be seen that the perturbed data output by the graph convolutional neural network training model using the double differential privacy mechanism exhibits unstable accuracy and an irregular curve. However, the accuracy curve using only the Gaussian and Laplace mechanisms still shows a steady improvement, making it easier for attackers to infer or restore the private information of the original user data from the gradient operator. Even if a thief obtains part of the perturbed data through a decentralized blockchain, they cannot guarantee that they can extract accurate private information. Therefore, the perturbed data with Laplace and Gaussian noise has higher privacy protection efficiency and stronger anti-robustness.

[0117] The above describes the preferred embodiments of the present invention; it should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can make many possible changes and modifications without departing from the technical solution of the present invention, or modify them into equivalent embodiments with equivalent changes, which does not affect the essential content of the present invention; therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solution of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A training method for a neural network model, characterized in that: The method comprises: Obtaining original user data, and determining node data and a first adjacency matrix in the original user data, where the first adjacency matrix is ​​used to represent the relationship between the node data; Calculating query results of at least two adjacent data sets using a query function, and determining the sensitivity of the data sets based on the maximum norm distance of the query results; wherein the sensitivity is used to characterize the variation range of the query results of the two adjacent data sets, and the two adjacent data sets differ by only one element, and the data sets include the first adjacency matrix and the first node feature matrix, the first adjacency matrix is ​​an N×N order matrix, and the first node feature matrix is ​​an N×M order matrix, where N is the number of node data in the original user data, and M is the maximum number of features in the node; Performing differential privacy processing on the adjacency matrix based on a Laplace noise mechanism to obtain a second adjacency matrix, where the second adjacency matrix is ​​the adjacency matrix after adding Laplace noise perturbation; determining a first privacy allocation ratio under the Laplace noise perturbation based on the data set, the differential privacy noise addition algorithm, and the privacy loss function; the first privacy allocation ratio is used to determine the differential privacy weight of the Laplace noise mechanism; wherein the first privacy loss under the Laplace noise mechanism is less than a second privacy budget, the second privacy budget is the product of the first privacy allocation ratio and the first privacy budget, and the first privacy budget provides a degree of differential privacy loss; Determining the second adjacency matrix according to the sum of the first adjacency matrix and Laplace noise disturbance; Performing differential privacy processing on the first node feature matrix of the node data based on a Gaussian noise mechanism to obtain a second node feature matrix, where the second node feature matrix is ​​the node feature matrix after adding Gaussian noise perturbation; Constructing a graph convolutional neural network prediction model based on the second adjacency matrix, the second node feature matrix, and the data set; Determining a corresponding degree matrix based on the number of sample nodes of the original user data and the second adjacency matrix of the data set; Based on the degree matrix, the second adjacency matrix, the second node feature matrix and the number of graph neural network layers, the second node feature matrix is ​​iteratively trained and processed by a normalization function to obtain a node classification information matrix; the node classification information matrix is ​​used to aggregate the information of neighbor nodes and its own information in the semi-supervised node classification task to predict the node label.

2. The method according to claim 1, characterized in that The performing differential privacy processing on the first node feature matrix of the node data based on the Gaussian noise mechanism to obtain a second node feature matrix includes: Determining a second privacy allocation ratio and a third privacy budget based on the first privacy allocation ratio, where the third privacy budget is the product of the second privacy allocation ratio and the first privacy budget; the second privacy allocation ratio is used to determine the differential privacy weight of the Gaussian noise mechanism; and the sum of the first privacy allocation ratio and the second privacy allocation ratio is 1; Determine a parameter estimate of a Gaussian distribution standard deviation based on the third privacy budget and the second privacy loss under the Gaussian noise mechanism, wherein the parameter estimate is used to determine a variance distribution condition of the Gaussian noise mechanism; The second node feature matrix is ​​determined according to the sum of the first node feature matrix and Gaussian noise disturbance.

3. The method according to claim 1 or 2, characterized in that When the query function is used for querying under the Laplace noise perturbation, the scale parameter consumed each time is the ratio of sensitivity to the second privacy budget; when the query function is used for querying under the Gaussian noise perturbation, the scale parameter consumed each time is the ratio of sensitivity to the third privacy budget.

4. The method according to claim 3, characterized in that After constructing the graph convolutional neural network prediction model, the method further includes: The original user data is differentially privacy processed based on the graph convolutional neural network prediction model to obtain user disturbance data.

Images