A privacy protection method for information center network-oriented group reinforcement learning

Abstract

The application discloses a privacy protection method for group reinforcement learning of an information center network and relates to the field of machine learning. The application performs privacy leakage analysis on information sent by a node to the outside, calculates a generated privacy data leakage loss in a numerical manner, and uses the leakage loss as a leakage degree measurement standard. A user can customize a privacy budget according to the safety degree of a model running environment and a user model safety requirement. When the customized privacy budget upper limit is exceeded, the node is caused to exit model updating of group learning, and the possibility of local privacy data leakage is prevented. The application prevents privacy leakage of a participating node and protects data safety of the node. The application has low deployment cost, good safety, strong robustness and reliable performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and in particular to a privacy protection method for group reinforcement learning of information-centric networks. BACKGROUND

[0002] The current Internet architecture is based on the TCP / IP protocol. However, the exponential growth of users and data has led to a significant increase in communication traffic, posing major challenges for effectively managing such large-scale data. Information-Centric Networking (ICN) moves away from the host-centric connection model of TCP / IP networks to a content-centric model. Compared with the TCP / IP model, ICN aims to provide a more efficient content distribution network architecture for users, with advantages such as improved cache utilization, reduced network communication redundancy, and reduced network congestion. Specifically, in an ICN network, a user's request for content is first searched in the cache of the local router, and then a request to a remote provider is considered. Therefore, the choice of cache strategy significantly affects the efficiency of the ICN network.

[0003] Traditional ICN cache replacement strategies include LRU (Least Recently Used), LFU (Least Frequently Used), and RAND (Random). These methods, while practical, only perform well in specific scenarios and cannot effectively adapt to dynamic environments. Learning-based methods, such as those using Deep Reinforcement Learning (DRL), can better adapt to changes in content demand caused by changes in users or scenarios, and have better prospects.

[0004] Collaborative learning allows participants in an ICN network to optimize local caches using global cache information, which can reduce the size of the local deep learning network and improve cache selection efficiency, albeit at the cost of communication overhead. Some research has begun to design collaborative caching mechanisms based on Federated Learning (FL). In federated learning, clients train local models using their source data and upload these models, while the server is only responsible for aggregating and distributing the global model formed after each iteration. However, FL faces some key issues, such as single-point failure and the need for a trusted third-party server, which compromise its security. In real-world network environments, attacks such as model inversion, member inference, and model inference are becoming increasingly common. The motivation of clients participating in training is often difficult to determine, and the trustworthiness of the central server is not always guaranteed. Malicious clients can manipulate model updates, thereby disrupting the aggregation of the global model. In addition, attackers can infer sensitive source data by analyzing model updates, and malicious servers can attempt to access the original data of clients. Research has shown that gradient information can leak users' private data.

[0005] As a new distributed framework, Swarm Learning (SL) provides a solution for the security and robustness of collaborative caching mechanism. SL is a decentralized machine learning solution that uses edge computing and blockchain technology for peer-to-peer collaboration. In SL, each participating node uses private data for local learning and parameter adjustment, and through a blockchain-based swarm network (SN), the model consensus and distribution are achieved to adapt to the dynamic joining and leaving of nodes. For this purpose, a new algorithm SRL (Swarm Reinforcement Learning) is proposed and applied to the ICN security collaborative caching strategy, which shows better performance than the general caching strategy and deep learning strategy in experiments. The swarm network in SRL based on blockchain technology can maintain the traceability of malicious attackers and provide non-repudiation, but it still has no defense capability against passive attacks such as eavesdropping. In particular, HBC (Honest but Curious) participants, these semi-honest nodes behave like honest nodes in the aggregation process of collaborative learning, faithfully executing protocol and algorithm steps. However, they secretly collect and analyze messages from other nodes, such as gradient updates received as leader nodes, using passive attacks to remain hidden. This allows them to collect information about the data privacy of other participating nodes, which may lead to privacy leakage. The solution to this is differential privacy technology. Differential privacy is widely used in distributed learning and can effectively solve security problems. In particular, when potential attackers collect model parameters from other nodes as leaders, central differential privacy can protect the confidentiality of content, but it faces the same problem as federated learning, which relies on a trusted aggregator to store all individuals' sensitive data to protect their privacy.

[0006] Therefore, the skilled person in the art is committed to developing a privacy protection method for swarm reinforcement learning in information-centric networks. Local differential privacy directly protects user data at the user end from being leaked to others. By running local differential privacy on each node, the model parameters of each node can be well protected from being discovered, thereby ensuring good model aggregation at the statistical level. In addition, consider the partial observation theory, where the attacker cannot control the complete information about the network other than the victim, but can only control a small number of observations that implement partial information. Under this assumption, the invention can analyze the privacy leakage of the information issued by the node to the outside world, and calculate the privacy data leakage loss in a numerical way as a measure of the degree of leakage. Users can customize the privacy budget according to the security level of the model running environment and the user's model security requirements. When the upper limit of the customized privacy budget is exceeded, the node is exited from the model update of the swarm learning to prevent the possibility of local privacy data leakage. SUMMARY

[0007] In view of the above defects of the prior art, the technical problem to be solved by the present application is the privacy protection problem of the group learning distributed framework in the information center network.

[0008] To achieve the above-mentioned purpose, the present application provides a privacy protection method for group reinforcement learning of information center network, comprising setting a privacy data leakage risk calculator locally in the user.

[0009] Further, the privacy loss generated by the user sharing parameters in each round is calculated by the partial observation theory.

[0010] Further, the privacy leakage risk of the user's local private data is numerized.

[0011] Further, the shared data is protected before the user participates in each round of aggregation of the group learning framework.

[0012] Further, by differential privacy technology, before the user shares parameters in each consensus phase, the model parameters to be shared are added with noise of a specified size according to the Gaussian mechanism.

[0013] Further, all users use the same noise size.

[0014] Further, the user customizes the security protection strength and the protection granularity, including the complete group learning process and the aggregation process of each round.

[0015] Further, the user has the authority to customize the threshold of the upper limit of the privacy budget.

[0016] Further, the threshold of the upper limit of the privacy budget can be determined by the user according to the security level of the learning task and the security situation of the running environment.

[0017] Further, before each round of aggregation, the user customizes the appropriate noise size according to the Gaussian mechanism.

[0018] When performing the group learning distributed framework in the ICN network, the user cannot determine the security of the current execution environment. The present application sets a local privacy data leakage risk calculator locally in each participating user. By applying the partial observation theory and the privacy calculation method, the privacy loss generated by sharing parameters in each round is calculated. The present application provides the numerical value of the privacy leakage risk of the user's local private data, so that the user has a more intuitive feeling of the security level of the execution environment.

[0019] The group learning distributed framework in the ICN network does not provide a secure privacy protection method for users, and is vulnerable to passive attacks and a series of behaviors that may harm user privacy data. The application applies privacy protection technology to the data shared by each participating user before each round of aggregation in the group learning framework to achieve the purpose of protection. Through differential privacy technology, before the user performs the consensus phase to share parameters each time, the model parameters to be shared are added with noise of a specified size according to the Gaussian mechanism. Through this method, all the model parameter data shared by the user is not the original data, but the data added with noise, and any attacker obtaining these data will not be able to know the real original model parameters. And when all people in the network use the same noise size, the expectation of the final aggregated model will remain convergent, consistent with the case without noise.

[0020] In the group learning distributed framework in the ICN network, the existing privacy protection method has poor environment adaptability and cannot adjust the protection ability according to user demand and environment security. The application provides user-defined security protection strength, and the protection granularity includes the complete group learning process and the aggregation process of each round. At the level of the complete group learning process, the user is provided with the authority to customize the threshold of the upper limit of the privacy budget, and the user can decide according to the confidentiality degree of the learning task and the security situation of the running environment; and before each round of aggregation, the user customizes the appropriate noise size according to the Gaussian mechanism, wherein when the environment security is poor, the protection ability of the privacy data is stronger when a larger noise is selected, and when the security is good, a smaller noise can be selected to achieve faster convergence and better model performance. The method of the application can provide user-defined privacy data protection strength, and the coverage is wider; the user will be able to determine the protection ability according to the demand, the model characteristics, the task characteristics, the environment security and other factors, to ensure the security while providing faster model convergence and better model effect as much as possible.

[0021] The application provides a privacy protection method for group reinforcement learning in an information center network, dynamically calculates privacy loss according to information transmission in the aggregation process, prevents privacy leakage of participating nodes, and protects data security of the nodes.

[0022] Technically, firstly, the application adopts a differential privacy encryption mechanism to add noise to the model parameter sharing process, protecting the model parameter privacy of the participating nodes; secondly, the application uses the partial observation theory to analyze the related information of the local privacy data of the victim node that may be collected from the attacker, so as to calculate the privacy loss that may be caused by the node when sharing the local model parameter; thirdly, the application provides a numerical privacy loss measurement standard and a security exit mechanism, after multiple rounds of node model parameter sharing, if the cumulative privacy loss exceeds the pre-set upper limit of the privacy budget, the current model parameter is reserved as the final result, and the node exits from the group network, thereby protecting the safety of the local privacy data on the premise of achieving the parameter sharing goal.

[0023] From the production implementation aspect, the application has great cost advantage, the cost of deployment is extremely low, the safety is good, and the robustness is relatively strong, and the performance is reliable. From the nature, the application provides the functions of the calculator and the accumulator, does not need additional hardware facilities, and is easy to deploy; the application takes the output model parameter and the added noise as the input, calculates the possible privacy loss caused by the current aggregation, wherein the single-step observation method is used to deal with the node that assumes the leadership for a single time, the multi-step observation is used to deal with the node that assumes the leadership for a long time, and when the node assumes the leadership, the node does not cause the privacy loss because the model parameter information is not sent out. The privacy loss calculated after each round of calculation is accumulated, and a user locally sets an upper limit threshold of the privacy budget. When the threshold is exceeded, the node exits the group learning framework, thereby providing safety protection for the user node; and as long as the number of remaining nodes is greater than or equal to the minimum number of participating nodes, the other nodes can still continue to aggregate, and the performance is reliable.

[0024] Compared with the prior art, the application has the following obvious substantial characteristics and significant advantages:

[0025] 1. The application dynamically calculates the privacy loss according to the information transmission in the aggregation process, prevents the privacy leakage of the participating nodes, and protects the data safety of the nodes.

[0026] 2. The application has great cost advantage, the cost of deployment is extremely low, the safety is good, and the robustness is relatively strong, and the performance is reliable.

[0027] The concept, specific structure and generated technical effects of the application will be further described below with reference to the drawings, so as to fully understand the purpose, features and effects of the application. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a privacy protection method for group reinforcement learning in an information center network of a preferred embodiment of the application;

[0029] Figure 2is a single-step observation and multi-step observation scenario of a preferred embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical content of the present application is more clear and easy to understand by introducing the preferred embodiments of the present application with reference to the accompanying drawings of the specification. The present application can be embodied in many different forms and the protection scope of the present application is not limited to the embodiments mentioned herein.

[0031] In the drawings, the same components of structure are denoted by the same reference numerals, and components of similar structure or function are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application is not limited to the size and thickness of each component. In order to make the drawing clearer, the thickness of the components is appropriately exaggerated in some places in the drawing.

[0032] As Figure 1 shown, the present application provides a privacy protection mechanism for information-centric network swarm reinforcement learning. It analyzes the privacy loss from two situations: single observation and multiple observations, which correspond to the attacker acting as a leader node for a single time step and a leader node for multiple time steps, respectively. First, introduce the general form of the model aggregation algorithm for information-centric network swarm reinforcement learning:

[0033]

[0034] The process of local learning, noise addition and inter-node consensus in the Swarm Reinforcement Learning (SRL) model is mathematically modeled and theoretically analyzed. The state of the node in the network at time step k is defined as the vector X(k) = [x1(k), x2(k), …, x n (k)] T The vector corresponds to the model parameter network of the local deep learning network. The final goal is to achieve model parameter aggregation and consensus through multiple iterations: Where x(0) represents the initial local parameters of each model. The learning matrix L(k) is used to represent the update result obtained after local learning and model parameter gradient backpropagation. The vector E i :x i (k) = E i x(k) is used to represent the selection of the node. The noise vector W(k) represents the noise following the Gaussian distribution. A is defined as the weight matrix. It is worth noting that in swarm learning, each element of the weight matrix is This indicates that all nodes are considered equal. Of course, specific weights can be assigned to the swarm network, but in any case, the weight matrix A must satisfy double randomness, i.e., the row sum is 1 and it is symmetric. In the subsequent theoretical analysis, taking the update of a parameter in the model network as an example, the vector representation can be obtained similarly.

[0035] Theoretically, the SRL consensus algorithm flow is described as follows: At time step k-1, each node locally stores the model parameters X(k-1)=[X1(k-1),……,X n (k-1)]. After local learning, the updated L(k-1) = [L 1(k-1) ,……,L n (k-1)]. Before participating in aggregation, the local parameters are subjected to noise addition: Z(k-1). The result of participating in consensus is X(k) = AZ(k-1). After completing one round of consensus, linear iteration continues, including processes such as local learning and noise addition.

[0036] The parameter consistency at time step k is derived, and the following results are obtained:

[0037]

[0038] First, we give the definition of double randomness:

[0039] Definition 1 (Double Randomness)

[0040] A matrix It is a doubly random matrix if and only if it is symmetric and the elements of each row satisfy a ij (k)≥0 and Where i∈{1,…,n}.

[0041] Regarding the weight matrix, the following two assumptions hold:

[0042] Assumption 1 (Weighting Rule)

[0043] There exists a scalar η such that 0 < η < 1, and for all i, j ∈ {1, ..., m}, we have:

[0044] For all k≥0, a ii (k)≥η

[0045] If a ij (k)>0, then for all k≥0, a ij (k)≥η

[0046] Assumption 2 (Connectivity)

[0047] Graph G = (V, E) is connected. There exists an integer B.ij ≥ 1, such that for each pair (j, i) ∈ E, agent j passes information to agent i within B ij steps.

[0048] Based on the assumptions and definitions, two lemmas are obtained, which are useful in the following derivations.

[0049] Lemma 1

[0050] Assumptions 1, 2 and Definition 1 hold. Then the following results hold:

[0051]

[0052] For all i, j ∈ {1,..., n} and k ≥ 0, a ij (k) > 0, where η is the lower bound in Assumption 1, n is the number of agents, B ij is the constant integer in Assumption 2.

[0053] Lemma 2

[0054] Assumptions 1 and Definition 1 hold. Then the following results hold:

[0055] For all i ∈ {1,..., n} and k ≥ 0, a ii (k) ≥ η k+1 ;

[0056] If a ij (k) > 0, then for all i, j ∈ {1,..., n} and k ≥ 0, a ij (k) ≥ η k+1 ;

[0057] where η is the lower bound in Assumption 1.

[0058] These two lemmas provide two lower bounds for the matrix elements. Therefore, when deriving the lower bound of the equivalent noise, two different results will be obtained, and the maximum of these two results will be taken finally.

[0059] To use the local privacy parameter of the differential privacy protection model, first introduce the Gaussian mechanism:

[0060] Definition 2 (Gaussian mechanism)

[0061] Let W be an m-dimensional multivariate Gaussian random variable N(0, Σ), where the covariance matrix Σ is positive definite. The Gaussian mechanism is defined as M(x) = x + W, where x ∈ R m is a certain fixed vector. Let f M (x) denote the probability density function (PDF) of the random variable M(x). The adjacent relationship is defined as |x - x0|k ≤ Δ, where Δ ∈ [0, 1] is arbitrary. Then the mechanism M is - differential privacy, where λ min is the smallest eigenvalue of Σ, is a constant independent of Σ and Δ (but dependent on δ).

[0062] In addition, two assumptions are made for theoretical derivation: first, there is a difference in the training data set, which leads to observable differences in subsequent local learning of model parameters. Second, L(k) is considered as a random variable following a Gaussian distribution, and it is distinguished from the Gaussian variable parameter of the noise. That is:

[0063]

[0064] where k is the time step, and σ1 and σ2 are the standard deviations of matrices L(k) and W(k), respectively.

[0065] Next, the observation of the potential attacker on the victim is defined. This can be divided into single observation and multiple observations. Single observation means that the potential attacker receives the parameter z(k) transmitted by the victim in the aggregation process at a certain time step k. Since in group learning, a leader is elected as the central server for parameter aggregation every round, and only the leader has the right to receive the parameter information sent by other nodes, single observation means that the potential attacker is elected as the leader at this time step. Multiple observations mean that the potential attacker is continuously elected as the leader in the K+1 rounds of time steps [0:K], so that they can obtain more information to infer the local state of the victim. As Figure 2 shown, the two cases will be discussed respectively.

[0066] (I) Single-step observation

[0067] First consider the scenario where the leader node of the network only aggregates the model parameters of other nodes at time step k. Given the complexity of local learning, reference is made to the case of partial observation, assuming that before the first aggregation of the group network, the local network model parameters of each node are different, i.e. for node i, the difference in local private data leads to a difference in a certain model parameter: x i (0) ≠ x i (0)', where the sensitivity is |x i (0) - x i (0)' | ≤ Δ i . From this, the following proposition is obtained:

[0068] Proposition 1

[0069] Consider the noisy distributed consensus algorithm under the following assumptions:

[0070] Assumption 1, 2 and Definition 1 hold.

[0071] x i (0)-x′ i (0)|≤Δ i , and for all l≠i, x l (0)=x′ l (0).

[0072] Noise w(k) = [w1(k);... ; w n (k) T are independent and identically distributed Gaussian zero-mean random variables, i.e.

[0073] When the attacker is the leader, for some K≥1, the attacker can observe z j (K)(if not the leader, the observation is the aggregated result x j (K+1)).

[0074] Then, for any δ, the algorithm is (∈, δ)-differentially private, where the value of ∈ is derived from equations 3, 6 and 7 under different conditions.

[0075] In summary, this proposition provides a method to calculate the potential privacy loss. Once the parameters are determined in the actual execution process, the exact upper bound of privacy loss can be estimated. When the cumulative privacy loss exceeds the preset threshold, it can be exited in time to prevent data leakage. Here, three cases from general to specific are discussed, and the corresponding upper limit of the privacy loss threshold in each case is derived: (1) the attacker is not responsible for aggregation at the current time step; (2) the attacker is responsible for aggregation at the current time step; (3) the attacker is responsible for aggregation and uses the "average" method consistent with Swarm Learning for aggregation.

[0076] Proof: The content will be discussed and proved from three angles.

[0077] Case 1

[0078] The attacker may not become the leader at a certain round, so first consider the case where the attacker makes inferences as an ordinary participant. When the attacker is not the leader, they can only obtain the consensus model parameters distributed to each node after aggregation. That is, the observation of the attacker will be:

[0079]

[0080] Due to the difference in the node training data set, the initial model parameter matrix $X(0)$ exists differently:

[0081] X(0) = [x_1(0), x_2(0),..., x_i(0),..., x_n(0)]

[0082] X'(0) = [x_1(0), x_2(0),..., x_i'(0),..., x_n(0)]

[0083] where |x i (0) - x i (0)' | < Δ i At time step K, for the observed X j (K+1):

[0084]

[0085] According to the Gaussian mechanism, the first term in the equation can be considered as a constant, while the second term is a noise term. Based on the assumption, both L(K) and W(k) follow Gaussian distribution. Therefore, based on the derivation in Partial Observation theory:

[0086]

[0087] According to Lemma 2: a jj (k) ≥ η k+1 Therefore:

[0088]

[0089] where η = min{η1, η2}

[0090] According to Lemma 1, we have:

[0091]

[0092] where

[0093] Equation 1 and Equation 2 both provide lower bounds for and are based on the same assumption, with the only difference being whether the node network requires connectivity. For Swarm Learning, this assumption is obviously true. Therefore, while using the conclusions of both equations, and taking the maximum lower bound:

[0094] In addition, according to the Gaussian mechanism, we have:

[0095]

[0096] where Therefore, we have:

[0097]

[0098] Case 2

[0099] When the attacking node actually corresponds to the leader node at the corresponding time period, it can obtain the noise model parameters, Z, of each node before the consensus aggregation i (K) = X i (K) + L i (K) + W i (K). According to the above derivation process, we can similarly obtain:

[0100]

[0101] According to the strictness of the assumption premise, we have:

[0102]

[0103] The sensitivity is given by the following formula:

[0104]

[0105] According to the above given bound, the result claimed in Proposition 1 is obtained. Next, further derivation can be made to obtain the corresponding privacy loss as:

[0106]

[0107] where, According to expressions 3 and 4 have lower bounds. Therefore, the privacy loss should have an upper bound.

[0108] Case 3

[0109] In addition, the weight matrix A is analyzed. No matter which node performs the aggregation, the weight of each node remains the same. Therefore, the weight matrix A first satisfies that the sum of each row is 1, and each row is the same. Considering the collaborative nature of the ICN network, these nodes should be regarded as equal nodes, i.e. with equal weights, which is consistent with the "mean" aggregation method of Swarm Learning. In this case, each node is aggregated with the same weight. Therefore, the weight matrix A will be a symmetric matrix, each element of which is satisfies the double randomness. In fact, in this case, the weight matrix A is deterministic:

[0110]

[0111] At this time, A k = A. This greatly simplifies the derivation process. Apply this equation to the above two cases, when the attacking node is not the leader node at time step K, we have:

[0112]

[0113] Then, the actual noise after aggregation is:

[0114]

[0115] Here, the initial sensitivity is:

[0116]

[0117] Therefore, the final privacy loss should be:

[0118]

[0119] When the attacking node is the leader at time step K, we can also get:

[0120]

[0121] And the observation is:

[0122]

[0123] Therefore, the attacker will obtain the data with noise, and the sensitivity is as follows:

[0124]

[0125] The final privacy loss should be:

[0126]

[0127] (II) Multi-step observation

[0128] In this section, we discuss the privacy loss that may be caused by the attacker's observation from time step 0 to K. The conditions and assumptions remain the same as in the single observation case. Then, there is a similar proposition:

[0129] Proposition 2:

[0130] Consider the noisy distribution consensus algorithm in Algorithm 1, which satisfies the following assumptions:

[0131] Assumptions I, II, and definition I hold.

[0132] |x i (0)-x′ i (0)|≤Δ i

[0133] Noise w(k) = [w0(k), …, wn (k] T are independent and identically distributed zero-mean Gaussian random variables, i.e.

[0134] The attacker observes z j (0),…,z j (K), where K > 1.

[0135] Then, for any δ, the algorithm is (ε, δ)-differentially private, where the value of ε follows inequalities 8 and 9 under different conditions.

[0136] The difference is that the observation target changes from a single data point to a sequence. Due to the complexity of the derivation, two specific cases will be discussed: 1) the attacker always acts as a leader node during time steps 0 to K; 2) the attacker not only always acts as a leader node, but also uses the “mean” aggregation method of Swarm Learning in the aggregation process. Below, the derivation and proof of each case are provided respectively.

[0137] Proof:

[0138] Case 1

[0139] When the attacker acts as a leader of the group network to aggregate model parameters of other nodes in multiple consecutive time steps, the noisy model parameters Z i (K) from the victim nodes participating in the aggregation can be directly observed:

[0140]

[0141] where, represents the noisy parameters in K+1 time steps.

[0142] Thus, there is

[0143]

[0144] where the three matrices are:

[0145]

[0146]

[0147] Therefore, under the multi-time step condition, the sensitivity can be represented as:

[0148]

[0149] Thus, it can be defined that:

[0150]

[0151] And let:

[0152] Σ = Σ2 + Σ3

[0153] Finally, the calculation result of privacy loss is obtained by the following formula:

[0154]

[0155] Wherein, Σ is the total sum of noise terms generated by M2 and M3.

[0156] Case 2

[0157] When using the "mean" method for aggregation in Swarm Learning, each element of matrix A is equal to the same weight At this time, according to the result in case 1, the sensitivity can be calculated as follows:

[0158]

[0159] It can be found that in the group network, the sensitivity increases exponentially with the increase of time step, following the square process. Therefore, the privacy loss will be:

[0160]

[0161] This conclusion is similar to the case of single observation analysis.

[0162] The above describes the preferred embodiments of the present application in detail. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A privacy protection method for information-centric network-oriented group reinforcement learning, characterized in that, Including in the user local set privacy data leakage risk calculator; Through the partial observation theory, the privacy loss generated by the user in each round of sharing parameters is calculated; Numerical user local private data privacy leakage risk; Before each round of aggregation in the user participation group learning framework, the shared data is protected by privacy; Through the differential privacy technology, before the user performs the consensus phase each time, the model parameters to be shared are added with noise of a specified size according to the Gaussian mechanism; The user customizes the security protection strength and, and the protection granularity includes the whole group learning process and the aggregation process of each round; The user has the authority to customize the threshold of the upper limit of the privacy budget; Before each round of aggregation, the user customizes the appropriate noise size according to the Gaussian mechanism; According to the information transmission in the aggregation process, the privacy loss is dynamically calculated; the differential privacy encryption mechanism is used to add noise to the model parameter sharing process; the partial observation theory is used to analyze the related information of the local privacy data of the victim user that may be collected from the attacker's perspective, and the privacy loss caused by the user when sharing the local model parameters is calculated; After multiple rounds of user model parameter sharing, the cumulative privacy loss exceeds the pre-set upper limit of the privacy budget, the current model parameters are retained as the final result, and the user exits from the group network.

2. The privacy protection method for information-centric network oriented group reinforcement learning according to claim 1, wherein, All users use the same noise size. 3.The privacy protection method for information-centric network oriented group reinforcement learning according to claim 1, wherein, The threshold of the upper limit of the privacy budget is determined by the user according to the confidentiality degree of the learning task and the security situation of the running environment.

Images