Group reinforcement learning privacy protection method oriented to information center network

Abstract

The invention discloses a privacy protection method for group reinforcement learning of an information center network, and relates to the field of machine learning. According to the method, privacy leakage analysis is carried out on information sent to the outside by a node, and the generated privacy data leakage loss is calculated in a numeralization mode and serves as a measurement standard of the leakage degree; the user can customize the privacy budget according to the security degree of the model operation environment and the security demand of the user model; and when the user-defined privacy budget upper limit is exceeded, updating the model for the node to quit the group learning to prevent the possibility of local privacy data leakage. The privacy leakage of participating nodes is prevented, and the data security of the nodes is protected; the method is extremely low in deployment cost, good in safety, high in robustness and reliable in performance.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to a privacy protection method for group reinforcement learning in an information-centric network. Background Art

[0002] The current Internet architecture is based on the TCP / IP protocol. However, the exponential growth of users and data has led to a significant increase in communication traffic, posing a major challenge to the effective management of such large-scale data. Information-centric networking (ICN) breaks away from the host-centric connection model of the TCP / IP network and moves to a content-centric model. Compared with the TCP / IP model, ICN aims to provide users with a more efficient content distribution network architecture with advantages such as improved cache utilization, reduced network communication redundancy, and reduced network congestion. Specifically, in an ICN network, a user's request for content is first searched in the cache of the local router before considering requests to remote providers. Therefore, the choice of caching strategy significantly affects the efficiency of the ICN network.

[0003] Traditional ICN cache replacement strategies include LRU (least recently used), LFU (least frequently used), and RAND (random). Although these methods are practical, they only perform well in specific scenarios and cannot effectively adapt to dynamic environments. Learning-based methods, such as those using deep reinforcement learning (DRL), can better adapt to changes in content demand caused by user or scenario changes and have better prospects.

[0004] Collaborative learning allows participants in an ICN network to optimize local caches using global cache information. Although this comes at the cost of communication overhead, it can reduce the size of local deep learning networks and improve cache selection efficiency. Some studies have begun to design collaborative caching mechanisms based on federated learning (FL). In federated learning, clients train local models with their source data and upload these models, while the server is only responsible for aggregating and distributing the global model formed after each iteration. However, FL faces some key issues, such as single point of failure and the need for trusted third-party servers, which compromises its security. In real network environments, attacks such as model inversion, membership inference, and model inference are becoming more and more common. The motivations of the clients participating in training are often difficult to determine, and the trustworthiness of the central server is not always guaranteed. Malicious clients may manipulate model updates to disrupt the aggregation of the global model. In addition, attackers can infer sensitive source data by analyzing model updates, and malicious servers may try to access the client's original data. Studies have shown that gradient information may leak users' private data.

[0005] Swarm Learning (SL), as an emerging distributed framework, provides a solution for the security and robustness of collaborative caching mechanisms. SL is a decentralized machine learning solution that uses edge computing and blockchain technology for peer-to-peer collaboration. In SL, each participating node uses private data for local learning and parameter adjustment, and achieves consensus and distribution of the model through a blockchain-based swarm network (SN) to adapt to the dynamic joining and leaving of nodes. To this end, a new algorithm SRL (Swarm Reinforcement Learning) is proposed and applied to the ICN secure collaborative caching strategy, which shows better performance than general caching strategies and deep learning strategies in experiments. The swarm network in SRL based on blockchain technology can maintain the traceability of malicious attackers and provide non-repudiation, but it still has no defense against passive attacks such as eavesdropping. In particular, HBC (Honest but Curious) participants, these semi-honest nodes behave like honest nodes in the aggregation process of collaborative learning, faithfully executing the protocol and algorithm steps. However, they secretly collect and analyze messages from other nodes, such as gradient updates received when acting as a leader node, and use passive attacks to remain hidden. This enables them to collect information about the data privacy of other participating nodes, which may lead to privacy leakage. The solution to this is differential privacy technology. Differential privacy is widely used in distributed learning and can effectively solve security issues. Especially when potential attackers collect model parameters from other nodes as leaders, central differential privacy can protect the confidentiality of content, but it faces the same problem as federated learning, which is to rely on trusted aggregators to store sensitive data of all individuals to protect their privacy.

[0006] Therefore, technicians in this field are committed to developing a privacy protection method for group reinforcement learning in information-centric networks. Local differential privacy protects user data directly on the user side to prevent it from being leaked to other parties. By running local differential privacy on each node, the model parameters of each node can be well protected from being discovered, thereby ensuring good model aggregation at the statistical level. In addition, consider the partial observation theory, in which the attacker cannot control the complete information about the network outside the victim, but can only control a small part to achieve the observation of partial information. Under this assumption, the present invention can perform privacy leakage analysis on the information sent by the node to the outside, and calculate the privacy data leakage loss generated in a numerical way as a measure of the degree of leakage. Users can customize the privacy budget according to the security level of the model operating environment and the user model security requirements. When the upper limit of the customized privacy budget is exceeded, the node will be withdrawn from the group learning model update to prevent the possibility of local privacy data leakage. Summary of the invention

[0007] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is the privacy protection problem of the distributed framework of group learning in the information center network.

[0008] To achieve the above-mentioned object, the present invention provides a privacy protection method for group reinforcement learning in an information-centric network, including setting a privacy data leakage risk calculator locally in the user.

[0009] Furthermore, through partial observation theory, the privacy loss caused by users sharing parameters in each round is calculated.

[0010] Furthermore, the privacy leakage risk of users' local private data is digitized.

[0011] Furthermore, before users participate in each round of aggregation in the group learning framework, the privacy of the shared data is protected.

[0012] Furthermore, through differential privacy technology, before the user shares parameters in the consensus phase each time, a specified amount of noise is added to the model parameters to be shared according to the Gaussian mechanism.

[0013] Furthermore, all users use the same noise size.

[0014] Furthermore, users customize the security protection strength and,protection granularity including the complete group learning process and,each round of aggregation process.

[0015] Furthermore, users have the authority to customize the threshold of the upper limit of the privacy budget.

[0016] Furthermore, the threshold of the upper limit of the privacy budget can be determined by the user according to the confidentiality of the learning task and the security of the operating environment.

[0017] Furthermore, before each round of aggregation, the user customizes a suitable noise size according to the Gaussian mechanism.

[0018] When executing a group learning distributed framework in an ICN network, users cannot determine the security of the current execution environment. The present invention sets a local privacy data leakage risk calculator locally for each participating user. By applying the partial observation theory and the privacy calculation method, the privacy loss generated when each round of shared parameters is calculated. The present invention provides a numeralization of the privacy leakage risk for the user's local private data, so that the user has a more intuitive feeling of the security of the execution environment.

[0019] The group learning distributed framework in the ICN network does not provide users with a safe privacy protection method, and is vulnerable to a series of behaviors such as passive attacks that may endanger user privacy data. The present invention applies privacy protection technology to the data shared by each participating user before each round of aggregation in the group learning framework to achieve the purpose of protection. Through differential privacy technology, before the user shares parameters in the consensus phase each time, a specified size of noise is added to the model parameters to be shared according to the Gaussian mechanism. Through this method, the present invention ensures that all model parameter data shared by users are not original data, but noisy data, and any attacker who obtains this data will not be able to know the true original model parameters. And when everyone in the network uses the same noise size, the expectations of the final aggregated model will remain converged, consistent with when no noise is added.

[0020] In the distributed framework of group learning in the ICN network, the existing privacy protection methods have poor environmental adaptability and cannot adjust the protection ability according to user needs and environmental security. The present invention provides user-defined security protection strength and, its protection granularity includes the complete group learning process and each round of aggregation process. At the level of the complete group learning process, the user is provided with the authority to customize the threshold of the upper limit of the privacy budget, and the user can decide for himself according to the confidentiality of the learning task and the security of the operating environment; and before each round of aggregation, the user customizes the appropriate noise size according to the Gaussian mechanism, wherein when the environmental security is poor, a larger noise is used to protect the privacy data more strongly, and when the security is good, a smaller noise is used to converge faster to obtain better efficiency and model performance. This method of the present invention can provide users with customized privacy data protection strength with a wider coverage; users will be able to comprehensively determine the size of the protection capability based on their own needs, model characteristics, task characteristics, environmental security and other factors, while ensuring security and providing faster model convergence and better model effects as much as possible.

[0021] The present invention provides a privacy protection method for group reinforcement learning in an information-centric network, which dynamically calculates the privacy loss according to the information transmission in the aggregation process, prevents the privacy leakage of participating nodes, and protects the data security of the nodes.

[0022] From a technical point of view, firstly, the present invention adopts a differential privacy encryption mechanism to add noise to the model parameter sharing process, thereby protecting the privacy of the model parameters of the participating nodes; secondly, the present invention uses partial observation theory to analyze the relevant information of the local privacy data of the victim node that may be collected from the attacker's perspective, so as to calculate the privacy loss that may be caused when the node shares the local model parameters; thirdly, the present invention provides a numerical privacy loss measurement standard and a safe exit mechanism. After multiple rounds of node model parameter sharing, if the cumulative privacy loss exceeds the pre-set privacy budget upper limit, the current model parameters are retained as the final result, and exit from the group network, thereby protecting the security of local privacy data while achieving the parameter sharing goal.

[0023] From the perspective of production implementation, the present invention has huge cost advantages, its deployment cost is extremely low, it has good security, strong robustness and reliable performance. In essence, the present invention provides the functions of a calculator and an accumulator, does not require additional hardware facilities, and is easy to deploy; it uses the output model parameters and the added noise as input, calculates and outputs the possible privacy loss caused by this round of aggregation, wherein a single-step observation method is used to deal with nodes that serve as leaders once, and a multi-step observation method is used to deal with nodes that serve as leaders for a long time. When this node serves as a leader, no privacy loss is caused because no model parameter information is sent out. After each round of calculation, the calculated privacy loss is accumulated, and the user locally sets a privacy budget upper limit threshold. When the threshold is exceeded, the node exits the group learning framework, thereby providing security protection for the user node; and as long as the number of remaining nodes is greater than or equal to the minimum number of participating nodes, other nodes can continue to aggregate, and the performance is reliable.

[0024] Compared with the prior art, the present invention has the following obvious substantial features and significant advantages:

[0025] 1. The present invention dynamically calculates the privacy loss according to the information transmission in the aggregation process, prevents the privacy leakage of participating nodes, and protects the data security of nodes.

[0026] 2. The present invention has a huge cost advantage, with extremely low deployment cost, good security, strong robustness and reliable performance.

[0027] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of a privacy protection method for group reinforcement learning in an information-centric network according to a preferred embodiment of the present invention;

[0029] Figure 2It is a single-step observation and multi-step observation scene diagram of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0031] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0032] like Figure 1 As shown, the present invention provides a privacy protection mechanism for group reinforcement learning in information-centric networks. It analyzes privacy loss from two situations: single observation and multiple observations, corresponding to the attacker acting as a leader node in a single time step and acting as a leader node in multiple time steps. First, the general form of the model aggregation algorithm for group reinforcement learning in information-centric networks is introduced:

[0033]

[0034] The process of local learning, noise addition and consensus among nodes in the Swarm Reinforcement Learning (SRL) model is mathematically modeled and theoretically analyzed. The state of a node in the network at time step k is defined as a vector X(k) = [x1(k), x2(k), …, x n (k)] T , which corresponds to the model parameter network of the local deep learning network. The ultimate goal is to achieve aggregation and consensus of model parameters through multiple iterations: Where x(0) represents the initial local parameters of each model. The learning matrix L(k) is used to represent the update results obtained after local learning and back propagation of model parameter gradients. The vector E is selected i :x i (k) = E i x(k) is used to represent the node selection. The noise vector W(k) represents the noise following the Gaussian distribution. A is defined as the weight matrix. It is worth noting that in group learning, each element of the weight matrix is Indicates that all nodes are considered equal. Of course, specific weights can also be specified for the group network, but in any case, the weight matrix A must satisfy double randomness, that is, the row sum is 1 and it is symmetric. In the subsequent theoretical analysis, taking the update of a parameter in the model network as an example, the vector form of representation can be obtained similarly.

[0035] Theoretically, the algorithm flow of SRL consensus is described as follows: At time step k-1, each node locally stores the model parameters X(k-1) = [X1(k-1), ..., X n After local learning, we get the update L(k-1)=[L 1(k-1) ,……,L n Before participating in aggregation, the local parameters will be added with noise: Z(k-1). The result of participating in consensus is X(k)=AZ(k-1). After completing a round of consensus, linear iteration continues, including local learning, noise addition and other processes.

[0036] The parameter consistency at time step k is derived and the following results are obtained:

[0037]

[0038] First, the definition of double randomness is given:

[0039] Definition 1 (Double Randomness)

[0040] A matrix is a doubly random matrix if and only if it is symmetric and the elements of each row satisfy a ij (k)≥0 and Where i∈{1,…,n}.

[0041] Regarding the weight matrix, the following two assumptions hold:

[0042] Assumption 1 (Weight Rule)

[0043] There exists a scalar η that satisfies 0<η<1, and for all i,j∈{1,…,m}, we have:

[0044] For all k ≥ 0, a ii (k)≥η

[0045] If a ij (k)>0, then for all k≥0, a ij (k)≥η

[0046] Assumption 2 (Connectivity)

[0047] The graph G = (V, E) is connected. There exists an integer Bij ≥1, so that for every pair (j,i)∈E, agent j is in B ij The information is passed to agent i within the step.

[0048] Based on the assumptions and definitions, we can derive two lemmas that will be useful in subsequent derivations.

[0049] Lemma 1

[0050] Assumptions 1, 2 and Definition 1 hold. Then we have the following results:

[0051]

[0052] For all i,j∈{1,…,n}, and satisfy a ij (k)>0, where η is the lower bound in Assumption 1, n is the number of agents, and B ij is the constant integer in Assumption 2.

[0053] Lemma 2

[0054] If Assumption 1 and Definition 1 hold, then we have the following results:

[0055] For all i∈{1,…,n} and k≥0, a ii (k)≥η k+1 ;

[0056] If a ij (k)>0, then for all i,j∈{1,…,n}$ and $k≥0, a ij (k)≥η k+1 ;

[0057] Here, η is the lower bound of Assumption 1.

[0058] These two lemmas provide two lower bounds for the matrix elements. Therefore, when deriving the lower bound of the equivalent noise, two different results will be obtained, and the maximum of the two results will be taken.

[0059] In order to use the local privacy parameters of the differential privacy protection model, the Gaussian mechanism is first introduced:

[0060] Definition 2 (Gaussian mechanism)

[0061] Let W be an m-dimensional multivariate Gaussian random variable N(0,Σ) with a covariance matrix Σ that is positive definite. The Gaussian mechanism is defined as M(x) = x + W, where x∈R m is a fixed vector. Let f M (x) represents the probability density function (PDF) of the random variable M(x). Assume that the neighbor relationship is defined as |x-x0|k ≤Δ, where Δ∈[0,1] is arbitrary. Then the mechanism M is -Differential privacy, where λ min is the smallest eigenvalue of Σ, is a constant that is independent of Σ and Δ (but depends on δ).

[0062] In addition, two assumptions are made for the theoretical derivation: first, there are differences in the training data set, which leads to observable differences in the local learning of the subsequent model parameters. Second, $L(k)$ is regarded as a random variable following a Gaussian distribution and is distinguished from the Gaussian variable parameters of the noise. That is:

[0063]

[0064] where k is the time step, σ1 and σ2 are the standard deviations of the matrices L(k) and W(k), respectively.

[0065] Next, define the potential attacker's observation of the victim. This can be divided into single observation and multiple observations. A single observation means that the potential attacker receives the parameter z(k) transmitted by the victim during the aggregation process at a specific time step k. Since in group learning, a leader is elected as the central server for parameter aggregation in each round, and only the leader has the right to receive parameter information sent by other nodes, a single observation means that the potential attacker is elected as the leader at this time step. Multiple observations mean that in the K+1 rounds of time step [0:K], the potential attacker is continuously elected as the leader, allowing them to obtain more information to infer the local state of the victim. For example Figure 2 As shown, these two cases will be discussed separately.

[0066] (I) Single-step observation

[0067] First, consider the scenario where the leader node of the network only aggregates the model parameters of other nodes at time step k. Given the complexity of local learning, we refer to some observations and assume that before the first aggregation of the group network, the local network model parameters of each node are different, that is, for node i, the difference in local private data leads to a difference in a certain model parameter: i (0)≠x i (0)', where the sensitivity is |x i (0)-x i (0)'|≤Δ i . Hence, the following proposition follows:

[0068] Proposition 1

[0069] Consider a distributed consensus algorithm with noise, under the following assumptions:

[0070] Assumptions 1, 2 and Definition 1 hold.

[0071] |x i (0)-x′ i (0)|≤Δ i , and for all l≠i, x l (0) = x′ l (0).

[0072] Noise w(k) = [w1(k); ...; w n (k) T are independent and identically distributed Gaussian zero-mean random variables, that is,

[0073] When the attacker is the leader, for some K ≥ 1, the attacker can observe z j (K)(If it is not the leader, the observed result is the aggregated result x j (K+1)).

[0074] Then, for any δ, the algorithm is (∈,δ)-differentially private, where the value of ∈ is derived from Equations 3, 6, and 7 under different conditions.

[0075] In summary, this proposition provides a method to calculate the potential privacy loss. Once the parameters are determined in the actual execution process, the accurate upper limit of the privacy loss can be estimated. When the cumulative privacy loss exceeds the preset threshold, it can be exited in time to prevent data leakage. Three cases from general to specific are discussed here, and the corresponding upper limit of the privacy loss threshold is derived for each case: (1) the attacker is not responsible for aggregation at the current time step; (2) the attacker is responsible for aggregation at the current time step; (3) the attacker is responsible for aggregation and uses the "average" method consistent with swarm learning for aggregation.

[0076] Proof: The content will be discussed and proved from three perspectives.

[0077] Case 1

[0078] The attacker may not become the leader in a certain round, so first consider the case where the attacker makes inferences as an ordinary participant. When the attacker is not the leader, they can only obtain the consensus model parameters that are aggregated and distributed to each node. In other words, the attacker's observations will be:

[0079]

[0080] Due to the differences in node training data sets, the initial model parameter matrix $X(0)$ is different:

[0081] X(0)=[x_1(0),x_2(0),...,x_i(0),...,x_n(0)]

[0082] X'(0)=[x_1(0),x_2(0),...,x_i'(0),...,x_n(0)]

[0083] where |x i (0)-x i (0)′|≤Δ i At time step K, for the observed X j (K+1):

[0084]

[0085] According to the Gaussian mechanism, the first term in the equation can be regarded as a constant, while the second term is a noise term. Based on the assumption, both L(K) and W(k) obey Gaussian distribution. Therefore, based on the derivation in the Partial Observation theory:

[0086]

[0087] According to Lemma 2: a jj (k)≥η k+1 ,therefore:

[0088]

[0089] where η=min{η1,η2}

[0090] According to Lemma 1, we have:

[0091]

[0092] in

[0093] Both equation 1 and equation 2 are The lower bound is provided, and both are based on the same assumption, the only difference is whether the node network requires connectivity. For Swarm Learning, this assumption is obviously valid. Therefore, the conclusions of these two equations are used at the same time, and the maximum lower bound is taken:

[0094] In addition, according to the Gaussian mechanism, we have:

[0095]

[0096] in Therefore, we have:

[0097]

[0098] Case 2

[0099] When the attacking node actually corresponds to the leader node in the corresponding time period, it can obtain the noise model parameters of each node before consensus aggregation, Z i (K) = X i (K)+L i (K)+W i (K). According to the above derivation process, we can similarly obtain:

[0100]

[0101] According to the strictness of the assumptions, we have:

[0102]

[0103] The sensitivity is given by:

[0104]

[0105] According to the given limits above, we get the result stated in Proposition 1. Next, by further deduction, we can get the corresponding privacy loss as:

[0106]

[0107] in, According to expressions 3 and 4, there is a lower bound. Therefore, the privacy loss should have an upper bound.

[0108] Case 3

[0109] In addition, the weight matrix A is analyzed. Regardless of which node performs the aggregation, the weight of each node remains the same. Therefore, the weight matrix A first satisfies that the sum of each row is 1, and each row is the same. Considering the collaborative nature of the ICN network, these nodes should be regarded as peer nodes, that is, with equal weights, which is consistent with the "mean" aggregation method of Swarm Learning. In this case, each node is aggregated with the same weight. Therefore, the weight matrix A will be a symmetric matrix with each element being Satisfies double randomness. In fact, in this case, the weight matrix A is deterministic:

[0110]

[0111] At this time, A k =A. This greatly simplifies the derivation process. Applying this equation to the above two cases, when the attacking node is not the leader node at time step K, we have:

[0112]

[0113] Then, the actual noise after aggregation is obtained as:

[0114]

[0115] Here, the initial sensitivity is:

[0116]

[0117] Therefore, the final privacy loss should be:

[0118]

[0119] When the attacking node is the leader node at time step K, we can also get:

[0120]

[0121] The observation results are:

[0122]

[0123] As a result, the attacker will obtain noisy data with the following sensitivities:

[0124]

[0125] The resulting privacy loss should be:

[0126]

[0127] (II) Multi-step Observation

[0128] In this section, we discuss the privacy loss that may be caused by an attacker during observations from time steps 0 to K. The conditions and assumptions remain the same as for the case of a single observation. Then, there is a similar proposition:

[0129] Proposition 2:

[0130] Consider the noisy distributed consensus algorithm in Algorithm 1, satisfying the following assumptions:

[0131] Assumptions I, II and Definition I hold.

[0132] |x i (0)-x′ i (0)|≤Δ i

[0133] Noise w(k)=[w0(k),…,wn (k)] T are independent and identically distributed zero-mean Gaussian random variables, that is,

[0134] The attacker observes z j (0),…,z j (K), where K>1.

[0135] Then, for any δ, the algorithm is (ε,δ)-differentially private, where the value of ∈ obeys inequalities 8 and 9 under different conditions.

[0136] The difference is that the observation target changes from a single data point to a sequence. Due to the complexity of the derivation, two specific cases will be discussed: 1) the attacker always acts as the leader node during time steps 0 to K; 2) the attacker not only always acts as the leader node, but also uses the "mean" aggregation method of Swarm Learning during the aggregation process. Below, the derivation and proof of each case are provided separately.

[0137] prove:

[0138] Case 1

[0139] When the attacker, as the leader of the swarm network, aggregates the model parameters of other nodes in multiple consecutive time steps, the noisy model parameters Z from the victim nodes participating in the aggregation i (K) It can be directly observed that:

[0140]

[0141] in, Represents the noisy parameters in K+1 time steps.

[0142] So, there is

[0143]

[0144] Among them, the three matrices are:

[0145]

[0146]

[0147] Therefore, under multiple time steps, the sensitivity can be expressed as:

[0148]

[0149] So we can define:

[0150]

[0151] And let:

[0152] Σ=Σ2+Σ3

[0153] Finally, the calculation result of privacy loss is obtained by the following formula:

[0154]

[0155] Where Σ is the sum of the noise terms generated by M2 and M3.

[0156] Case 2

[0157] When using the "mean" method for aggregation in Swarm Learning, each element of the matrix A is equal to the same weight At this point, based on the results in Case 1, the sensitivity can be calculated as follows:

[0158]

[0159] It can be found that in the group network, as the time step increases, the sensitivity increases exponentially, following a quadratic process. Therefore, the privacy loss will be:

[0160]

[0161] This conclusion is similar to that of the single observation analysis.

[0162] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A privacy protection method for group reinforcement learning in information-centric networks, characterized in that: Including setting up a privacy data leakage risk calculator locally on the user.

2. The privacy protection method for group reinforcement learning in information-centric networks according to claim 1, characterized in that: Through partial observation theory, the privacy loss caused by users sharing parameters in each round is calculated.

3. The privacy protection method for group reinforcement learning in information-centric networks according to claim 1, characterized in that: Digitize the privacy leakage risk of users' local private data.

4. The privacy protection method for group reinforcement learning in information-centric networks according to claim 1, characterized in that: Before users participate in each round of aggregation in the group learning framework, the shared data is protected for privacy.

5. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 1, characterized in that: Through differential privacy technology, before users share parameters in the consensus phase each time, noise of a specified size is added to the model parameters to be shared according to the Gaussian mechanism.

6. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 5, characterized in that: All users use the same noise level.

7. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 1, characterized in that: User-defined security protection strength and,protection granularity include the complete group learning process and,each round of aggregation process.

8. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 1, characterized in that: Users have the ability to customize the threshold of the upper limit of the privacy budget.

9. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 8, characterized in that: The threshold of the upper limit of the privacy budget is determined by the user according to the confidentiality of the learning task and the security of the operating environment.

10. The privacy protection method for group reinforcement learning in information-centric networks as claimed in claim 1, characterized in that: Before each round of aggregation, the user customizes the appropriate noise size according to the Gaussian mechanism.

Images