A social network anonymization method and device based on multi-agent reinforcement learning

Abstract

This invention relates to a method and apparatus for social network anonymization based on multi-agent reinforcement learning. The method includes: S1: Constructing a reinforcement learning environment and initial state; acquiring the social network graph data to be processed, modeling the anonymization task as a Markov decision process, and establishing the state representation and reward feedback mechanism of the environment; S2: Executing dual-agent collaborative decision-making; constructing a structural perturbation agent and a feature confusion agent, with the structural perturbation agent deciding on edge-connecting actions at nodes, and subsequently, the feature confusion agent deciding on feature modification actions at nodes based on structural perturbation information; S3: Constructing a joint parameter optimization objective based on a proximal policy optimization algorithm; evaluating the value of the global state using a centralized value network, and calculating the advantage function of the dual agents in conjunction with immediate rewards; constructing the policy loss and value loss of the dual agents; S4: Executing an iterative training process to optimize network parameters; collecting the interaction trajectories between the dual agents and the environment, and synchronously updating the parameters of the policy network and value network according to the joint parameter optimization objective; S5: Obtaining the social network anonymization results and evaluating the model utility; using the trained policy network to perform deterministic perturbations on the input test set graph data, outputting the anonymized graph data results, and calculating the anonymization success rate.

Description

Technical Field

[0001] This invention belongs to the field of network security technology and data privacy protection, specifically relating to a method and apparatus for social network anonymization based on multi-agent reinforcement learning. Background Technology

[0002] With the widespread adoption of social networks, massive amounts of user relationships, behavioral patterns, and interaction patterns are continuously collected and used for user profiling, recommendation systems, and content analysis. This type of social data contains rich information about social structure, but its high correlation and structural stability also bring serious privacy risks: attackers can leverage user connections, community structure, node degree distribution, or other metadata to infer users' real identities or sensitive relationship chains, thereby achieving deanonymization attacks. Therefore, how to effectively prevent privacy leaks while preserving the value of data analysis to the greatest extent has become a key technical issue in the publication and sharing of social network data.

[0003] Existing social network anonymization methods mainly include: Heuristic methods such as K-anonymization and L-diversity modify the structure or features at a fixed scale by constraining the perturbation range through predefined rules. While simple to implement, these strategies lack specificity and adaptability, failing to self-adjust based on target attack models and network characteristics. Excessive perturbation leads to severe shifts in statistical properties, compromising network usability, while insufficient perturbation fails to effectively protect privacy. To address these shortcomings, some research has focused on structure optimization based on topological attributes. For example, the patent CN119379466A, "A Method and Device for Social Network Structure Anonymization Based on Core Value Preservation Algorithm," proposes an anonymization scheme based on k-core preservation, which preserves node core structural attributes through local edge swapping. This method can improve the network structure shift problem, but it only focuses on a single topological feature, ignoring the joint influence of node attributes and semantic features, making it difficult to simultaneously achieve anonymization protection in both structural and feature dimensions. This results in sensitive information still being exposed in the high-dimensional feature space after anonymization. Summary of the Invention

[0004] This invention aims to overcome the aforementioned problems of existing social network anonymization methods and proposes a social network anonymization method and apparatus based on multi-agent reinforcement learning.

[0005] This invention introduces a multi-agent collaborative mechanism and action space pruning technology to intelligently perturb the structure and node features of social networks from multiple dimensions, achieving the goal of low-disruption, high-privacy, and highly usable anonymization. The first aspect of this invention relates to a reinforcement learning-based social network anonymization method, comprising the following steps: S1: Construct the reinforcement learning environment and initial state; acquire the social network graph data to be processed, model the anonymization task as a Markov decision process, and establish the state representation and reward feedback mechanism of the environment; S2: Perform dual-agent collaborative decision-making; construct a structural perturbation agent and a feature confusion agent. The structural perturbation agent makes decisions on the connection actions of the nodes, and then the feature confusion agent makes decisions on the feature modification actions of the nodes by combining the structural perturbation information. S3: Construct a joint parameter optimization objective based on the proximal policy optimization algorithm; evaluate the value of the global state using a centralized value network and calculate the advantage function of the two agents by combining immediate rewards; construct the policy loss and value loss of the two agents; S4: Execute an iterative training process to optimize network parameters; collect the interaction trajectories between the two agents and the environment, and update the parameters of the policy network and value network synchronously based on the joint parameters to optimize the objective. S5: Obtain the anonymization results of the social network and evaluate the effectiveness of the model; use the trained policy network to perform deterministic perturbation on the input test set graph data, output the anonymized graph data results and calculate the anonymization success rate.

[0006] Step S1 specifically includes the following steps: This invention constructs the anonymization task as a multi-agent Markov decision process, based on a training architecture of centralized training and distributed execution. The environment is defined by the social network data modeling method, user injection mechanism, and pre-trained agent classification model. The state is defined as the social network graph, the actions are adding edges between injected accounts and existing users and modifying the features of injected accounts, and the reward is the difference in classification confidence of the model for the social network before and after anonymization. The objective is to maximize the expected reward. Specifically, it includes: S1.1: Filtering from social network data Using users with sensitive attributes as the central node, samples are taken from them. By skipping neighbor nodes and their edges, we obtain the graph dataset used for training. This includes graph data and corresponding graph labels; S1.2: Define each social network graph ,in For inclusion A set of nodes for each user. For the user's set of edges, It is an adjacency matrix; The characteristic matrix, For feature dimensions; S1.3: Traversing the dataset Count and determine the global maximum number of nodes. As the action space dimension of the structural perturbation agent, the insufficient dimension is masked through a masking mechanism; S1.4: Directed Graph Inject a single anonymous account node Extract the adjacency matrix of the current time-to-time graph. and characteristic matrix The states that constitute a reinforcement learning environment are defined as follows: ; S1.5: Pre-trained agent classification model It consists of a graph encoder and a classifier, whose parameters are frozen during the reinforcement learning phase. The surrogate classification model is then computed. The degree of decrease in confidence of social network graph prediction after action perturbation is used as the immediate reward shared by the two agents. .

[0007] Step S2 specifically includes the following steps: S2.1: Structural Disturbance Agent Using a graph encoder Extracting the node embedding matrix of the graph And generate global graph embeddings through average pooling. The gradient of the computation node with respect to the loss of the surrogate classification model is used as the structural sensitivity. Based on this, the connected nodes are sorted and masked out to obtain the set of valid actions. The observation matrix is ​​constructed by concatenating the injected node embedding, the effective node embedding, and the global graph-level embedding. ;Structured strategy network pairs Process and output the probability distribution of edge connection actions. Sampling target node Perform the edge connection operation and update the adjacency matrix. The features of the target node are extracted and concatenated with the global graph-level embedding to construct a structured message vector. .

[0008] S2.2: Feature-Confused Agent Call the graph encoder The current graph state is recoded, combining its own embedding, global graph-level embedding, and the structured message vector. fusion to construct observation vectors Feature-based policy networks Process the action probability distribution of the output attribute category Target attribute dimension of sampling Perform a permutation operation and update the feature matrix as follows: .

[0009] Step S3 specifically includes the following steps: S3.1 Centralized Value Network Predict the value assessment of the current state and the state at the next moment, and combine it with the immediate reward. Calculate timing difference error With generalized advantage function The mean squared error between the value network prediction and the actual return target is calculated as the value loss. ; S3.2: For both agents, calculate the probability ratio of the new policy to the old policy when performing actions. Using the cropping operation Restrict it to the clipping parameters Within the controlled interval, combined with the dominance function The structural perturbation strategy loss was calculated separately. Feature obfuscation strategy loss The aforementioned strategy losses and value losses are weighted according to preset coefficients. , Weighted summation to construct the joint total loss function .

[0010] Step S4 specifically includes the following steps: Initialize network parameters, total number of iterations With time step limit In each round of training, social network graphs are sampled. Injected into a single anonymous account Construct the initial state Clear the acquisition buffer ; at time step Inside, Based on current observations Output action probability distribution And sample structural actions The system updates its state and extracts the structured message vector. ; then Based on observation Output action probability distribution Sample and execute feature actions The state transitions to the next time step. Transform the state transition tuple Store in the acquisition buffer .when After each time step is completed, the buffer is traversed. The complete trajectory in the calculation , And various losses, based on the total loss function Calculate the gradient and synchronously update the parameters of the policy network, value network, and shared graph encoder using the backpropagation algorithm; the system then clears the buffer. Then proceed to the next round of graph sampling and training iterations until the number of training rounds reaches the preset limit. .

[0011] Step S5 specifically includes the following steps: S5.1: Load the optimal policy network and For the test set graph, the total number is The graph instance is injected with anonymized accounts to construct the initial state; at each time step Within the system, the two agents, based on current observations, sequentially and directly select the action with the highest probability from the probability distribution. and Perform deterministic perturbations; S5.2: After After each update step, the final anonymized graph is obtained; this is then input into the target graph neural network classifier. Inference is performed to statistically predict the proportion of samples whose labels change, and the anonymization success rate is calculated. .

[0012] A second aspect of the present invention relates to a social network anonymization apparatus based on multi-agent reinforcement learning, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a social network anonymization method based on multi-agent reinforcement learning of the present invention.

[0013] This invention introduces a multi-agent reinforcement learning framework and performs adaptive perturbation optimization in both structural and feature dimensions, thereby significantly improving anonymization strength and privacy protection performance while ensuring the usability and functional integrity of social networks.

[0014] The advantages of this invention are: (1) This invention proposes a two-dimensional anonymization optimization framework based on multiple agents, which decomposes the anonymization task into two sub-tasks: structural perturbation and feature confusion. Different agents are responsible for decision-making and optimization in their respective dimensions in parallel. This achieves three-dimensional anonymization with synchronous collaboration between structure and features, thereby significantly enhancing social anonymization performance while maintaining controllable loss of statistical properties and downstream analysis performance.

[0015] (2) This invention proposes an adaptive dual action space pruning mechanism, which significantly reduces the computational and search overhead caused by a large action space by automatically eliminating redundant and invalid candidate actions during policy updates. This mechanism uses gradient priority to filter potentially effective actions and combines feasibility masks to ensure that the policy network only samples the action subspace with actual contributions, making the training process efficient and stable. With the effective pruning of the action space, this invention achieves faster training iterations, higher convergence stability, and better quality of anonymized policy generation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the social network anonymization method based on multi-agent reinforcement learning of the present invention.

[0018] Figure 2 This is a flowchart illustrating the training and optimization process of the reinforcement learning model in this invention.

[0019] Figure 3 This is a schematic diagram of the social network anonymization device of the present invention.

[0020] Figure 4 This is a graph showing the convergence curve of the training reward value of the algorithm model of this invention.

[0021] Figure 5 This is a graph showing the comparison of the anonymization success rates of the algorithm model of this invention with those of different methods. Specific implementation methods

[0022] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0023] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0024] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While only preferred methods and materials have been described herein, any methods and materials similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe methods and / or materials associated with those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

[0025] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be readily apparent to those skilled in the art. This application specification and embodiments are merely exemplary.

[0026] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0027] Unless otherwise specified, the term "parts" in this invention refers to parts by weight.

[0028] Example 1

[0029] This embodiment proposes a social network anonymization method based on multi-agent reinforcement learning, referring to... Figure 1 and Figure 2 The specific implementation steps are as follows: S1: Construct the reinforcement learning environment and initial state; acquire the social network graph data to be processed, model the anonymization task as a Markov decision process, and establish the state representation and reward feedback mechanism of the environment; S2: Perform dual-agent collaborative decision-making; construct a structural perturbation agent and a feature confusion agent. The structural perturbation agent makes decisions on the connection actions of the nodes, and then the feature confusion agent makes decisions on the feature modification actions of the nodes by combining the structural perturbation information. S3: Construct a joint parameter optimization objective based on the proximal policy optimization algorithm; evaluate the value of the global state using a centralized value network and calculate the advantage function of the two agents by combining immediate rewards; construct the policy loss and value loss of the two agents; S4: Execute an iterative training process to optimize network parameters; collect the interaction trajectories between the two agents and the environment, and update the parameters of the policy network and value network synchronously based on the joint parameters to optimize the objective. S5: Obtain the anonymization results of the social network and evaluate the effectiveness of the model; use the trained policy network to perform deterministic perturbation on the input test set graph data, output the anonymized graph data results and calculate the anonymization success rate.

[0030] Step S1 specifically includes the following steps: This invention constructs the anonymization task as a multi-agent Markov decision process, based on a training architecture of centralized training and distributed execution. The environment is defined by the social network data modeling method, user injection mechanism, and pre-trained agent classification model. The state is defined as the social network graph, the actions are adding edges between injected accounts and existing users and modifying the features of injected accounts, and the reward is the difference in classification confidence of the model for the social network before and after anonymization. The objective is to maximize the expected reward. Specifically, it includes: S1.1: Construct a graph classification dataset based on real-world interaction data from the online social platform Reddit. Each node in the dataset represents a Reddit user. If two users have direct interaction (such as replying, retweeting, or mentioning) within a preset time window, an undirected edge is established between them. Each social network graph instance corresponds to a specific subreddit community, and the community name serves as the graph's category label. Select categories from the dataset... Each user with sensitive attributes is used as a central node, and samples are taken from each central node. By skipping neighbor nodes and their edges, we obtain the graph dataset used for training. This includes graph data and corresponding graph labels; S1.2: Define each social network graph ,in For inclusion A set of nodes for each user. For the user's set of edges, The adjacency matrix represents the current social network structure, which is updated by the structural perturbation agent establishing connections between the injected account and the existing user. The characteristic matrix, The feature dimension is defined as follows: Node features are composed of multiple categorical attribute fields, each corresponding to a user attribute such as interest topics, activity level, age, and gender, and are encoded using one-hot encoding within their respective value spaces; S1.3: To adapt to the fixed-dimensional output requirements of the agent policy network, the dataset is pre-traversed. Count and determine the global maximum number of nodes. The action space dimension of the subsequent structural perturbation agent is uniformly set as Regarding the number of nodes In the graph instance, the missing dimensions are masked during the calculation process using a masking mechanism.

[0031] S1.4: To the current social network graph Inject a single anonymous account node As the sole operator for subsequent coordinated perturbations, the initialization environment after injection is defined as follows: The set of nodes for initializing the environment is as follows: , edge set Adjacency matrix , characteristic matrix The initial characteristics of the injected account are generated by randomly sampling several attribute fields. The state at time t is defined as follows ,in for The adjacency matrix at time 10:00. for The node feature matrix at time step; S1.5: Pre-train a proxy classification model The system consists of a graph encoder and a classifier. The graph encoder uses a two-layer graph convolutional network, with the first layer outputting a 128-dimensional graph and the second layer outputting a 64-dimensional graph. Each layer is followed by a ReLU activation function, and global average pooling is used to generate a 64-dimensional graph-level embedding. The classifier is a single-layer fully connected network that maps the 64-dimensional graph-level representation to the number of categories in the output. The function outputs a class probability distribution. Each input graph data first passes through an encoder. The encoder maps node features and structural information into representation vectors, which are then fed into the classifier to output the prediction confidence of the target category. During the pre-training phase, the encoder... Both the classifier and the classifier are jointly optimized using cross-entropy loss:

[0032] After training, the model parameters are kept completely frozen for calculating the reward function in subsequent reinforcement learning. The agent directly reuses this frozen encoder. The output embedding serves as an environmental observation. Two agents share one in... Rewards calculated after the time disturbance ends The reward is designed to measure a classifier's performance on anonymized social network graphs. The degree of classification error. The reward function can be defined as:

[0033] in, The model predicts anonymized social network graphs. This represents the confidence level of the non-real labels. The goal of this invention is to maximize this reward, i.e., the greater the deviation of the model's prediction of the anonymized social network graph from the real labels, the higher the reward.

[0034] Step S2 specifically includes the following steps: S2.1: Structural Disturbance Agent Responsible for structural anonymization. Its policy network. From parameters Parameterization determines the injection account Which existing user should be involved? Establish a connection. First, use a graph encoder. right Moment State adjacency matrix With characteristic matrix Feature extraction is performed to obtain the node embedding matrix of the entire graph. ,in For the hidden layer feature dimension. Average pooling is performed on the embeddings of all nodes to generate a graph. Graph-level embedding of global semantics :

[0035] This graph-level embedding is used to provide global context information for the policy network, given the complete action space. The target space is too large and contains many invalid targets. This invention introduces an adaptive dual action space clipping mechanism to address this issue. Focus only on sensitive users who play a key role in classification to reduce computational overhead. Define node structure sensitivity. Loss function for proxy classification model Node embedding Gradient norm: A higher value indicates a greater contribution from the user to the graph classification results. Nodes are sorted in descending order of sensitivity, and the top-ranked nodes are selected. % of the nodes are selected as the pruned candidate user node set. After obtaining the candidate node set based on sensitivity pruning, a masking mechanism is further used to remove nodes that have already established connections with the injected nodes according to the edge connection state of the current graph, resulting in the final set of valid actions. . Structural strategy network Only accept this set of valid actions. The embedding of middle nodes is used as input to avoid the generation of illegal actions. Assume the entire graph, after double action space clipping, retains... There are 10 valid nodes, whose embeddings constitute a submatrix. ∈ ,in =64. Further construct the observation matrix. ∈ , its first Behavior:

[0036] in This represents a vector concatenation operation. For the embedding of the injection node, For the first Embedding of individual nodes, This is a global graph-level embedding. Structured Policy Network It consists of fully connected layers ranging from 128 to 64 dimensions and the nonlinear activation function ReLU, used to fit the connection value of node pairs in the current global context. It receives an observation matrix as input, where each row of the matrix corresponds to the augmented feature of a valid node. The policy network independently calculates a score for each row and then... The function converts the score into an action probability distribution:

[0037] in This represents the unnormalized logical value output by the neural network. To ensure the exploratory nature of the agent, this invention does not directly select the action with the highest probability, but instead randomly samples the target node at the current moment according to the above probability distribution:

[0038] in This indicates a random sampling operation based on probability. This operation randomly selects node indices based on probability magnitude; nodes with higher probabilities are more likely to be selected, but nodes with lower probabilities may still be selected, thus implementing the exploration mechanism in reinforcement learning. (Execute action) The adjacency matrix of the post-social network was updated to This matrix will serve as the final graph structure state determined at the current time step, and the state transition will proceed to... To achieve accurate responses to structural actions during the feature obfuscation stage and prevent the aggregation operation of graph convolution from diluting the specific semantics of newly added edges, an extraction policy network is needed. For the selected target node The original embedding This is concatenated with the original graph-level embedding to construct a structured message vector. :

[0039] S2.2: Feature-Confused Agent Responsible for selecting and modifying injection nodes based on structural disturbances. A certain characteristic. Its policy network. A feature policy network The structure first uses a fully connected layer of 128 to 64 dimensions, applying the ReLU activation function, followed by an output fully connected layer to map the 64-dimensional hidden states. Dimensional attribute field. Determined by parameters. Parameterization. This is done after the structural action is executed and the adjacency matrix is ​​updated. Then, call the graph encoder. The current graph state is re-encoded to obtain the updated full graph node embedding matrix. and the updated global graph-level embedding Feature-confusing agent constructs current observations The updated state and message vector information are then merged. The following information is concatenated into a vector:

[0040] Feature-confusing agents will and message vectors from structurally perturbed agents As a policy network Input, Feature Policy Network Nonlinear feature extraction is performed on the observation vector, outputting a probability distribution covering all feature categories: The action at the current time step is sampled based on this distribution. :

[0041] in Represents the first eigenvector in the global feature vector. Each dimension. The action This indicates that the feature vector to be injected into the node will be of dimension 1. The subvector corresponding to the attribute is updated to one-hot encoding, that is, the first subvector in the group. Set one dimension to 1, and set the remaining dimensions in the same group to 0. This yields the updated feature matrix. Environmental transfer to .

[0042] Step S3 specifically includes the following steps: S3.1 Centralized Value Network and Generalized Advantage Estimation. This invention employs a centralized value network. Used to evaluate the global state The value network provides a baseline estimate for the policy network. It shares the same graph neural network encoder and observation module with both agents. Its structure is a two-layer fully connected network. The first layer maps the 128-dimensional input to 64 dimensions, and after ReLU activation, the second layer outputs a single scalar as the value estimate of the current state. The goal is to make the predicted state value This invention approximates the cumulative discounted returns derived from the environment. It uses temporal difference error as the basis for estimation bias and combines it with generalized advantage estimation to reduce the high variance of return estimation, thereby improving the stability of policy updates. First, the temporal difference error is calculated based on the immediate reward sequence. :

[0043] in From Instant rewards earned at any time, discount factor =0.99. The shared advantage function is calculated using the generalized advantage estimation method. :

[0044] Where the smoothing parameter . Reflects the state Below, the actual benefit of the joint action of two agents is compared to the advantage estimated by the value network benchmark. If This indicates that the joint action is better than expected, and the probability of sampling this action should be increased. Parameters of the value network. Updates are performed by minimizing the mean squared error between the temporal difference objective and the predicted value. The value network loss function is defined. :

[0045] This loss makes To more accurately fit its temporal difference objective and improve the stability and accuracy of value estimation.

[0046] S3.2 agent's near-end strategy optimization update. and The goal is to maximize the pruning objective of the near-end policy optimization. The function defines the policy probability ratio:

[0047] To prevent large policy update magnitudes from causing training instability, a pruning operation is introduced to limit the ratio to a certain range. .in Setting it to 0.2 is a pruning parameter used to control the update magnitude and prevent strategy collapse due to excessively large single update steps. The pruned ratio is defined as:

[0048] Based on the pruning policy objective, the policy losses of the two agents are as follows:

[0049] To achieve joint optimization of the policy and value network, a total loss function is constructed:

[0050] Among them, the trade-off coefficient , .

[0051] Step S4 specifically includes the following steps: The training process of this invention is based on a multi-agent reinforcement learning framework with centralized training and distributed execution. The specific training process is as follows: First, the structural perturbation policy network is initialized. Feature Confusion Strategy Network Centralized value network Randomly initialize the network parameters. Set the training hyperparameters: total number of iterations. The maximum number of time steps per decision round The updated batch size is 32, and the cropping parameters are... Load the social network graph dataset to be trained. and pre-trained frozen proxy classification model All trainable parameters of the policy network and value network are updated via the Adam optimizer with a learning rate of [missing value]. .

[0052] The system begins executing multiple rounds of training iterations. At the start of each training round, data is retrieved from the dataset. Resample a social network graph As the current context, a single anonymized account is injected into this graph. It initializes the connection between the current environment and existing users to zero, and initializes the feature vector to a zero vector or a random vector, thereby constructing the initial state of the current environment. At the same time, clear the acquisition buffer. Prepare to store the experience data for this round.

[0053] Within the current round, the system proceeds according to the preset time step. Execute the loop interaction process. At each time step... In the middle, the structural perturbation agent is based on the current observation By combining sensitivity pruning and legality masking mechanisms, a normalized action probability distribution for valid candidate nodes is output. The agent acts according to this distribution sampling structure. Determine the target of the connection. Then perform the edge connection operation to update the adjacency matrix to the intermediate state. After the structural action is executed, the system calls the shared graph encoder to re-encode the updated graph state. Simultaneously, the decision context of the structural agent is extracted to construct the structural message vector. The feature-confusing agent constructs observations by combining its recoded self-embedding, global embedding, and received structured message vectors. The policy network outputs the probability distribution of actions for the attribute fields. The intelligent agent samples the features and actions accordingly. Perform a one-hot permutation operation to update the feature matrix to the state of the next time step. The system records the current state, the combined actions performed, the immediate reward obtained, and the state at the next moment, and then transitions this state into a tuple. Store in the acquisition buffer .when Once each time step has been executed, the current round terminates.

[0054] Sampling is paused when the buffer is full, and the data in the buffer is used to update the parameters. The system traverses the buffer. The complete trajectory in, utilizing value networks Calculate the timing difference error at each time step And calculate the shared advantage function based on the generalized advantage estimation algorithm. The system calculates the probability ratio between the old and new strategies. And combined with the calculated advantage function With cutting parameters Construct policy pruning loss functions for structural agents and feature agents respectively. and Simultaneously, the mean squared error between the value network's predicted value and the actual return target is calculated as the value loss. Finally, the system is based on the total loss function. Calculate the gradient and simultaneously update the parameters of the two policy networks using the backpropagation algorithm. , Value network parameters And the shared graph encoder parameters. After the parameters are updated, the system clears the buffer. Then proceed to the next round of graph sampling and training iterations until the number of training rounds reaches the preset limit. .

[0055] Step S5 specifically includes the following steps: After training, load the optimal policy network of the two agents obtained in step S4. and Prepare the test dataset. Process each social network graph instance in the test set at a time, where The total number of social network graph instances in the test set. For a specific graph instance sampled. Execute from arrive The complete anonymization process. Always towards Inject a single anonymous account Initialize its edges and features to construct the initial state. At each time step Within the system, the agent executes a predetermined strategy. The structurally perturbative agent selects the candidate target node with the highest connection probability based on current observations:

[0056] Perform edge connections and update the adjacency matrix. Reencode the updated graph and extract the message vector. The feature-confusing agent selects the attribute category with the highest permutation probability based on the current observations:

[0057] Perform the action and update the adjacency matrix. After... After a deterministic perturbation, the final anonymous graph is obtained. The generated anonymous graph The target evaluation classification model is used for forward inference. If the confidence level of the target evaluation classification model in predicting the true label of the image is significantly lower than that of the original image, or if the predicted label changes, then the sample is considered successfully anonymized. The system performs statistical analysis on the entire test set. The Anonymization Success Rate (ASR) serves as the final quantitative indicator for evaluating the technical effectiveness of this invention. This indicator reflects the proportion of graph samples misclassified by the target classifier after editing using the method of this invention, thus demonstrating the degree of performance degradation. The Anonymization Success Rate is defined as the proportion of graph samples in the test set that are misclassified by the target classifier after editing using the method of this invention. Its calculation formula is as follows:

[0058] in For target graph neural network classifier, For the original tag, This is a social network diagram after anonymization.

[0059] To verify the effectiveness of the proposed method, a validation test was conducted on a graph classification dataset based on real-world social platform Reddit interaction data. The experiment was set to a total of 1000 iterations, executing reinforcement learning iterations. Figure 4 As shown, with the increase in training steps, the instantaneous reward value shared by the two agents initially increases and eventually remains stable and converges; the model is tested on the anonymized graph data output on the test set, as shown... Figure 5As shown, the results demonstrate that the anonymization success rate of the target classifier is improved after the dual-agent cooperative perturbation method of this invention. This proves that the multi-agent cooperative optimization mechanism proposed in this invention has strong robustness in complex graph network environments, and the policy network can quickly and stably learn the optimal action distribution. Compared with traditional single-agent reinforcement learning methods or heuristic structure modification methods, the algorithm proposed in this invention can achieve better anonymization results with more efficient feature and structure joint action sequences.

[0060] This invention can be applied to anonymization tasks in real social network scenarios, improving anonymization strength and privacy protection performance.

[0061] In summary, this invention overcomes the problems of existing methods that compromise network availability and fail to effectively protect privacy due to insufficient perturbation by introducing a multi-agent reinforcement learning framework and performing adaptive perturbation optimization in both structural and feature dimensions, thus significantly improving anonymization strength and privacy protection performance.

[0062] Example 2

[0063] Reference Figure 3 This embodiment relates to a social network anonymization device based on multi-agent reinforcement learning, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a social network anonymization method based on multi-agent reinforcement learning according to Embodiment 1.

[0064] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A social network anonymization method based on multi-agent reinforcement learning, characterized in that, Includes the following steps: S1: Construct the reinforcement learning environment and initial state; acquire the social network graph data to be processed, model the anonymization task as a Markov decision process, and establish the state representation and reward feedback mechanism of the environment; S2: Perform dual-agent collaborative decision-making; construct a structural perturbation agent and a feature confusion agent. The structural perturbation agent makes decisions on the connection actions of the nodes, and then the feature confusion agent makes decisions on the feature modification actions of the nodes by combining the structural perturbation information. S3: Construct a joint parameter optimization objective based on the proximal policy optimization algorithm; evaluate the value of the global state using a centralized value network and calculate the advantage function of the two agents by combining immediate rewards; construct the policy loss and value loss of the two agents; S4: Execute an iterative training process to optimize network parameters; collect the interaction trajectories between the two agents and the environment, and update the parameters of the policy network and value network synchronously based on the joint parameters to optimize the objective. S5: Obtain the anonymization results of the social network and evaluate the effectiveness of the model; use the trained policy network to perform deterministic perturbation on the input test set graph data, output the anonymized graph data results and calculate the anonymization success rate.

2. The social network anonymization method based on multi-agent reinforcement learning according to claim 1, characterized in that: Step S1 specifically includes the following steps: S1.1: Filtering from social network data Using users with sensitive attributes as the central node, samples are taken from them. By skipping neighbor nodes and their edges, we obtain the graph dataset used for training. This includes graph data and corresponding graph labels; S1.2: Define each social network graph ,in For inclusion A set of nodes for each user. For the user's set of edges, It is an adjacency matrix; The characteristic matrix, For feature dimensions; S1.3: Traversing the dataset Count and determine the global maximum number of nodes. As the action space dimension of the structural perturbation agent, the insufficient dimension is masked by a masking mechanism; S1.4: Directed Graph Inject a single anonymous account node Extract the adjacency matrix of the current time-to-time graph. and characteristic matrix The states that constitute a reinforcement learning environment are defined as follows: ; S1.5: Pre-trained agent classification model It consists of a graph encoder and a classifier, whose parameters are frozen during the reinforcement learning phase; the surrogate classification model is then calculated. The degree of decrease in confidence of social network graph prediction after action perturbation is used as the immediate reward shared by the two agents. .

3. The social network anonymization method based on multi-agent reinforcement learning according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1: Structural Disturbance Agent Using a graph encoder Extracting the node embedding matrix of the graph Global graph embeddings are generated through average pooling. The gradient of the node's loss to the surrogate classification model is used as the structural sensitivity. Based on this, the connected nodes are sorted and masked out to obtain the set of valid actions. The observation matrix is ​​constructed by concatenating the injected node embedding, the effective node embedding, and the global graph-level embedding. ;Structured strategy network pairs Process and output the probability distribution of edge connection actions. Sampling target node Perform the edge connection operation and update the adjacency matrix. Extract the target node features and concatenate them with the global graph-level embedding to construct a structured message vector. ; S2.2: Feature-Confused Agent Call the graph encoder The current graph state is recoded, combining its own embedding, global graph-level embedding, and the structured message vector. fusion to construct observation vectors Feature-based policy network pairs Process the action probability distribution of the output attribute category Target attribute dimension of sampling Perform a permutation operation and update the feature matrix as follows: .

4. The social network anonymization method based on multi-agent reinforcement learning according to claim 1, characterized in that: Step S3 specifically includes the following steps: S3.1 Centralized Value Network Predict the value assessment of the current state and the state at the next moment, and combine it with the immediate reward. Calculate timing difference error With generalized advantage function The mean squared error between the value network prediction and the actual return target is calculated as the value loss. ; S3.2: For both agents, calculate the probability ratio of the new policy to the old policy when performing actions. Using the cropping operation Restrict it to the clipping parameters Within the controlled interval, combined with the dominance function The structural perturbation strategy loss was calculated separately. Feature obfuscation strategy loss The aforementioned strategy losses and value losses are weighted according to preset coefficients. , Weighted summation to construct the joint total loss function .

5. The social network anonymization method based on multi-agent reinforcement learning according to claim 1, characterized in that: Step S4 Specifically, the following steps are included: Initialize network parameters, total number of iterations With time step limit In each round of training, social network graphs are sampled. Injected into a single anonymous account Construct the initial state Clear the acquisition buffer ; at time step Inside, Based on current observations Output action probability distribution And sample structural actions The system updates its state and extracts the structured message vector. ; then Based on observation Output action probability distribution Sample and execute feature actions The state transitions to the next time step. Transform the state transition tuple Store in the acquisition buffer ;when After each time step is completed, the buffer is traversed. The complete trajectory in the calculation , And various losses, based on the total loss function Calculate the gradient and synchronously update the parameters of the policy network, value network, and shared graph encoder using the backpropagation algorithm; the system then clears the buffer. Then proceed to the next round of graph sampling and training iterations until the preset upper limit of training rounds is reached. .

6. The social network anonymization method based on multi-agent reinforcement learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: S5.1: Load the optimal policy network and For the test set graph, the total number is The graph instance is injected with anonymized accounts to construct the initial state; at each time step Within the system, the two agents, based on current observations, sequentially and directly select the action with the highest probability from the probability distribution. and Perform deterministic perturbations; S5.2: After After each update step, the final anonymized graph is obtained; this is then input into the target graph neural network classifier. Inference is performed to statistically predict the proportion of samples whose labels change, and the anonymization success rate is calculated. .

7. A social network anonymization device based on multi-agent reinforcement learning, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a social network anonymization method based on multi-agent reinforcement learning as described in any one of claims 1-6.

Images