Method and system for inhibiting graph disassembly based on DQN

Through the DQN-based suppression graph dismantling method, using graph neural network and reinforcement learning, the problem of poor performance of graph confrontation attacks in the existing technology in the suppression graph dismantling task is solved, and the ability to efficiently find key nodes is achieved, which improves the Suppress the effect of image disassembly.

CN119052387APending Publication Date: 2024-11-29ANHUI UNIV +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411135911.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing graph adversarial attack algorithms are not effective in suppressing graph disassembly tasks, mainly because they cannot propagate during gradient optimization, resulting in poor results in the optimization process.

Method used

Using the DQN-based suppression graph disassembly method, a small-scale synthetic graph is generated through a random graph generation model, characterized as an undirected graph, and a graph neural network is used to reduce dimensionality, generate virtual nodes, and use DQN algorithm to train agents for attack exploration. Find key nodes that need to be hidden.

Benefits of technology

Through the enhanced learning of DQN algorithm and the fitting ability of deep learning of graphs, the agent can learn the optimal attack strategy, effectively find key nodes in the target network, improve the effect of suppressing graph disassembly, and reduce the complexity of action. Avoid irreversible damage to the undirected graph information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052387A_ABST
    Figure CN119052387A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of network security, in particular to a method and system for inhibiting graph disassembly based on DQN. According to the method, the DQN algorithm is applied to suppression of graph disassembly, and by means of the fitting ability of graph deep learning and the DQN normal form in reinforcement learning, an agent explores in virtual nodes obtained based on processing of a plurality of small-scale generated graphs, so that the agent learns an optimal attack strategy; and the method is further applied to a target network to find key nodes needing to be hidden. Through simulation comparison, compared with an existing graph antagonism attack algorithm, the method disclosed by the invention has a better effect when a graph disassembling task is inhibited. According to the method, the problem that the effect is poor when an existing graph antagonism attack is applied to a graph disassembling suppression task is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and more specifically, to: 1. a method for disassembling a suppression graph based on DQN, and 2. a system for disassembling a suppression graph based on DQN. Background Art

[0002] With the rapid development of society, there are increasingly complex relationships between entities in various artificial or natural systems. In order to better study the relationships between these entities, researchers usually model them as network (graph) structures. There is an information cascade phenomenon in the network, that is, some nodes may have a huge impact on the entire network. Based on this situation, network disassembly (graph disassembly) is born: it can efficiently find key nodes in the network and remove key nodes from the network, thereby greatly destroying the network structure and function. Therefore, graph disassembly may cause network security issues.

[0003] In order to suppress graph disassembly, the inventor considered using graph adversarial attacks - by adding or deleting edges and modifying node features to deceive the graph neural network model and make it misclassify nodes. However, the inventor later discovered that the existing graph adversarial attack algorithm is based on gradient optimization - the attack is guided by calculating the edge gradient matrix. However, the indicator for evaluating the task of suppressing graph disassembly cannot be derived from the elements in the adjacency matrix, making it impossible to propagate the gradient information of the optimization process, resulting in poor results when the existing graph adversarial attack is applied to the task of suppressing graph disassembly. Summary of the invention

[0004] Based on this, it is necessary to provide a DQN-based suppression graph disassembly method and system to address the problem that existing graph adversarial attacks have poor effects when applied to suppression graph disassembly tasks.

[0005] The present invention is implemented by the following technical solutions:

[0006] In a first aspect, the present invention discloses a method for suppressing graph decomposition based on DQN, comprising the following steps:

[0007] Step 1: Generate multiple small-scale synthetic images using a random graph generation model;

[0008] Step 2: Representing the multiple small-scale synthetic graphs in step 1 into undirected graph structures to obtain multiple undirected graphs; wherein the number of the undirected graphs is the same as that of the small-scale synthetic graphs and they correspond one to one;

[0009] Step 3, using a graph neural network to reduce the dimensions of the multiple undirected graphs in step 2, and obtain multiple virtual nodes that can simulate the information of the undirected graphs after the dimension reduction; wherein the number of virtual nodes is the same as that of the undirected graphs, and they correspond one to one;

[0010] Step 4: Use the multiple virtual nodes in step 3 as the state space and use the DQN algorithm to attack, explore, and train the agent.

[0011] The actions performed by the agent include: for two nodes at both ends of an edge, one of the nodes is selected to be hidden;

[0012] For a certain action, the worse the state of the state space is after it is executed, the greater the reward corresponding to the action;

[0013] Step 5: Process the target network with the trained intelligent agent to obtain the key nodes that need to be hidden in the target network.

[0014] This DQN-based suppression graph decomposition method implements the method or process according to an embodiment of the present disclosure.

[0015] In a second aspect, the present invention discloses a system for DQN-based inhibition graph disassembly, which uses the DQN-based inhibition graph disassembly method of the first aspect.

[0016] The DQN-based system for inhibiting graph disassembly includes: a graph generation module, a graph representation module, a graph simulation module, an agent training module, and a graph disassembly inhibition module.

[0017] The graph generation module is used to generate multiple small-scale synthetic graphs using a random graph generation model. The graph representation module is used to represent multiple small-scale synthetic graphs into undirected graph structures to obtain multiple undirected graphs. The graph simulation module is used to use graph neural networks to reduce the dimensions of multiple undirected graphs and obtain multiple virtual nodes that can simulate the information of the reduced-dimensional undirected graphs. The agent training module is used to use multiple virtual nodes as state spaces and use the DQN algorithm to attack, explore, and train the agent. The graph disassembly and suppression module is used to process the target network with the trained agent to obtain the key nodes that need to be hidden in the target network.

[0018] The system for DQN-based suppression graph decomposition implements a method or process according to an embodiment of the present disclosure.

[0019] In a third aspect, the present invention discloses a readable storage medium having computer program instructions stored therein, and when the computer program instructions are read and executed by a processor, the method for disassembling a DQN-based inhibition graph as disclosed in the first aspect is executed.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. The present invention applies the DQN algorithm to suppress graph disassembly. With the help of the fitting ability of graph deep learning and the DQN paradigm in reinforcement learning, the intelligent agent is enabled to explore in the virtual nodes obtained based on multiple small-scale generated graphs, so that the intelligent agent can learn the optimal attack strategy, which is then applied to the target network to find the key nodes that need to be hidden.

[0022] 2. The present invention uses a graph neural network to process an undirected graph generated by a small-scale generated graph, and then generates virtual nodes that can simulate the information of the undirected graph. On the one hand, it can reduce the feature dimension and facilitate the application of DQN for processing; on the other hand, it avoids the DQN intelligent agent from directly processing the undirected graph, preventing irreversible damage to the information of the undirected graph.

[0023] 3. The present invention applies the DQN algorithm to suppress graph decomposition, transforming the selection of opposite edges into hiding the nodes at both ends of the opposite edges, which effectively reduces the complexity of the action and is conducive to accelerating the processing progress of DQN. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 It is a flowchart of the method for disassembling a suppression graph based on DQN in Embodiment 1 of the present invention;

[0026] Figure 2 for Figure 1 Specific flow chart of step three in the figure;

[0027] Figure 3 This is a structural diagram of the system for disassembling the DQN-based suppression graph in Example 2 of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is considered to be "fixed to" another component, it may be directly fixed on the other component or there may be a central component at the same time.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" used herein includes any and all combinations of one or more of the related listed items.

[0031] Example 1

[0032] See also Figure 1 , Figure 1 The process of the method for suppressing graph disassembly based on DQN proposed in this embodiment 1 is shown, which includes the following steps:

[0033] Step 1: Use the random graph generation model to generate multiple small-scale synthetic graphs.

[0034] The purpose of step one is mainly to provide sufficient learning samples for the subsequent DQN agent.

[0035] Small-scale synthetic graphs are often used to simulate real networks such as social networks and neural networks, and have specific structural characteristics, such as high clustering coefficient and short average path length.

[0036] Generally, random graph generation models such as ER model and WS model are used to generate a sufficient number of small-scale synthetic graphs. The number of small-scale synthetic graphs is set to P, and the value of P can range from dozens to hundreds, which can be adjusted according to actual conditions and computing resources.

[0037] Step 2: Represent the multiple small-scale synthetic graphs in step 1 into undirected graph structures to obtain multiple undirected graphs; wherein the number of the undirected graphs is the same as that of the small-scale synthetic graphs and they correspond one to one.

[0038] Compared with small-scale synthetic graphs, undirected graph structures are a more general mathematical representation model that can be used to represent various types of networks and relationships, and their structural features can be very diverse.

[0039] Then, the small-scale synthetic graph is represented as an undirected graph structure, which clarifies the network relationship in the graph. In other words, the p-th small-scale synthetic graph corresponds to the p-th undirected graph, p∈[1,P].

[0040] Specifically, the mathematical expression of an undirected graph is: G=G(V,E).

[0041] In the formula, G represents an undirected graph; G(V,E) represents the graph structure composed of V and E;

[0042] V represents the set of all nodes in G; V = {v1,v2,…,v N}; v1~v N Represents the N nodes in V;

[0043] E represents the set of all undirected edges in G; E = {e1, e2, …, e M}; e1~e M Represents the M undirected edges in E.

[0044] In other words, the graph structure of an undirected graph can be viewed as consisting of N nodes and M edges.

[0045] In addition, the adjacency matrix A corresponding to the undirected graph is a two-dimensional matrix, in which the value A(v i ,v j ) represents the node v i and node v j Is there an undirected edge between:

[0046] Step three, use a graph neural network to reduce the dimensions of the multiple undirected graphs in step two, and obtain multiple virtual nodes that can simulate the information of the undirected graphs after dimension reduction; wherein the number of virtual nodes is the same as that of the undirected graphs, and they correspond one to one.

[0047] This step is to simulate the information of the undirected graph to facilitate DQN processing and avoid the DQN agent from directly performing hidden operations on the undirected graph and causing real damage to the graph.

[0048] Since the operation of this step for undirected graphs is similar, we will use any undirected graph in step 2 (represented by I) for specific explanation - see Figure 2 , step three includes:

[0049] S301, map I to a graph neural network so that each node of I is mapped to a low-dimensional vector space to achieve dimensionality reduction of I.

[0050] The graph neural network can be regarded as a low-dimensional vector space. After mapping each node of I to the low-dimensional vector space, the nodes of the graph neural network can be used as learning nodes to capture the structural and semantic information corresponding to the nodes of I.

[0051] Among them, the learning node can be expressed as: Z = f(A, X, Θ GNN );

[0052] In the formula, represents the adjacency matrix of an undirected graph, represents the initial features of the nodes of the undirected graph, represents the representation vector of the learning node, Θ GNN Represents the set of trainable parameters in a graph neural network.

[0053] S302, initializing the initial features of each node of the graph neural network;

[0054] Generally, a random strategy is used to initialize the initial features of each node in the graph neural network.

[0055] 303, update the features of all nodes of the graph neural network to obtain updated features of all nodes of the graph neural network.

[0056] As is known, the graph neural network is multi-layered, so the method for updating the features of the nodes of the graph neural network is designed as follows:

[0057] For any node in any layer of the graph neural network, the graph neural network first aggregates the updated features of the nodes directly connected to the node in the previous layer to obtain the neighbor features of the node, and then combines the neighbor features of the node with the initial features of the node itself to obtain the updated features of the node;

[0058] Traverse all nodes of the graph neural network and obtain the updated features of all nodes of the graph neural network.

[0059] That is to say, if node v is located in the lth layer of the graph neural network, then the feature update process of node v can be expressed as:

[0060]

[0061] In the formula, Represents the neighbor characteristics of node v; Aggregate l represents the aggregation function at level l; represents the representation vector of node u in the l-1 layer; Neig(v) represents the set of nodes directly connected to node v in the l-1 layer; represents the representation vector of node v at layer l; z v Represents the initial features of node v; Combine l Represents the l-th layer combination function.

[0062] S304, initialize a virtual node connected to all nodes of the graph neural network.

[0063] It should be noted that the connection between the virtual node and the nodes of the graph neural network is unidirectional. If the virtual node is represented by s, then Neig(s) represents all the nodes of the graph neural network.

[0064] S305, adding or merging the updated features of all nodes of the graph neural network as the feature representation of the virtual node, and then simulating the information of I after dimensionality reduction;

[0065] Since the virtual node is connected to all nodes of the graph neural network, the updated features of all nodes of the graph neural network can be added or merged using feature processing methods such as weighted average function to obtain a feature representation that reflects the updated features of all nodes of the graph neural network - which is used as the feature representation of the virtual node. And since the nodes of the graph neural network capture the structural information and semantic information corresponding to the nodes of I, the virtual node can simulate the information of I after dimensionality reduction.

[0066] Step 4: Use the multiple virtual nodes in step 3 as the state space and use the DQN algorithm to attack, explore, and train the agent.

[0067] The process of the DQN algorithm can be summarized as follows: let the agent try to perform actions in the state space, and calculate the reward corresponding to the action based on the state after the action is executed; the agent keeps trying and finds the action corresponding to the maximum reward; at the same time, in the process of trying, experience is added to the experience pool; in addition, an experience replay mechanism is introduced to extract experience from the experience pool for training.

[0068] Then, when the DQN algorithm is applied in this method, each action performed by the agent is equivalent to an attack. Then, in each round of attack exploration and training learning, the agent first adds the experience generated during the attack exploration to the experience pool, and then randomly extracts part of the experience from the experience pool for training and learning.

[0069] Specifically, the action performed by the agent includes: for two nodes at both ends of a certain edge, one of the nodes is selected to be hidden. In this way, the action of the agent is simplified from selecting a certain edge to selecting a certain node, which reduces the complexity and helps to speed up the processing progress of DQN.

[0070] For a certain action, the worse the state of the state space is after it is executed, the greater the reward corresponding to the action. In other words, if the stability of the virtual node is worse after an action is executed, it means that the node hidden by this action is the key node.

[0071] It should be noted that the agent's initial attack strategy is random attack, which makes it easier for DQN to start the entire process; during attack exploration, the agent uses the ε-greedy strategy to select actions, which can accelerate the learning process of the agent. As DQN progresses, the agent gradually learns a better attack strategy - that is, it has the ability to find a series of key nodes.

[0072] Step 5: Process the target network with the trained intelligent agent to obtain the key nodes that need to be hidden in the target network.

[0073] Since the agent has acquired the ability to find a series of key nodes after step 4, it can be applied to the target network to find the key nodes that need to be hidden in the target network.

[0074] Simulation comparison

[0075] This embodiment 1 compares the above-mentioned DQN-based suppression graph disassembly method (abbreviated as BDSAGE) with the existing graph adversarial attack algorithm.

[0076] Among them, the existing graph adversarial attack algorithms include: 1. Random Attack method (RA for short); 2. Highest Product of Degree Adaptive method (HPDA for short); 3. Highest Product of Collective Influence method (HPCI for short).

[0077] Four datasets (including dolphins, polbooks, crime, maayan-faa) were selected for processing using the above four methods, and the attack ratio was set to 0.3; three ANG indicators (including HD, CI, HAD) and a comprehensive indicator SUM were compared respectively.

[0078] Among them, SUM = ΔANC(G,G',HD)+ΔANC(G,G',CI)+ΔANC(G,G',HDA); where ΔANC(.) represents the difference in ANC index before and after the graph attack; G represents the original graph sample; G' represents the adversarial sample.

[0079] The comparison results of the four methods are shown in Table 1.

[0080] Table 1 Performance comparison of four methods

[0081]

[0082]

[0083] The larger the value of the performance index is, the better the effect of the method is. As can be seen from Table 1, the DQN-based suppression graph decomposition method proposed in Example 1 is superior to the other three methods in various performance indicators, indicating that it has excellent use effect.

[0084] Example 2

[0085] This embodiment 2 discloses a system for DQN-based suppression graph decomposition, which uses the DQN-based suppression graph decomposition method of embodiment 1.

[0086] See also Figure 3 The DQN-based system for inhibiting graph disassembly includes: a graph generation module, a graph representation module, a graph simulation module, an agent training module, and a graph disassembly inhibition module.

[0087] The graph generation module is configured to generate multiple small-scale synthetic graphs using a random graph generation model. The graph representation module is configured to represent multiple small-scale synthetic graphs as undirected graph structures to obtain multiple undirected graphs. The graph simulation module is configured to use a graph neural network to reduce the dimensions of multiple undirected graphs and obtain multiple virtual nodes that can simulate the information of the reduced-dimensional undirected graphs. The agent training module is configured to use multiple virtual nodes as state spaces and use the DQN algorithm to attack, explore, and train the agent. The graph disassembly and suppression module is configured to process the target network with the trained agent to obtain the key nodes that need to be hidden in the target network.

[0088] Since the present DQN-based suppression graph decomposition system uses the DQN-based suppression graph decomposition method in Example 1, it also has the same effect as Example 1, which will not be repeated here.

[0089] Example 3

[0090] This embodiment 3 discloses a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method for suppressing graph decomposition based on DQN disclosed in embodiment 1 are implemented.

[0091] This embodiment 3 also discloses a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the method for DQN-based suppression graph decomposition disclosed in embodiment 1 are executed.

[0092] This embodiment 3 also discloses a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method for suppressing graph decomposition based on DQN disclosed in embodiment 1 are implemented.

[0093] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for suppressing graph disassembly based on DQN, characterized in that: The following steps are involved: Step 1: Generate multiple small-scale synthetic images using a random graph generation model; Step 2: Representing the multiple small-scale synthetic graphs in step 1 into undirected graph structures to obtain multiple undirected graphs; wherein the number of the undirected graphs is the same as that of the small-scale synthetic graphs and they correspond one to one; Step 3, using a graph neural network to reduce the dimensions of the multiple undirected graphs in step 2, and obtain multiple virtual nodes that can simulate the information of the undirected graphs after the dimension reduction; wherein the number of virtual nodes is the same as that of the undirected graphs, and they correspond one to one; Step 4: Use the multiple virtual nodes in step 3 as the state space and use the DQN algorithm to attack, explore, and train the agent. The actions performed by the agent include: for two nodes at both ends of an edge, one of the nodes is selected to be hidden; For a certain action, the worse the state of the state space is after it is executed, the greater the reward corresponding to the action; Step 5: Process the target network with the trained intelligent agent to obtain the key nodes that need to be hidden in the target network.

2. The method for suppressing graph decomposition based on DQN according to claim 1, characterized in that: In step 1, the random graph generation model includes but is not limited to: ER model and WS model.

3. The method for suppressing graph decomposition based on DQN according to claim 1, characterized in that: In step 2, the mathematical expression of the undirected graph is: G = G (V, E); In the formula, G represents an undirected graph; G(V,E) represents the graph structure composed of V and E; V represents the set of all nodes in G; V = {v1,v2,…,v N }; v1~v N Represents the N nodes in V; E represents the set of all undirected edges in G; E = {e1, e2, …, e M }; e1~e M Represents the M undirected edges in E.

4. The method for suppressing graph decomposition based on DQN according to claim 2, characterized in that: Step three includes: S301, mapping I to a graph neural network so that each node of I is mapped to a low-dimensional vector space to achieve dimensionality reduction of I; S302, initializing the initial features of each node of the graph neural network; S303, updating the features of all nodes of the graph neural network to obtain updated features of all nodes of the graph neural network; S304, initializing a virtual node connected to all nodes of the graph neural network; S305, adding or merging the updated features of all nodes of the graph neural network as the feature representation of the virtual node, and then simulating the information of I after dimensionality reduction; Where I represents any undirected graph in step 2.

5. The method for suppressing graph decomposition based on DQN according to claim 4, characterized in that: S303 includes: For any node in any layer of the graph neural network, the graph neural network first aggregates the updated features of the nodes directly connected to the node in the previous layer to obtain the neighbor features of the node, and then combines the neighbor features of the node with the initial features of the node itself to obtain the updated features of the node; Traverse all nodes of the graph neural network and obtain the updated features of all nodes of the graph neural network.

6. The method for suppressing graph decomposition based on DQN according to claim 1, characterized in that: In step 4, during each round of attack exploration and training learning, the agent first adds the experience generated during the attack exploration to the experience pool, and then randomly extracts part of the experience from the experience pool for training learning.

7. The method for suppressing graph decomposition based on DQN according to claim 6, characterized in that: In step 4, the agent's initial attack strategy is random attack.

8. The method for suppressing graph decomposition based on DQN according to claim 6, characterized in that: In step 4, during attack exploration, the agent uses the ε-greedy strategy to select actions.

9. A system for suppressing graph disassembly based on DQN, characterized in that: It uses the DQN-based suppression graph disassembly method as described in any one of claims 1-8; The DQN-based suppression graph disassembly system includes: A graph generation module, which is used to generate multiple small-scale synthetic graphs using a random graph generation model; A graph representation module, which is used to represent multiple small-scale synthetic graphs into undirected graph structures to obtain multiple undirected graphs; A graph simulation module, which is used to reduce the dimensions of multiple undirected graphs using a graph neural network and obtain multiple virtual nodes that can simulate the information of the undirected graphs after the dimension reduction; The agent training module is used to use multiple virtual nodes as state spaces and use the DQN algorithm to conduct attack exploration and training learning on the agent; as well as The graph disassembly suppression module is used to process the target network with the trained intelligent agent to obtain the key nodes that need to be hidden in the target network.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for suppressing graph decomposition based on DQN as claimed in any one of claims 1 to 8 are implemented.