Black box graph neural network injection attack method based on reinforcement learning

Abstract

The invention belongs to the technical field of artificial intelligence and information security, and particularly relates to a black-box graph neural network injection attack method based on reinforcement learning, which comprises the following steps: firstly, constructing a black-box multilateral graph injection attack framework guided by reinforcement learning, and the framework comprises a double-track feature generator and a multilateral connection selector; and then modeling the whole injection process as a Markov decision process, and optimizing an attack strategy under a limited query budget through an actuator-evaluator algorithm and a composite reward mechanism, thereby realizing maximum misleading of target node classification on the premise of not performing any modification on an original graph, and improving the classification accuracy of the target node. According to the method, a reinforcement learning modeling injection process is a Markov decision process, an attack strategy is directly optimized based on query feedback, dependence on a gradient or proxy model of a target model is not needed, and the problem of unreliable attack guidance under black box setting is effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information security technology, specifically to a black-box graph neural network injection attack method based on reinforcement learning. Background Technology

[0002] With the development of artificial intelligence, Graph Neural Networks (GNNs) have been widely applied in fields such as Industrial Control Systems (ICS) and Industrial Internet of Things (IIoT) due to their ability to effectively model complex relationships between nodes. These GNNs are used for equipment status identification, communication topology modeling, and process data analysis. However, GNNs commonly exhibit adversarial vulnerabilities in practical applications. This means that when the structure or features of the input graph are maliciously perturbed, its classification or prediction results can be easily misled. This vulnerability can lead to misjudgments of system status in safety-critical scenarios, thereby triggering serious security risks.

[0003] In existing technologies, research on adversarial attacks against graph neural networks mainly includes two categories: graph modification attacks (GMA) and graph injection attacks (GIA).

[0004] (1) Graph modification attack methods mislead model predictions by directly modifying the edge structure or node features of the original graph (such as adding or deleting edges, changing node attributes, etc.). For example, Nettack and Metatack methods use gradient or meta-learning strategies to optimize the location and magnitude of perturbations, which can significantly reduce model accuracy under white-box conditions. However, such methods generally assume that the attacker has the authority to modify the original graph data, which is often difficult to implement in controlled systems or industrial environments with strict audits, thus limiting their application scenarios.

[0005] (2) Graph injection attack methods follow the constraint of "addition only," that is, without changing the original graph structure or node features, the attack is achieved by injecting fake nodes and their connecting edges into the graph. These methods have higher practical feasibility. Typical methods include TDGIA, G-NIA, GANI, and LPGIA, which respectively achieve pseudo-node construction and connection optimization through strategies such as feature smoothing, single node generation, or label propagation.

[0006] However, existing graph injection attack methods still have the following main problems and shortcomings:

[0007] (1) Reliance on gradients or surrogate models leads to poor black-box adaptability. Many methods require access to the parameters or gradient information of the target model, or rely on structurally similar surrogate models to guide the attack. When the target system is a black box or the surrogate model does not match the real model, the attack guidance is significantly biased, resulting in a decrease in the success rate.

[0008] (2) Edge-by-edge decision-making is inefficient and difficult to meet query budget constraints. Existing methods mostly adopt single-step or sequential edge insertion strategies. When the number of queries is limited, they cannot quickly coordinate the joint effect of multiple edges, thus limiting the attack effect.

[0009] (3) The generation of pseudo-node features lacks distribution consistency constraints. Some methods fail to ensure that the generated features are consistent with the feature distribution of the original image, resulting in pseudo-nodes being too abnormal in the feature space, making them easy to be identified or filtered by defense mechanisms, thereby reducing the stealth and effectiveness of the attack.

[0010] The main reasons for the above problems are: (1) Gradient-dependent optimization methods cannot be directly applied in real industrial black-box environments; (2) Greedy edge-by-edge insertion decisions ignore the joint correlation between edges and are prone to falling into suboptimal solutions; (3) The lack of distributed modeling of feature generation methods destroys the structure and attribute balance of graph data.

[0011] Therefore, existing technologies lack a graph injection attack method that can simultaneously achieve multilateral joint planning and distribution consistency feature generation under black-box conditions, without accessing gradient information. To address these shortcomings, this invention proposes a black-box graph neural network injection attack method based on reinforcement learning, aiming to improve attack success rate, stealth, and robustness under limited query budget. Summary of the Invention

[0012] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0013] The black-box graph neural network injection attack method based on reinforcement learning includes the following specific steps:

[0014] S1: Construct a reinforcement learning-guided black-box polygon graph injection attack framework, which includes a dual-track feature generator and a polygon connection selector;

[0015] S2. Generate pseudo-node features that are consistent with the feature distribution of the original graph using a dual-track feature generator;

[0016] S3. Use a multi-edge connection selector to jointly select multiple complementary edges from a compact set of candidate nodes and connect them.

[0017] S4. The entire injection process is modeled as a Markov decision process, and the attack strategy is optimized under a limited query budget through an executor-judge algorithm and a compound reward mechanism, thereby maximizing the misleading of the target node classification without making any modifications to the original graph.

[0018] As a preferred embodiment of the black-box graph neural network injection attack method based on reinforcement learning described in this invention, the specific steps of the dual-track feature generator are as follows:

[0019] S21, Contextual semantic extraction: First, extract the target node. of Jumping board Then, local descriptors are obtained through graph convolutional networks and multi-head attention mechanisms. and global pooling vector And after concatenation, a latent vector is generated. :

[0020] ;

[0021] S22, Semantic Shift Compensation: Calculate the latent vector With Global Reference Center The cosine distance is used as the drift compensation coefficient. Used to enhance perturbations in sensitive directions:

[0022]

[0023] in, ;

[0024] S23, Discrete Feature Generation: Discrete feature vectors are generated using biased Gumbel-Softmax sampling. Each dimension :

[0025]

[0026] And through cosine regularization loss Ensure alignment with the original distribution;

[0027] S24, Continuous Feature Generation: First, a normalized flow technique is employed, through a series of coupling layers... latent vectors Mapping to continuous features :

[0028]

[0029] Next, distribution alignment and volume balance are achieved by minimizing the negative log-likelihood:

[0030] .

[0031] As a preferred embodiment of the black-box graph neural network injection attack method based on reinforcement learning described in this invention, the specific steps of the multi-side connection selector are as follows:

[0032] S31, Community Affiliation Determination: Nodes are classified into communities using the Louvain algorithm. and for the injected node Choose the best community :

[0033]

[0034] S32, Core Candidate Set Construction: In the Target Community Within, calculate the core score of the node and select the Top- Nodes:

[0035]

[0036] in This is a normalization of degree centrality. Local clustering coefficients;

[0037] S33, Bridge Candidate Set Construction: Based on Betweenness Centrality Filter Top- Each bridging node, together with the core set, forms the final candidate set. ;

[0038] S34, Multilateral Joint Selection: First, for each candidate node... Constructing compatibility feature vectors And calculate the edge selection probability through learnable projection:

[0039]

[0040] Next, use Gumbel-Top- Sampling strategy joint selection edge set :

[0041] .

[0042] As a preferred embodiment of the black-box graph neural network injection attack method based on reinforcement learning described in this invention, the specific steps of S4 are as follows:

[0043] S41, State Representation: Define the injection process state as follows:

[0044]

[0045] in, This is the subgraph of the target node. For candidate node descriptors, For the remaining budget;

[0046] S42, Policy Execution and Edge Selection: The executor determines the state... Output edge selection distribution And using a noisy Top- Sampling joint selection of edge subset ;

[0047] S43, Composite Reward Construction: After each round of edge connection, calculate the composite reward:

[0048]

[0049] in, For cross-entropy loss, For policy entropy, Confidence-based supplementary rewards;

[0050] S44, Strategy Optimization: First, estimate the state value:

[0051]

[0052] Next, calculate the advantage function. This allows the executor to be updated using a gain-weighted policy gradient:

[0053]

[0054] The final objective function integrates the policy gradient and the feature generation loss:

[0055] .

[0056] Compared with existing technologies:

[0057] 1. Strong gradient-free black-box attack capability: This invention uses reinforcement learning to model the injection process as a Markov decision process, and directly optimizes the attack strategy based on query feedback. It does not rely on the gradient of the target model or the proxy model, effectively solving the problem of unreliable attack guidance under black-box settings.

[0058] 2. Multilateral joint connection mechanism to improve query efficiency and attack effectiveness: This invention replaces the traditional edge-by-edge greedy strategy by constructing a compact candidate set and implementing a multilateral joint selection mechanism, thereby achieving rapid coordination of multiple complementary edges under strict query budget and significantly improving the perturbation efficiency of the target node.

[0059] 3. Consistent feature generation enhances concealment and adaptability: This invention uses a discrete-continuous dual-track strategy to generate pseudo-node features. Through techniques such as normalized flow and cosine regularization, it ensures that the generated features are aligned with the distribution of the original graph, effectively avoiding detection risks caused by feature deviation.

[0060] 4. Strong robustness across models and defense mechanisms: This invention maintains a high attack success rate under various graph neural network backbones (such as GCN, GAT, APPNP) and defense enhancement models (such as GNN-Jaccard, Pro-GNN, GNNGuard), demonstrating excellent transferability and defense adaptability. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the framework of the present invention;

[0062] Figure 2 This is a comparison chart of the attack success rates of the undefended GCN model and three defense-enhanced models on four benchmark datasets using the BMNI method of this invention.

[0063] Figure 3 The four datasets of this invention are in different , Attack rate comparison chart under different values; where sub-charts (a)-(d) respectively correspond to The horizontal axis represents The vertical axis represents the attack success rate (ASR), and it remains constant.

[0064] Figure 4 This is a statistical information graph of the dataset used in the experiments of this invention;

[0065] Figure 5 The graph shows the attack performance (%) of different methods of this invention on various datasets;

[0066] Figure 6 The graph shows the attack performance (accuracy %) of this invention on different GNN backbone networks;

[0067] Figure 7 This is an attack rate graph showing the removal rate of a specific module in a GCN using the attack method described in this invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0069] This invention provides a black-box graph neural network injection attack method based on reinforcement learning. Please refer to [link to relevant documentation]. Figures 1-7 The specific steps are as follows:

[0070] S1: Construct a reinforcement learning-guided black-box multi-edge graph injection attack framework, which includes a dual-strand feature generator (DSFG) and a multi-edge link selector (MELS).

[0071] S2. Generate pseudo-node features that are consistent with the feature distribution of the original graph using a dual-track feature generator;

[0072] S3. Use a multi-edge connection selector to jointly select multiple complementary edges from a compact set of candidate nodes and connect them.

[0073] S4. The entire injection process is modeled as a Markov decision process, and the attack strategy is optimized under a limited query budget through the executor-judge (A2C) algorithm and a compound reward mechanism, thereby maximizing the misleading of the target node classification without making any modifications to the original graph.

[0074] based on Figure 1 In this invention, a clean image is given. The target model is a pure black-box graph neural network classifier. Its query budget does not exceed For a single target node Inject pseudo-nodes And jointly plan at most k edges to maximize the target node. In the classifier The framework comprises two core components: a dual-track feature generator (DSFG) and a multilateral connection selector (MELS). Specifically, DSFG extracts the K-hop contextual semantics of the target node and uses a discrete-continuous dual-track strategy to generate pseudo-node features that conform to the original distribution characteristics. MELS constructs a compact candidate set C based on community coreness and bridging connectivity. Within this set, multiple complementary edges are jointly selected to connect pseudo-nodes, avoiding redundancy caused by edge connections. The evaluator calculates the composite reward through black-box queries, while the Advantage Executor-Judge (A2C) algorithm calculates the reward within the budget. The update strategy under constraints forms a closed-loop process of feature generation and multilateral joint connection.

[0075] Overall, under pure query conditions, the executor consists of DSFG and MELS. This component injects a single pseudo-node into the clean graph in a single operation. The policy is updated by constructing a reward based on the predicted score. DSFG (Dual-Track Feature Generator): Based on... Exploring the context, a discrete-continuous dual-track strategy is employed to generate pseudo-node features consistent with the original distribution. MELS (Multi-sided Join Selector): Constructs a candidate set based on community segmentation and core / bridging scores, then applies softmax to select the frontier linker within the candidate set. A number of complementary neighbor nodes are used to achieve joint multilateral connections.

[0076] The specific steps of the dual-track feature generator are as follows:

[0077] To generate pseudo-node features that are consistent with the feature distribution of the original graph and have sufficient perturbation strength, a dual-track feature generation strategy is designed, which is applicable to both discrete and continuous feature spaces.

[0078] S21, Contextual semantic extraction: First, extract the target node. of Jumping board Then, local descriptors are obtained through graph convolutional networks and multi-head attention mechanisms. and global pooling vector And after concatenation, a latent vector is generated. :

[0079] ;

[0080] S22, Semantic Shift Compensation: Calculate the latent vector With Global Reference Center The cosine distance is used as the drift compensation coefficient. Used to enhance perturbations in sensitive directions:

[0081]

[0082] in, ;

[0083] S23, Discrete Feature Generation: Discrete feature vectors are generated using biased Gumbel-Softmax sampling. Each dimension :

[0084]

[0085] And through cosine regularization loss Ensure alignment with the original distribution;

[0086] S24, Continuous Feature Generation: First, a normalized flow technique is employed, through a series of coupling layers... latent vectors Mapping to continuous features :

[0087]

[0088] Next, distribution alignment and volume balance are achieved by minimizing the negative log-likelihood:

[0089] .

[0090] The specific steps of the multilateral connection selector are as follows:

[0091] To achieve rapid and coordinated multilateral connections under strict query budgets, we designed a multilateral joint selection mechanism based on community hub-bridge scoring;

[0092] S31, Community Affiliation Determination: Nodes are classified into communities using the Louvain algorithm. and for the injected node Choose the best community :

[0093]

[0094] S32, Core Candidate Set Construction: In the Target Community Within, calculate the core score of the node and select the Top- Nodes:

[0095]

[0096] in This is a normalization of degree centrality. Local clustering coefficients;

[0097] S33, Bridge Candidate Set Construction: Based on Betweenness Centrality Filter Top- Each bridging node, together with the core set, forms the final candidate set. ;

[0098] S34, Multilateral Joint Selection: First, for each candidate node... Constructing compatibility feature vectors And calculate the edge selection probability through learnable projection:

[0099]

[0100] Next, use Gumbel-Top- Sampling strategy joint selection edge set :

[0101] .

[0102] The specific steps of S4 are as follows:

[0103] S41, State Representation: Define the injection process state as follows:

[0104]

[0105] in, This is the subgraph of the target node. For candidate node descriptors, For the remaining budget;

[0106] S42, Policy Execution and Edge Selection: The executor determines the state... Output edge selection distribution And using a noisy Top- Sampling joint selection of edge subset ;

[0107] S43, Composite Reward Construction: After each round of edge connection, calculate the composite reward:

[0108]

[0109] in, For cross-entropy loss, For policy entropy, Confidence-based supplementary rewards;

[0110] S44, Strategy Optimization: First, estimate the state value:

[0111]

[0112] Next, calculate the advantage function. This allows the executor to be updated using a gain-weighted policy gradient:

[0113]

[0114] The final objective function integrates the policy gradient and the feature generation loss:

[0115] .

[0116] Through the above mechanism, RBM-GIA (Reinforcement Learning-guided Black-box Multi-edge Graph Injection Attack) can efficiently generate consistent adversarial node features under black-box, insertion-only constraints, and achieve fast and coordinated multi-edge connections, thereby achieving higher average attack success rate and query efficiency under various target models and defense configurations.

[0117] Based on the above, the present invention provides an embodiment including but not limited to the following:

[0118] Experiment setup and dataset selection

[0119] Experiments were conducted on eight widely recognized benchmark datasets, covering both discrete and continuous feature spaces. Specific dataset statistics are as follows: Figure 4 As shown.

[0120] Baseline Methodology and Evaluation Metrics

[0121] To comprehensively evaluate the effectiveness of the BMNI method, nine representative methods were selected as comparison baselines: NIPA, Node+NIPA, Rand+Nettack, Node+Nettack, AFGSM, TDGIA, G-NIA, G²A2C, GANI, and LPGIA. All methods were evaluated under the same node, edge, and query budget constraints, with attack success rate (ASR) as the primary evaluation metric.

[0122] BMNI parameter configuration

[0123] In all experiments, the parameters of BMNI were configured as follows:

[0124] Node generator configuration:

[0125] Bullish attention head count: =8

[0126] Cosine similarity loss weights:

[0127] Discrete characteristic Gumbel-Softmax temperature:

[0128] Continuous characteristic temperature parameters:

[0129] Feature regularization weights:

[0130] Side sampler configuration:

[0131] Number of convolutional layers in the graph:

[0132] PageRank random teleportation probability:

[0133] PageRank iteration count:

[0134] General training settings:

[0135] Hidden Dimensions:

[0136] Learning rate:

[0137] Number of repeated experiments:

[0138] Early shutdown mechanism:

[0139] Attack performance assessment

[0140] Under a black-box setup, the injection is restricted to a single adversarial node, but the establishment of two connecting edges is allowed. The specific process is as follows:

[0141] Sample ten candidate neighbors from the target node's neighborhood;

[0142] The optimal two edges are jointly selected from the candidate set using an executor-evaluator strategy.

[0143] Update the graph structure and query the target model to calculate the attack success rate.

[0144] BMNI achieved the best or tied-best attack success rates on all eight datasets, with representative results as follows:

[0145] Citeseer: 81.7% (compared to the strongest baseline of 59.5%)

[0146] Cora: 72.5% (compared to 58.7%)

[0147] Wiki-CS: 92.1% (compared to 87.2%)

[0148] PubMed: 81.8% (compared to 75.3%)

[0149] Cross-architecture migration assessment

[0150] The target model was replaced with SGC, GAT, and APPNP, and the transfer performance of BMNI was evaluated on the Cora, Citeseer, Pubmed, and Wiki-CS datasets. BMNI achieved the highest attack success rate across all 12 backbone network-dataset pairings, for example:

[0151] Under the SGC architecture, Cora: 67.8%, Citeseer: 78.9%.

[0152] Under the GAT architecture, PubMed: 86.6%, Wiki-CS: 95.5%.

[0153] Attack performance under the defense enhancement model

[0154] BMNI's performance was evaluated on four datasets targeting four classes of models: original GCN, GNN-Jaccard, Pro-GNN, and GNNGuard. BMNI maintained a high attack success rate (≥71%) across all model-dataset combinations and achieved or surpassed the attack performance against original GCN in multiple cases.

[0155] Candidate set size With the number of planned edges Impact Analysis

[0156] Under a fixed query budget, system tests showed different results. The overall performance trend of the combination is as follows:

[0157] fixed At that time, the attack success rate increased. The increase in the value initially shows a rise followed by a fall or a tendency to stabilize; the optimal value usually appears at [a specific point in time]. Place;

[0158] fixed hour, Increasing the score from 5 to 10 / 15 can significantly improve the attack success rate. The performance gain may saturate or even decrease.

[0159] BMNI's "optimal performance point" is generally located at Interval.

[0160] Ablation experiment

[0161] Three ablation studies were conducted on BMNI:

[0162] BMNI-FN: Remove the feature generation module;

[0163] BMNI-NoSel: Removes the candidate node selection mechanism;

[0164] BMNI-Seq: Replaces the "single-time multi-sided joint join" strategy with a sequential greedy join strategy.

[0165] The full version of BMNI significantly outperformed its three variants across all datasets, validating the irreplaceable role of each component in improving attack success rates.

[0166] In summary, compared with existing graph injection attack methods, the reinforcement learning-guided black-box multi-sided graph injection attack framework (RBM-GIA) proposed in this invention shows significant advantages under multiple benchmark datasets and model settings. The specific effects and advantages are as follows:

[0167] On the Citeseer dataset, the success rate of G²A2C attacks is [missing information]. The present invention achieves ,promote On the Cora dataset, G²A2C is... The present invention is ,promote On the PubMed dataset, G²A2C is... The present invention is ,promote On the Wiki-CS dataset, G²A2C is... The present invention is ,promote .

[0168] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for black-box graph neural network injection attack based on reinforcement learning, characterized in that, The specific steps include the following: S1: Construct a reinforcement learning guided black box multi-edge graph injection attack framework, which includes a double-track feature generator and a multi-edge connection selector; S2. Generate pseudo node features consistent with the original graph feature distribution through the double-track feature generator; S3. Jointly select multiple complementary edges for connection in the compact candidate node set through the multi-edge connection selector; S4. Model the entire injection process as a Markov decision process, and optimize the attack strategy under the limited query budget through the executor-evaluator algorithm and the composite reward mechanism, so as to maximize the misleading of the target node classification without any modification to the original graph.

2. The method of claim 1, wherein the method is based on a black-box graph neural network injection attack using reinforcement learning. The specific steps of the double-track feature generator are as follows: S21, context semantic extraction: first extract the target node of the skip graph , then get its local descriptor and global pooling vector through graph convolution network and multi-head attention mechanism , and generate latent vector after splicing : ； S22, semantic drift compensation: compute latent vector cosine distance to global reference center as drift compensation factor for enhancing perturbation in sensitive directions: ； wherein ; S23, discrete feature generation: generate discrete feature vector using biased Gumbel-Softmax sampling of each dimension : ； and by cosine regularization loss Ensuring alignment with the original distribution; S24, continuous feature generation: first, a standardization flow technique is adopted to map the latent vectors into continuous features through a series of coupled layers ​ ； Then, distribution alignment and volume balance are achieved by minimizing negative log-likelihood: 。 3. The method of claim 1, wherein, The specific steps of the multi-edge connection selector are as follows: S31, community membership determination: nodes are divided into communities by Louvain algorithm , and the injection node is selected as the optimal community : ； S32, core candidate set construction: in the target community calculate the core scores of the nodes and select the top- nodes: ； wherein is a normalization of the degree centrality, is a local clustering coefficient; S33, bridge candidate set construction: based on intermediate centrality Filtering Top- bridge nodes, and merging with the core set to form the final candidate set ; S34, multi-edge joint selection: first for each candidate node Constructing compatibility feature vector and compute edge selection probability by learnable projection: ； Next, the Gumbel-Top- Sampling strategy jointly selects edge sets : 。 4. The method of claim 1, wherein, The specific steps of S4 are as follows: S41, state representation: define the injection process state as: ； wherein, is a subgraph of target nodes, is a candidate node descriptor, is a remaining budget; S42, policy execution and edge selection: the executor executes the policy based on the state output edge selection distribution and employs a noisy Top- sampled joint selection of edge subsets ; S43, composite reward construction: after each round of edge connection, calculate the composite reward: ； wherein, is the cross-entropy loss, is the policy entropy, is the confidence-based auxiliary reward; S44, strategy optimization: first estimate the state value: ； The advantage function is then computed such that the actor is updated by advantage-weighted policy gradients: ； The final objective function integrates the policy gradient and feature generation loss: 。

Images