A deep map neural network training method based on topological structure disturbance

Abstract

The application relates to a kind of deep map neural network training methods based on topology structure disturbance, belong to data science and artificial intelligence technical field.The method steps are as follows:(1) aggregation multi-order neighbor: the receptive field of each node is expanded;(2) implement node level disturbance: traverse all nodes in topology structure, and decide whether to disturb each node with probability p, so as to introduce randomness in node set;(3) implement edge level disturbance: in the adjacency matrix of k-hop topology structure of node, a certain ratio of edge is randomly removed;(4) multiple running and result merging.The application can effectively solve the oversmoothing problem in graph neural network, and when capturing topology graph structure, the nodes and edges in the graph are randomly disturbed using the topology disturbance idea, different local topology graph structures are generated, so that different depth graph structure information is learned, and the feature difference between nodes is expanded.

Description

Technical Field

[0001] This invention relates to a deep graph neural network training method based on topological perturbation, belonging to the fields of data science and artificial intelligence technology. Background Technology

[0002] With the advent of the big data era, machine learning and graph neural networks (GNNs) have demonstrated unprecedented potential in the field of complex data processing and analysis. Graph data, as a data structure widely existing in the real world, is characterized by complex connections between nodes, offering the possibility of revealing deeper information hidden behind the data. However, despite the significant progress made by graph neural networks in processing graph-structured data, the oversmoothing phenomenon encountered during deep stacking and the limitations of aggregating neighborhood information have become key factors restricting further performance improvements.

[0003] Oversmoothing refers to the phenomenon that, during the multi-layer iteration of a graph neural network, as nodes continuously aggregate information from their neighbors, especially when using a simple weighted summation method for aggregation, the differences between nodes gradually decrease, eventually tending towards similar representations. This makes it difficult for the model to distinguish different nodes, thus affecting the accuracy of downstream tasks such as classification and prediction. This challenge severely limits the effectiveness and generalization ability of graph neural networks when processing deep graph structure data.

[0004] To address this challenge, scholars both domestically and internationally have conducted extensive and in-depth research, exploring various strategies to alleviate the oversmoothing problem. Among these, random dropout methods, as an effective technique, achieve a similar effect to data augmentation by randomly reducing the amount of information used during model training, thus mitigating the oversmoothing phenomenon to some extent. These methods include, but are not limited to, Dropout, DropNode, DropEdge, and DropMessage, each of which perturbs the graph neural network from different angles to improve the model's robustness and generalization performance.

[0005] However, while existing random dropout methods each have their merits, they also have limitations. For example, Dropout and DropNode focus on directly perturbing the graph structure, ignoring the complexity of information transmission between nodes; while DropMessage, although acting directly on the message matrix, fails to fully consider the impact of the graph's topology on information flow. Therefore, this invention proposes an innovative random dropout strategy that deeply integrates an understanding of graph topology, aiming to fundamentally solve the oversmoothing problem in graph neural networks through a more refined and reasonable perturbation method. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a deep graph neural network training method based on topological perturbation. This method effectively solves the oversmoothing problem in graph neural networks (GNNs), namely, the problem that as the number of network layers increases, the differences in node features gradually decrease, leading to a decline in model performance. When capturing the topological graph structure, the invention uses the idea of ​​topological perturbation to randomly perturb the nodes and edges in the graph, generating different local topological graph structures, thereby learning graph structure information at different depths and expanding the feature differences between nodes.

[0007] The technical solution of the present invention is as follows:

[0008] A deep graph neural network training method based on topological perturbation, comprising the following steps:

[0009] (1) Aggregate multi-order neighbors: expand the receptive field of each node, including first-order, second-order and higher-order neighbor nodes;

[0010] (2) Implement node-level perturbation: Traverse all nodes in the topology and decide whether to perturb each node with probability p, thereby introducing randomness into the node set;

[0011] (3) Implement edge-level perturbation: randomly remove a certain percentage of edges from the adjacency matrix of the k-hop topology of the node;

[0012] (4) Multiple runs and result merging: The model is run multiple times, and topology perturbation is implemented during the run to generate diverse topology structures. Then, the results of these different runs are merged to enhance the generalization ability and obtain the final model output.

[0013] Preferably, in step (1), the implementation method for aggregating multi-level neighbors is as follows:

[0014] Let G = (V, E) denote a graph, where V represents the set of nodes and E is the set of edges between nodes. The relationships formed when information is aggregated through different hop numbers k are defined as the node topology of that node, denoted by T. v k This indicates that the topology belongs to graph G;

[0015]

[0016] In the above formula, This represents the hidden representation of node v at level l. This represents the hidden representation of node v at level l-1. This represents the hidden representation of node u at level l-1; It is the set of k-hop topologies near node v; γ(x) is a differentiable function; Λ() represents the aggregation function;

[0017] To obtain more comprehensive and complete aggregated topological information, for each node v, we determine all its neighbors from order 1 to k, thus constructing the topological structure N for each node. d (v) and N s (v), where N s (v) represents the nearest neighbor of node v, through N s (v) can reflect shallow topological information in GNNs, N d (v) represents the higher-order neighbor node of v. The nearest neighbor node of v is called the first-order neighbor node of v, and the node that is more than 1 away from v is called the higher-order neighbor node.

[0018] The node with a distance of 2 and the node with a distance of 3 are called the second-order neighbor and third-order neighbor of node v, respectively, belonging to N. s (v) and N d (v) The nodes of the two sets and the edges connecting them constitute the local topology graph of the current node v. By defining an aggregation function Λ to merge the features of the node and its neighbors, and integrating the features of the node and its multi-level neighbors at each layer, the expression for updating the GNN node in this step is finally expressed as follows after introducing a deep topology structure:

[0019]

[0020] Where ψ and Λ represent the combinatorial function and the aggregate function, respectively. This represents the hidden representation of node v at level l. This represents the hidden representation of node u at level l-1.

[0021] Preferably, in step (2), the implementation method of node-level perturbation is as follows:

[0022] Based on the k-hop topology of node v Analysis reveals that by adding topological perturbations at the node level in this region, the aggregation effect of nodes is affected, thereby mitigating oversmoothing. Specifically, this method sets a perturbation probability p and randomly selects nodes within the k-hop topology of each node for perturbation.

[0023]

[0024] If node v is perturbed at layer l (according to probability p), its feature representation may be set to zero or other perturbation operations.

[0025] Preferably, in step (3), the implementation method of the edge-level perturbation is as follows:

[0026] Choose the longest path from node v to k. A t Represented as The adjacency matrix for A t Randomly removing a certain percentage of edges from the topology is essentially a random forced adjacency matrix A. t Some non-zero elements in the array are zero, and this perturbation is not used during verification and testing.

[0027] Preferably, in step (4), assuming the probability of each node being disturbed is p, let τ = |Γ|, where τ represents the number of nodes in the k-th order neighborhood (excluding node v). The probability of perturbing any node u in Γ can be easily expressed as p·(1-p). τ It is easy to know if Then the probability of node u being disturbed will reach its maximum value. This means that if the number of runs is greater than or equal to e·(τ+1), then node u can be disturbed at least once in these runs. The nodes in the network can form multiple different node topologies. When performing the multi-level neighbor aggregation operation in step (1), the aggregation of different structural information can alleviate over-smoothing and increase the differences between nodes.

[0028] Based on a deep analysis and understanding of graph topology, this invention solves the oversmoothing problem encountered by graph neural networks when processing complex graph structures. By introducing the concept of topological perturbation, it enhances the model's sensitivity to graph structure changes and improves its performance and generalization ability in various graph analysis tasks.

[0029] The beneficial effects of this invention are as follows:

[0030] Traditional graph neural networks still suffer from oversmoothing in different graph scenarios, where different types of nodes are identified as the same type after multiple iterations. To address this oversmoothing problem and inspired by the idea of ​​topological perturbation, this invention proposes a deep graph neural network training method based on topological perturbation. This method utilizes deep topological structure information and applies the idea of ​​topological perturbation. By expanding the local neighborhood of a node, it senses the local topological structure within a multi-hop neighborhood and performs topological perturbation on it. This perturbation includes randomly perturbing the nodes and edges within that region.

[0031] Traditional graph neural networks (GNNs), despite their widespread application in diverse graph structures, still face a significant challenge: oversmoothing. This problem typically manifests after multiple rounds of iterative training, causing the model to incorrectly classify nodes of different categories as belonging to the same category. To address this challenge and drawing upon the advanced concept of topological perturbation, this invention innovatively proposes a novel training strategy—a deep graph neural network method based on topological perturbation. The core of this method lies in deepening the understanding of graph topology and cleverly incorporating topological perturbation. By expanding the perceptual range of nodes, it extends beyond direct neighbors to capture local topological structures within multi-hop neighborhoods at greater distances, thus providing a more comprehensive understanding of complex patterns in the graph. Furthermore, this method applies random perturbations to these local topological structures, including probabilistically adjusting node features and connecting edges within selected regions. This topological perturbation mechanism aims to break the convergence of node features that may occur during iteration in traditional GNNs, effectively mitigating the oversmoothing problem by increasing the model's sensitivity to subtle changes in the graph structure. Furthermore, through this innovative training method, the model can learn more robust and discriminative node representations, thereby demonstrating superior performance in various graph analysis tasks. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating an embodiment of the present invention. Detailed Implementation

[0033] The present invention will be further described below with reference to the embodiments and accompanying drawings, but is not limited thereto.

[0034] Example 1:

[0035] This embodiment provides an architecture for a personalized music recommendation system based on topological perturbation, with the following steps:

[0036] (1) Define the graph structure. In this step, based on a series of behaviors such as listening to songs, collecting songs, and downloading songs on the website, extract the user's short-term and long-term preferences for listening to songs, and construct a music preference graph for each user. The nodes represent users and songs, and the edges represent the interactions between users and songs, such as playing, rating, or collecting.

[0037] (2) Node feature allocation: Assign a feature vector to each node. For user nodes, features include age, gender, historical playlists, etc.; for song nodes, features include genre, artist, album, lyrics, etc.

[0038] (3) Implement node-level perturbation. In this step, according to the set perturbation strategy, some user nodes or song nodes are randomly selected for perturbation. For example, the changes in user preferences can be simulated by randomly setting some users' historical playback records to zero.

[0039] (4) Implement edge-level perturbation. In this step, some edges (i.e. user-song interactions) in the interaction matrix are randomly selected for perturbation. For example, the weight of the edge is set to zero to simulate possible user forgetting or changes in interest.

[0040] (5) Aggregate neighbor information. In this step, for each layer of the GNN, for an undisturbed node, aggregate the information of its neighbor nodes to update the node representation. The aggregation can be a simple average, a weighted average, or a more complex function.

[0041] (6) Training and optimization: In this step, the GNN model is trained using historical interaction data, the model parameters are optimized, and the strength and probability of topological perturbation are adjusted to improve the generalization ability and recommendation quality of the model. Based on the updated node representation, personalized music recommendations are generated through a recommendation module.

[0042] In the steps described above, incorporating topological perturbation is crucial. Identifying different graph topologies and expanding the differences between node features effectively alleviates the oversmoothing problem in graph social networks, preventing discrepancies between nodes. Smoothing is then applied during training, thereby improving the expressive power of the graph neural network.

[0043] Through the above embodiments, the training method of the present invention introduces perturbations into the graph topology to establish an inverse smoothing graph neural network based on topological perturbations, which successfully solves the problem of oversmoothing of graph neural networks in recommendation systems, effectively improves the efficiency and accuracy of recommendation systems, and provides new ideas and methods for further research and application of graph neural networks.

[0044] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for training a deep graph neural network based on topological perturbation, characterized in that, The steps are as follows: (1) Aggregate multi-level neighbors: expand the receptive domain of each node; The specific implementation method is as follows: Define the graph structure. First, based on the user's listening, song collection, and song download behavior on the website, extract the user's short-term and long-term preferences for listening to songs, and construct a music preference graph for each user. Nodes represent users and songs, and edges represent the interaction between users and songs, such as playing, rating, or collecting. Assign a feature vector to each node. For user nodes, the features include age, gender, and historical playlists; for song nodes, the features include genre, artist, album, and lyrics. Let G = (V, E) denote a graph, where V represents the set of nodes and E is the set of edges between nodes. The relationships formed when information is aggregated through different hop numbers k are defined as the node topology of that node. This indicates that the topology belongs to graph G; ； In the above formula, Indicates that node v is at the th Hidden representation of layers, Indicates that node v is at the th The hidden representation of layer -1 Indicates that node u is at the th -1 level hidden representation; It is near node v A set of jump topologies; It is a differentiable function; Represents aggregate functions; For each node v, determine all its neighbors from order 1 to k, and construct the topology of each node. ,in Representing the nearest neighbor of node v, through In GNNs, it reflects shallow topological information. The higher-order neighbor node of v is defined as the nearest neighbor node of v, and the node that is more than 1 away from v is defined as the higher-order neighbor node. The node with a distance of 2 and the node with a distance of 3 are called the second-order neighbor and third-order neighbor of node v, respectively. and The nodes of two sets and the edges connecting them constitute the local topology of the current node v. Define an aggregation function. This involves merging the features of nodes and their neighbors, and integrating the features of nodes and their multi-level neighbors at each layer. Finally, after introducing a deep topological structure, the expression for updating GNN nodes is expressed as: ； in, and These represent combinatorial functions and aggregate functions, respectively. Indicates that node v is at the th Hidden representation of layers, Indicates that node u is at the th Hidden representation of layers; (2) Implement node-level perturbation: Traverse all nodes in the topology and decide whether to perturb each node with probability p, thereby introducing randomness into the node set; The specific implementation method is as follows: According to the set perturbation strategy, some user nodes or song nodes are randomly selected for perturbation, and the perturbation is based on the nodes. of Jump topology The analysis involves setting a perturbation probability p and randomly selecting nodes within the k-hop topology of each node for perturbation: ； If node v is perturbed at layer l, its feature representation is set to zero or other perturbed operations. (3) Implement edge-level perturbation: randomly select user-song interactions for perturbation, and randomly remove a certain percentage of edges from the adjacency matrix of the k-hop topology of the node; (4) Multiple runs and result merging: The model is run multiple times, and topology perturbation is implemented during the run to generate diverse topology structures. Then, the results of these different runs are merged to enhance the generalization ability and obtain the final model output to generate personalized music recommendations.

2. The deep graph neural network training method based on topological perturbation as described in claim 1, characterized in that, In step (3), the implementation method of edge-level perturbation is as follows: Choose the longest path from node v to k. , Represented as The adjacency matrix for Randomly remove a certain percentage of edges from the topology and randomly force the adjacency matrix. Some non-zero elements in the array are zero, and this perturbation is not used during verification and testing.

3. The deep graph neural network training method based on topological perturbation as described in claim 2, characterized in that, In step (4), assuming the probability of each node being disturbed is p, let... =|Γ|, Represented by the number of nodes in the k-th order neighborhood, selected any node The probability of making a perturbation is expressed as: ,if Then the probability of node u being disturbed will reach its maximum value, meaning that if the number of runs is greater than or equal to... This ensures that node u is disturbed at least once in these runs, through perturbation. The nodes in the process form multiple different node topologies. When performing the multi-level neighbor aggregation operation in step (1), the aggregation of different structural information alleviates over-smoothing and increases the differences between nodes.

Images