Depth map neural network training method based on topological structure disturbance

Abstract

The invention relates to a depth map neural network training method based on topological structure disturbance, and belongs to the technical field of data science and artificial intelligence. The method comprises the following steps: (1) aggregating multi-order neighbors: expanding a sensing domain of each node; (2) implementing node level disturbance: traversing all nodes in the topological structure, and determining whether to disturb each node or not according to a probability p, so as to introduce randomness into a node set; (3) edge-level disturbance is implemented, wherein edges with a certain ratio are randomly removed from the adjacent matrix of the k-hop topological structure of the nodes; and (4) multiple times of operation and result merging. According to the method, the over-smoothing problem in the graph neural network can be effectively solved, and when the topological graph structure is captured, the nodes and the edges in the graph are randomly disturbed by using the topological disturbance thought to generate different local topological graph structures, so that graph structure information of different depths is learned, and the feature difference between the nodes is expanded.

Description

Technical Field

[0001] The present invention relates to a deep graph neural network training method based on topological structure perturbation, and belongs to the technical field of data science and artificial intelligence. Background Art

[0002] With the advent of the big data era, machine learning and graph neural networks (GNNs) have shown unprecedented potential in the field of complex data processing and analysis. Graph data, as a data structure widely existing in the real world, is characterized by complex connections between nodes, which makes it possible to reveal the deep information hidden behind the data. However, although graph neural networks have made significant progress in processing graph structured data, the over-smoothing phenomenon encountered in the deep stacking process and the limitations of aggregating neighborhood information have become key factors restricting their performance from being further improved.

[0003] Oversmoothing refers to the phenomenon that, in the multi-layer iteration process of graph neural networks, as nodes continue to aggregate information about their neighbors, especially when a simple weighted sum is used for aggregation, the differences between nodes gradually decrease and eventually tend to similar representations, making it difficult for the model to distinguish different nodes, which in turn affects the accuracy of downstream tasks such as classification and prediction. This challenge severely limits the effectiveness and generalization ability of graph neural networks when processing deep graph structure data.

[0004] In order to meet this challenge, domestic and foreign scholars have conducted extensive and in-depth research and explored various strategies to alleviate the over-smoothing problem. Among them, the random dropout method, as an effective technical means, achieves a similar effect to data enhancement by randomly reducing the amount of information used in the model training process, thereby alleviating the over-smoothing phenomenon to a certain extent. Such methods include but are not limited to Dropout, DropNode, DropEdge, and DropMessage, each of which perturbs the graph neural network from different angles to improve the robustness and generalization performance of the model.

[0005] However, although the existing random drop methods have their own advantages, they also have limitations. For example, Dropout and DropNode focus on direct perturbations to the graph structure, ignoring the complexity of information transmission between nodes; and although DropMessage acts directly on the message matrix, it fails to fully consider the impact of the graph topology on information flow. In view of this, the present invention proposes an innovative random drop strategy that deeply integrates the understanding of the graph topology, aiming to fundamentally solve the over-smoothing problem in graph neural networks through a more sophisticated and reasonable perturbation method. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a deep graph neural network training method based on topological structure perturbation, which can effectively solve the over-smoothing problem in graph neural networks (GNNs), that is, as the number of network layers increases, the node feature differences gradually decrease, resulting in a decrease in model performance. When capturing the topological graph structure, the topological perturbation idea is used to randomly perturb the nodes and edges in the graph to generate different local topological graph structures, thereby learning graph structure information of different depths and expanding the feature differences between nodes.

[0007] The technical solution of the present invention is as follows:

[0008] A deep graph neural network training method based on topological structure perturbation, the steps are as follows:

[0009] (1) Aggregate multi-order neighbors: Expand the receptive field of each node to include first-order, second-order, and higher-order neighbor nodes;

[0010] (2) Implement node-level perturbation: traverse all nodes in the topology and decide whether to perturb each node with probability p, thereby introducing randomness in the node set;

[0011] (3) Implement edge-level perturbation: randomly remove a certain ratio of edges from the adjacency matrix of the k-hop topology of the node;

[0012] (4) Multiple runs and result merging: The model is run multiple times, topological perturbations are implemented during the run, and diverse topological structures are generated. Then, the results of these different runs are merged to enhance the generalization ability and obtain the final model output.

[0013] Preferably, in step (1), the implementation method of aggregating multi-order neighbors is:

[0014] Let G = (V, E) represent a graph, where V represents the set of nodes and E is the set of edges between nodes. The relationship formed when information is aggregated through different hop numbers k is defined as the node topology of the node, denoted by T v k Indicates that the topological structure belongs to graph G;

[0015]

[0016] In the above formula, represents the hidden representation of node v at layer l, represents the hidden representation of node v at layer l-1, represents the hidden representation of node u at layer l-1; is the set of k-hop topological structures around node v; γ(x) is a differentiable function; Λ() represents the aggregation function;

[0017] In order to make the aggregated topological structure information more complete, for each node v, determine all its neighbors from the first to the kth order, and construct the topological structure N of each node d (v) and N s (v), where N s (v) represents the nearest neighbor of node v, through N s (v) Shallow topological information can be reflected in GNN, N d (v) represents the high-order neighbor node of v. The nearest neighbor node of v is called the first-order neighbor node of v, and the node whose distance to v is greater than 1 is called the high-order neighbor node;

[0018] The nodes with a distance of 2 and the nodes with a distance of 3 are called the second-order neighbor nodes and third-order neighbor nodes of node v, respectively, and belong to N s (v) and N d (v) The nodes of the two sets and the edges connecting them constitute the local topological graph of the current node v. An aggregation function Λ is defined to merge the features of the node and its neighbors, and the features of the node and its multi-order neighbors are integrated at each layer. Finally, after introducing the deep topological structure, the expression of the GNN node update in this step is expressed as:

[0019]

[0020] Among them, ψ and Λ represent the combination function and aggregation function respectively, represents the hidden representation of node v at layer l, represents the hidden representation of node u at layer l-1.

[0021] Preferably, in step (2), the implementation method of node level disturbance is:

[0022] According to the k-hop topology of node v We analyze the situation by adding certain topological disturbances at the node level in the region to affect the aggregation effect of the nodes, thereby alleviating oversmoothing. Specifically, this method sets a disturbance probability p and randomly selects nodes for disturbance in the k-hop topological structure of each node:

[0023]

[0024] Here, if node v is perturbed at layer l (according to probability p), its feature representation may be set to zero or other perturbation operations.

[0025] Preferably, in step (3), the implementation method of edge-level perturbation is:

[0026] Choose the longest path from node v to k. A t Expressed as The adjacency matrix of A t , randomly remove a certain ratio of edges in the topology, essentially randomly forcing the adjacency matrix A t Some non-zero elements in are zero, and this perturbation is not used during validation and testing.

[0027] Preferably, in step (4), assuming that the probability of each node being disturbed is p, let τ = |Γ|, where τ represents the number of nodes in the k-th order neighborhood (excluding node v), and the probability of selecting any node u in Γ for disturbance can be easily expressed as p·(1-p) τ , it is easy to know if Then the probability of node u being disturbed will reach the maximum value, which means that if the number of runs is greater than or equal to e·(τ+1), it can be achieved that the number of times node u is disturbed is at least 1 in these runs. The nodes in the network can form multiple different node topological structures. When performing the multi-order neighbor aggregation operation in step (1), over-smoothing can be alleviated by aggregating different structural information, thereby increasing the differences between nodes.

[0028] Based on an in-depth analysis and understanding of the graph topology, this paper solves the over-smoothing problem encountered by graph neural networks when processing complex graph structures. By introducing the concept of topological perturbations, it enhances the model's sensitivity to changes in graph structure, thereby improving its performance and generalization capabilities in a variety of graph analysis tasks.

[0029] The beneficial effects of the present invention are:

[0030] Traditional graph neural networks still have the problem of over-smoothing in different graph scenarios, which occurs when the graph neural network identifies different types of nodes as the same type of nodes after multiple iterations. In response to the over-smoothing problem in graph neural networks and inspired by the idea of ​​topological perturbation, the present invention proposes a deep graph neural network training method based on topological structure perturbation. This method utilizes deep topological structure information and the idea of ​​applying topological perturbation. By expanding the local area of ​​the node, the local topological structure in the multi-hop neighborhood is sensed and topologically perturbed. The perturbation includes random perturbations of the nodes and edges in the area.

[0031] In traditional graph neural networks, although they have been widely used in diverse graph structure scenarios, they still face a significant challenge: the over-smoothing problem. This problem usually appears after the network has undergone multiple rounds of iterative training, causing the model to mistakenly identify nodes of different categories as the same category. In order to cope with this problem and draw on the advanced concept of topological perturbation, this paper innovatively proposes a novel training strategy-a deep graph neural network method based on topological structure perturbation. The core of this method is to deepen the understanding of the graph topology and cleverly integrate topological perturbation. By expanding the perception range of nodes, it is not limited to direct neighbors, but can capture local topological structures in more distant multi-hop neighborhoods, so as to more comprehensively understand the complex patterns in the graph. Furthermore, this method implements random perturbations on these local topological structures, which includes probabilistic random adjustments to node features and connecting edges in the selected area. The introduction of this topological perturbation mechanism aims to break the node feature convergence phenomenon that may be caused by traditional GNNs during the iteration process, and effectively alleviate the over-smoothing problem by increasing the sensitivity of the model to small changes in the graph structure. In addition, through this innovative training method, the model is able to learn more robust and discriminative node representations, thereby demonstrating excellent performance in various graph analysis tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but is not limited thereto.

[0034] Embodiment 1:

[0035] This embodiment provides a personalized music recommendation system architecture based on topological perturbation, and the steps are as follows:

[0036] (1) Define the graph structure. In this step, we first extract the short-term and long-term preferences of users for listening to songs based on a series of behaviors they perform on the website, such as listening to songs, collecting songs, and downloading songs. We then construct a music preference graph for each user, in which nodes represent users and songs, and edges represent interactions between users and songs, such as playing, rating, or collecting.

[0037] (2) Node feature assignment: Assign a feature vector to each node. For user nodes, the features include age, gender, historical playlists, etc.; for song nodes, the features include genre, artist, album, lyrics, etc.

[0038] (3) Implement node-level perturbation. In this step, some user nodes or song nodes are randomly selected for perturbation according to the set perturbation strategy. For example, the change of user preferences can be simulated by randomly setting the historical playback records of some users to zero.

[0039] (4) Implement edge-level perturbation. In this step, some edges in the interaction matrix (i.e., user-song interactions) are randomly selected for perturbation, such as setting the edge weight to zero to simulate possible forgetfulness or changes in interest of users.

[0040] (5) Aggregate neighbor information. In this step, for each layer of the GNN, the information of the neighbor nodes of the undisturbed nodes is aggregated to update the node representation. The aggregation can be a simple average, weighted average, or a more complex function.

[0041] (6) Training and optimization. In this step, the GNN model is trained using historical interaction data, the model parameters are optimized, and the intensity and probability of topological perturbations are adjusted to improve the generalization ability and recommendation quality of the model. Based on the updated node representation, personalized music recommendations are generated through a recommendation module.

[0042] In the above steps, adding topological perturbations is the key. Identifying different graph topologies and expanding the differences between node features can effectively alleviate the over-smoothing problem in graph social networks and prevent differences between nodes. Smoothing them during training improves the expressiveness of graph neural networks.

[0043] Through the above embodiments, the training method of the present invention introduces perturbations in the graph topology and establishes an inverse smoothing graph neural network based on topological perturbations, thereby successfully solving the over-smoothing problem of the graph neural network in the recommendation system, effectively improving the efficiency and accuracy of the recommendation system, and providing new ideas and methods for further research and application of graph neural networks.

[0044] The above descriptions are only preferred embodiments of the present invention. It should be pointed out that those skilled in the art can make improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be considered as within the protection scope of the present invention.

Claims

1. A deep graph neural network training method based on topological structure perturbation, characterized in that: Here are the steps: (1) Aggregate multi-order neighbors: expand the receptive field of each node; (2) Implement node-level perturbation: traverse all nodes in the topology and decide whether to perturb each node with probability p, thereby introducing randomness in the node set; (3) Implement edge-level perturbation: randomly remove a certain ratio of edges from the adjacency matrix of the k-hop topology of the node; (4) Multiple runs and result merging: The model is run multiple times, topological perturbations are implemented during the run, and diverse topological structures are generated. Then, the results of these different runs are merged to enhance the generalization ability and obtain the final model output.

2. The deep graph neural network training method based on topological structure perturbation according to claim 1, characterized in that: In step (1), the implementation method of aggregating multi-order neighbors is: Let G = (V, E) represent a graph, where V represents the set of nodes and E is the set of edges between nodes. The relationship formed when information is aggregated through different hop numbers k is defined as the node topology of the node, and is expressed as Indicates that the topological structure belongs to graph G; In the above formula, represents the hidden representation of node v at layer l, represents the hidden representation of node v at layer l-1, represents the hidden representation of node u at layer l-1; is the set of k-hop topological structures around node v; γ(x) is a differentiable function; Λ() represents the aggregation function; For each node v, determine all its neighbors from order 1 to order k, and construct the topological structure N of each node d (v) and N s (v), where N s (v) represents the nearest neighbor of node v, through N s (v) Reflecting shallow topological information in GNN, N d (v) represents the high-order neighbor node of v. The nearest neighbor node of v is called the first-order neighbor node of v, and the node whose distance to v is greater than 1 is called the high-order neighbor node; The nodes with a distance of 2 and the nodes with a distance of 3 are called the second-order neighbor nodes and third-order neighbor nodes of node v, respectively, and belong to N s (v) and N d (v) The nodes of the two sets and the edges connecting them constitute the local topological graph of the current node v. An aggregation function Λ is defined to merge the features of the node and its neighbors, and the features of the node and its multi-order neighbors are integrated at each layer. Finally, after introducing the deep topological structure, the expression of GNN node update is expressed as: Among them, ψ and Λ represent the combination function and aggregation function respectively, represents the hidden representation of node v at layer l, represents the hidden representation of node u at layer l-1.

3. The deep graph neural network training method based on topological structure perturbation according to claim 2 is characterized in that: In step (2), the implementation method of node-level perturbation is: According to the k-hop topology of node v For analysis, by setting a perturbation probability p, randomly select nodes for perturbation in the k-hop topology of each node: If a node v is perturbed at layer l, its feature representation is set to zero or other perturbations.

4. The deep graph neural network training method based on topological structure perturbation according to claim 3 is characterized in that: In step (3), the implementation method of edge-level perturbation is: Choose the longest path from node v to k. A t Expressed as The adjacency matrix of A t , randomly remove a certain ratio of edges in the topological structure, and randomly force the adjacency matrix A t Some non-zero elements in are zero, and this perturbation is not used during validation and testing.

5. The deep graph neural network training method based on topological structure perturbation according to claim 4 is characterized in that: In step (4), assume that the probability of each node being disturbed is p, let τ = |Γ|, τ represents the number of nodes in the k-th order neighborhood, and the probability of selecting any node u in Γ for disturbance is expressed as p·(1-p) τ ,if Then the probability of node u being disturbed will reach the maximum value, which means that if the number of runs is greater than or equal to e·(τ+1), it can be achieved that the number of times node u is disturbed is at least 1 in these runs. The nodes in the network form multiple different node topological structures. When performing the multi-order neighbor aggregation operation in step (1), over-smoothing is alleviated by aggregating different structural information, thereby increasing the differences between nodes.

Images