Method for accelerating training of graph neural network and related product

Abstract

The invention provides a method for accelerating training of a graph neural network and a related product, and the method comprises the steps: obtaining original graph data stored in a database, the original graph data comprising a plurality of nodes and edges; finding out redundant nodes in the original graph data according to a preset node index similarity judgment rule; according to a preset redundant node filtering rule, finding out redundant nodes meeting the filtering rule; redundant nodes meeting the filtering rule are removed, and optimized graph data are obtained; and replacing the original graph data with the obtained optimized graph data to serve as training input data of the graph neural network model. According to the method, the optimized graph data is obtained by eliminating partial redundant nodes with less information content in the original graph data, so that the scale of the original graph data can be effectively reduced on the premise of keeping the key information and the representation capability of the graph structure data; and the training efficiency of the large-scale graph data on the graph neural network model is improved.

Description

[0001] Technology Neighborhood

[0002] The present application generally relates to the technical field of graph neural networks. More specifically, the present application relates to a method for accelerating the training of graph neural networks and related products. Background Art

[0003] Graph Neural Networks (GNN) are a type of neural network that models and analyzes graph structured data. With the widespread application of deep learning in various fields, graph neural networks, as a new network structure for processing graph structured data, have become an important part of deep learning.

[0004] The core of graph neural networks is to use the message passing mechanism to allow each node to aggregate its own information and that of its neighboring nodes to model the entire network topology. This enables graph neural networks to effectively capture dependencies in network data and can be applied to a variety of graph analysis tasks such as node classification and link prediction. However, as the number of nodes and edges increases, the computational complexity of graph neural networks will increase. In addition, graph neural networks require multiple rounds of information transmission, resulting in slow convergence. Especially when processing large-scale graph data, a large number of nodes makes the training time too long and the training process is very slow.

[0005] In view of this, there is an urgent need to provide a method for accelerating the training of graph neural networks in order to improve the training efficiency of graph neural networks. Summary of the invention

[0006] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a method and related products for accelerating the training of graph neural networks in multiple aspects.

[0007] In a first aspect, the present application provides a method for accelerating the training of a graph neural network, comprising: obtaining original graph data stored in a database, wherein the original graph data includes multiple nodes and edges; finding redundant nodes in the original graph data according to a preset node indicator similarity judgment rule; finding redundant nodes that meet the filtering rules according to a preset redundant node filtering rule; removing the redundant nodes that meet the filtering rules to obtain optimized graph data; and replacing the original graph data with the obtained optimized graph data as training input data for a graph neural network model.

[0008] In one embodiment, the method of finding redundant nodes in the original graph data according to a preset node index similarity judgment rule includes: obtaining redundancy indexes of all nodes in the original graph data; determining a first judgment threshold based on the obtained redundancy indexes of all nodes; and determining nodes whose redundancy index is greater than the first judgment threshold as the redundant nodes.

[0009] In one embodiment, obtaining the redundancy index of all nodes in the original graph data includes performing the following steps for each node in the original graph data: calculating the neighborhood heterogeneity of the node, wherein the neighborhood heterogeneity is determined based on the distribution of different types of nodes in the neighboring nodes of the node; calculating the probability of the existence of a link between the node and other random nodes in the graph, wherein the probability of the existence of a link includes the probability of the existence of a connecting edge between two nodes; and calculating the redundancy index of the node based on the neighborhood heterogeneity of the node and the probability of the existence of a link for the node pair containing the node.

[0010] In one embodiment, for each node in the original graph data, the redundancy index WNH of the node is calculated using the following calculation formula according to the neighborhood heterogeneity of the node and the probability of the link between the node and all other nodes: v :

[0011]

[0012] In the formula, v represents the node in the graph, D v represents the degree of node v, N(v) represents the set of neighbor nodes of node v, and p vu represents the probability of the link between node v and node u, c v and c u represents the one-hot encoding of node v and node u labels, and ‖‖2 represents the two-norm calculation.

[0013] In one embodiment, determining the first judgment threshold based on the obtained redundancy index of all nodes includes: obtaining sample data containing redundancy indexes of N nodes; sorting all nodes in the sample data in order from large to small; calculating the quantile of the redundancy index sample data; dividing the redundancy index sample data into L intervals according to the quantile, where L is a positive integer less than N; finding the interval where the node ranked X1 after sorting is located, where X1∈[N*0.05, N*0.1] and X1 is a positive integer; and determining the upper limit value of the interval as the first judgment threshold.

[0014] In one embodiment, the calculation of the probability of link existence between the node and the node pairs formed by other random nodes in the graph includes: constructing a node pair link binary classification model; obtaining a node feature matrix and an adjacency matrix of the original graph data, wherein the node feature matrix represents a feature vector of each node in the original graph data; the adjacency matrix represents the connection relationship and connection status between the nodes in the original graph data; and inputting the node feature matrix and the adjacency matrix of the original graph data into the link binary classification model to obtain the link existence probability of the node pairs in the original data graph.

[0015] In one embodiment, the method of finding redundant nodes that meet the filtering rules according to preset redundant node filtering rules includes: obtaining redundancy indicators of all nodes in the original graph data; obtaining degrees of all nodes in the original graph data, wherein the degrees include the number of edges connected to the nodes; determining a second judgment threshold according to the redundancy indicators of all nodes in the original graph data; determining a third judgment threshold according to the degrees of all nodes in the original graph data; redundant nodes whose redundancy indicators are greater than the second judgment threshold and whose degrees are greater than the third judgment threshold are redundant nodes that meet the filtering rules.

[0016] In one embodiment, the method of determining the third judgment threshold based on the degrees of all nodes in the original graph data includes: obtaining an adjacency matrix of the original graph data; calculating the column sum of all columns in the adjacency matrix to obtain a node degree list including the degrees of N nodes; sorting the node degree list in descending order according to the numerical values ​​of the node degrees; calculating the quantile of the node degree list; dividing the node degree list into M intervals according to the quantile of the node degree list, where M is a positive integer less than N; finding the interval where the node ranked X2 after sorting is located, where X2∈[N*0.05, N*0.1] and X2 is a positive integer; and determining the upper limit value of the interval as the third judgment threshold.

[0017] In a second aspect, the present application provides an electronic device comprising: a processor; and a memory storing program instructions for a graph neural network acceleration method, wherein when the program instructions are executed by the processor, the device implements any one of the methods described in the first aspect.

[0018] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for a graph neural network acceleration method, and when the computer-readable instructions are executed by one or more processors, the method described in any one of the first aspects is implemented.

[0019] Through the scheme for accelerating the training of graph neural networks provided by the above multiple aspects and their embodiments, the present application can find redundant nodes in the original graph data and eliminate some redundant nodes with less information, thereby obtaining optimized graph data. Thereafter, by using the optimized graph data as the training input data of the graph neural network model, the scale of the original graph data can be effectively reduced while maintaining the key information and representation capabilities of the graph structure data. Therefore, the scheme of the present application can improve the training efficiency of large-scale graph data on the graph neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0021] Figure 1 An exemplary flow chart of a method for accelerating the training of a graph neural network according to some embodiments of the present application is shown;

[0022] Figure 2 An exemplary flow chart showing a method for finding redundant nodes in original graph data according to other embodiments of the present application is shown;

[0023] Figure 3 An exemplary flow chart of a method for obtaining a redundancy index of each node in original graph data according to some embodiments of the present application is shown;

[0024] Figure 4 An exemplary flow chart of a method for determining a first judgment threshold according to redundancy indicators of all nodes in some embodiments of the present application is shown;

[0025] Figure 5 An exemplary flow chart showing a method for calculating link existence probability of a node pair in some embodiments of the present application;

[0026] Figure 6 An exemplary flow chart of a method for calculating and finding redundant nodes that meet filtering rules in some embodiments of the present application is shown;

[0027] Figure 7 An exemplary flow chart of a method for determining a third judgment threshold according to the degrees of all nodes in some embodiments of the present application is shown;

[0028] Figure 8 An exemplary structural block diagram of an electronic device according to some embodiments of the present application is shown. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0030] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0031] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, the singular forms of "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0032] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0033] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0034] In order to facilitate a better understanding of the technical solutions described in the present application, the application scenarios involved in the embodiments of the present application are first explained below.

[0035] Many real-world data are in the form of graph structures, such as the chemical structure of protein molecules, urban traffic networks, knowledge graphs, etc. These graph data contain complex relationships between entities. In recent years, the demand for deep learning technologies in various research fields to process graph structured data has continued to increase, making deep learning research based on graph structured data one of the current hot topics. The main data types processed by deep learning include images, text, voice, and video, which are typical Euclidean space structured data and are regular samples in Euclidean space. Graph data is essentially non-regular data in non-Euclidean space data and has the characteristics of network structure. Therefore, it is impossible to directly apply traditional deep learning models that process Euclidean space data to graph data. Based on this, researchers have proposed a variety of improved models to process graph data based on existing deep learning methods. This requires the design of new deep learning models that can process graph structured data. Among them, the Graph Neural Network (GNN) series of models has become one of the most influential models for processing graph data.

[0036] The graph neural network model is highly interpretable in processing graph structured data and can be widely used in various downstream graph learning tasks, making it highly usable in applications. A variety of variants and extended models of graph neural networks continue to emerge, such as Graph Convolutional Networks (GCNs), Graph Attention Networks (GAT), and Graph Isomorphism Networks (GIN). These models further expand the scope of application of graph neural networks while maintaining the original characteristics of graph neural networks.

[0037] Traditional graph neural network models, especially graph convolutional neural network models, mostly use full batch training. During the training process, the graph neural network updates the nodes by aggregating the neighbor features of the nodes. As the number of network layers increases, the calculation process will grow exponentially, resulting in low training efficiency of the graph neural network model and high consumption of computing and storage resources. In addition, as the scale of graph data in practical applications continues to expand, full batch training often fails to run normally due to "out of memory" problems. In order to improve the training efficiency of graph neural networks, researchers have been actively adopting sampling-based small batch training methods. This type of method uses a sampler to extract subgraphs from the complete graph data and trains in small batches to achieve acceptable convergence speed and training efficiency.

[0038] However, as the scale of graph data continues to increase, the efficiency of sampling-based training is also subject to certain limitations. Currently, most work is only performed on small data sets, resulting in limited scalability and generalization capabilities of the model on large-scale graph data. To achieve accuracy improvements on large-scale datasets, it usually takes much longer to train than on small datasets. The training efficiency of graph neural networks is significantly related to the size of the input graph. The larger the graph, the lower the training efficiency in the graph neural network. In order to improve the training efficiency of graph neural networks, a simple and efficient method is to reduce the size of the input graph, but directly reducing the size of the input graph will have a great impact on the accuracy of the model.

[0039] In view of this, an embodiment of the present application provides a method for accelerating the training of graph neural networks. By screening redundant nodes in the original graph data and deleting some redundant nodes, the method can reduce the time required for large-scale graph training while maintaining model accuracy and improve the training efficiency of graph neural networks.

[0040] Figure 1 An exemplary flowchart of a method 100 for accelerating the training of a graph neural network according to some embodiments of the present application is shown.

[0041] like Figure 1 As shown, at step S101, the original graph data stored in the database is obtained, wherein the original graph data includes multiple nodes and edges. In an implementation scenario, the original graph data refers to the original data containing complex network relationships generated in an actual scenario, wherein each node represents an entity in the network, and the edge represents the connection and correlation between the nodes. These original graph data stored in the database can be used to construct and analyze complex networks in different scenarios, such as commodity purchase networks, social networks, and knowledge graphs.

[0042] Next, at step S102, it is necessary to find the redundant nodes in the original graph data according to the preset node indicator similarity judgment rules. In an implementation scenario, the node indicator mainly refers to various indicators defined on the node in the graph data analysis task, which are used to describe and represent certain properties or characteristics of the node. It can include characteristic information such as the degree, type, attributes and labels of the node. It can be understood that node indicators can be used to analyze the importance of nodes in the network, and redundant nodes refer to nodes with highly similar indicators in the original graph data, which contain repeated feature information. Furthermore, node indicator similarity is often used to measure the degree of similarity between two node indicators.

[0043] By detecting and processing redundant nodes, the training efficiency of the graph neural network can be improved. However, if all redundant nodes are completely deleted, the graph structure of the original graph may be too sparse, affecting model training. Therefore, retaining an appropriate amount of redundant nodes instead of deleting them all can balance the training speed and the integrity of the graph structure to a certain extent. Therefore, at step S103, it is necessary to find redundant nodes that meet the filtering rules according to the preset redundant node filtering rules. After obtaining the redundant nodes that meet the filtering rules at step S103, at step S104, it is necessary to remove the redundant nodes that meet the filtering rules to obtain optimized graph data.

[0044] In an implementation scenario, most of the neighbors of redundant nodes overlap with each other, which easily forms a loop in the process of information dissemination and falls into a closed loop of repeated dissemination. Therefore, redundant nodes consume more computing resources but play a smaller role in the process of information dissemination. Therefore, the removal of redundant nodes has little impact on the graph structure. As an example, for the redundant nodes found, graph data sparse methods such as deleting duplicate nodes and merging similar nodes can be used to remove them, which can effectively reduce computing and storage costs and retain key structures and information as much as possible.

[0045] At step S105, the obtained optimized graph data can replace the original graph data as the training input data of the graph neural network model. In some embodiments, the original graph data stored in the database is used as the input of the graph neural network model for model training. The present application adopts the method of first finding redundant nodes and eliminating some redundant nodes according to the graph data sparsification method, which can effectively reduce the data scale while maintaining the key information and accuracy of the graph structure data, thereby improving the training efficiency of large-scale graph data on the graph neural network model.

[0046] The following will be combined Figure 2 The specific method of searching for redundant nodes in the aforementioned step S102 is described. Figure 2 An exemplary flow chart of a method 200 for searching for redundant nodes in original graph data according to some other embodiments of the present application is shown.

[0047] like Figure 2 As shown, at step S201, the redundancy index of all nodes in the original graph data is obtained. In an implementation scenario, the redundancy index is usually established on the set of neighbor nodes of the node to measure the similarity between these neighbor nodes. If the neighbor nodes of a node are very similar, for example, they are connected to nodes with highly overlapping functions, then the redundancy index of the node will be higher, and the more similar the information transmitted by the neighbor nodes representing the node is, the more redundant the characteristics of the node will be.

[0048] At step S202, a first judgment threshold is determined based on the redundancy index of all nodes obtained. After obtaining the first judgment threshold, at step S203, nodes whose redundancy index is greater than the first judgment threshold are determined as redundant nodes. In an implementation scenario, the distribution of redundancy indexes of all nodes in the original graph data can be statistically analyzed, and the first judgment threshold can be determined by selecting an area with large redundancy and relatively sparse distribution. It is also possible to conduct multiple experiments, adjust parameters and compare results to determine the optimal first judgment threshold. It is understandable that a higher first judgment threshold will retain relatively more redundant nodes, with less information loss, but a large calculation scale. A lower first judgment threshold will delete relatively more valuable nodes, with more information loss, but a large calculation scale. Therefore, when determining the first judgment threshold, a reasonable first judgment threshold can be selected according to the scale of the graph data and demand.

[0049] Figure 3 An exemplary flowchart of a method 300 for obtaining a redundancy index of each node in original graph data according to some embodiments of the present application is shown.

[0050] like Figure 3 As shown, in some embodiments, the acquisition of the redundancy index of all nodes in the aforementioned step S201 can be performed on each node by executing Figure 3 The node neighborhood heterogeneity is implemented by the steps S301, S302 and S303 shown in the figure. Specifically, at step S301, the neighborhood heterogeneity of the node is calculated, wherein the neighborhood heterogeneity is determined based on the distribution of different types of nodes in the neighboring nodes of the node. In some implementation scenarios, each node has a different number of neighboring nodes, and the neighboring node types of different types of nodes have different compositions. Therefore, the heterogeneity of the node neighborhood can be defined as the number of nodes in the neighbors of the node that are different from the node category. It can be understood that the higher the heterogeneity of the node neighborhood, the more categories of neighboring nodes that identify the node and the richer the information.

[0051] Next, at step S302, the probability of link existence between the node and the node pairs formed by other random nodes in the graph is calculated, wherein the probability of link existence includes the probability of the existence of a connecting edge between the two nodes. In an implementation scenario, the node pair consists of two nodes in the graph, and for each node E in the original network graph, it is necessary to randomly sample other F nodes to form F node pairs with node E, and then determine the probability of link existence between these F node pairs.

[0052] In step S303, based on the neighborhood heterogeneity of the node and the link existence probability of the node pair containing the node, the redundancy index WNH of the node is calculated using the following calculation formula: v :

[0053]

[0054] In the formula, v represents the node in the graph, D v represents the degree of node v, N(v) represents the set of neighbor nodes of node v, and p vu represents the probability of the link between node v and node u, c v and c u represents the one-hot encoding of node v and node u labels, and ‖‖2 represents the two-norm calculation.

[0055] Among them, the unique hot encoding of node labels refers to the process of converting the node's category label into a vector representation. Specifically, a flag is set for each node category. For a certain node, the flag corresponding to its category label is set to 1, and the remaining flags are set to 0. For example, assuming there are 3 node categories A, B, and C, and node 1 is category A, then its unique hot encoding is (1, 0, 0). Node 2 is category B, then its unique hot encoding is (0, 1, 0).

[0056] The two-norm calculation refers to the calculation of the two-norm of a vector. It should be noted here that the two-norm calculation belongs to the scope of the prior art. Therefore, the specific operation of the two-norm calculation in the present invention will not be repeated.

[0057] Figure 4 An exemplary flow chart of a method 400 for determining a first judgment threshold according to redundancy indicators of all nodes in some embodiments of the present application is shown.

[0058] like Figure 4 As shown, at step S401, sample data of redundancy indexes of N nodes are obtained. Then, at step S402, all nodes in the sample data are sorted in descending order. In an application scenario, there are N nodes in the original graph data. After the redundancy index sample data of the N nodes are calculated according to the above steps, the redundancy index sample data of the N nodes are sorted in order of size, so as to subsequently divide the intervals according to the size of the sample data.

[0059] At step S403, the quantile of the redundancy index sample data is calculated. In some embodiments, the quantile is an indicator that describes the distribution of values ​​in a statistical data set. It represents the value at a certain percentage position after the data set is arranged in order of size. The quantile of the redundancy index needs to be determined based on the amount of redundancy index sample data, that is, the scale of the original graph data. At step S404, the redundancy index sample data is divided into L intervals according to the obtained quantile, where L is a positive integer less than N. In some embodiments, the quantile corresponds to the interval. For example, if it is determined to be 100 quantiles, all sample data are divided into 100 intervals.

[0060] And at step S405, find out the interval where the node ranked as X1 after sorting is located, where X1∈[N*0.05, N*0.1] and X1 is a positive integer. Then, at step S406, determine the upper limit value of the interval as the first judgment threshold. In an implementation scenario, according to the sorting of the redundancy index of the node, the interval where the redundancy value of the redundant node is located is determined, and the first judgment threshold is further determined. The nodes whose redundancy index is less than the first judgment threshold have rich information and should be retained. In addition, it should be noted that the value of X1 is not limited. The screening intensity of redundant nodes can be controlled by adjusting the value of X1. The smaller the value of X1, the more redundant nodes are screened.

[0061] Figure 5 An exemplary flow chart of a method 500 for calculating link existence probabilities of node pairs according to some embodiments of the present application is shown.

[0062] like Figure 5 As shown, at step S501, a node pair link binary classification model is constructed. Then, at step S502, the node feature matrix and adjacency matrix of the original graph data are obtained, wherein the node feature matrix represents the feature vector of each node in the original graph data; the adjacency matrix represents the connection relationship and connection status between nodes in the original graph data. At step S503, the node feature matrix and adjacency matrix of the original graph data are input into the link binary classification model to obtain the link existence probability of the node pairs in the original data graph.

[0063] In some implementation scenarios, by inputting a partially observed graph into the link prediction model, it can predict the complete connection state in the graph, that is, predict the connection between pairs of nodes that did not originally exist but may exist. The node feature matrix of the original graph data represents the attribute characteristics of each node, and the adjacency matrix records the connection relationship between some known nodes in the graph. The link observation model can output a fully predicted adjacency matrix, showing the complete connection state between the predicted nodes.

[0064] Figure 6 An exemplary flow chart of a method 600 for calculating and finding redundant nodes that satisfy filtering rules according to some embodiments of the present application is shown.

[0065] like Figure 6 As shown, at step S601, the redundancy index of all nodes in the original graph data is obtained. Then, at step S602, the degree of all nodes in the original graph data is obtained, where the degree includes the number of edges connected to the node. In some embodiments, the degree of a node represents the number of edges connected to the node, which reflects the connection status of the node in the original graph data. The higher the degree of a node, the more connections the node has with other nodes, and the more important it is in the original graph, which is an important indicator for measuring node redundancy.

[0066] At step S603, a second judgment threshold is determined based on the redundancy index of all nodes in the original graph data; at step S604, a third judgment threshold is determined based on the degree of all nodes in the original graph data. It should be noted that the second judgment threshold is similar to the method of the first judgment threshold, and both use the concept of statistical quantiles, which will not be elaborated here. Among them, the value of the second judgment threshold can be the same as the value of the first judgment threshold, or it can be considered to set the first judgment threshold to a lower quantile and the second judgment threshold to a higher quantile, while ensuring the screening quality and increasing the filtering ratio of redundant nodes to achieve further filtering of redundant nodes, so as to reduce the scale of the original graph data and accelerate model calculations.

[0067] At step S605, the redundant nodes whose redundancy index is greater than the second judgment threshold and whose degree is greater than the third judgment threshold are the redundant nodes that meet the filtering rules. In some implementation scenarios, the larger the redundancy index is, the more redundant the information of the node is, and the nodes with large redundancy index usually have high degrees. The height of the node represents that the node needs to calculate and update the relationship with many neighboring nodes, which is more computationally resource-intensive. In addition, the highly redundant nodes themselves have limited representation capabilities. Even if the degree is large, the improvement in representing network information is limited. Therefore, deleting such highly redundant nodes with high degrees that are both information redundant and computationally resource-intensive can significantly reduce the waste of computing resources and accelerate model training. At the same time, deletion also has a very limited impact on the network representation capabilities of the original graph data.

[0068] Figure 7 An exemplary flow chart of a method 700 for determining a third judgment threshold according to the degrees of all nodes in some embodiments of the present application is shown.

[0069] like Figure 7 As shown, at step S701, the adjacency matrix of the original graph data is obtained. In an implementation scenario, the adjacency matrix of the original graph data is a matrix representing the connection relationship between the nodes in the graph. The rows and columns of the matrix represent all the nodes in the graph. If there is a connection between two nodes, the element value in the corresponding matrix is ​​1, otherwise it is 0. Then, at step S702, the column sum of all columns in the adjacency matrix is ​​calculated to obtain a node degree list including the degrees of N nodes. In an implementation scenario, each column of the adjacency matrix corresponds to a node in the graph. Each row in the matrix represents the connection relationship between a node and other nodes in the graph. By summing all the elements in a column, the number of edges connecting the node to other nodes can be counted. By traversing and calculating the column sum of all columns of the adjacency matrix, the degree statistics of all nodes in the graph can be obtained, and a node degree list including the degrees of N nodes is obtained.

[0070] At step S703, the node degree list obtained at step S702 is sorted in descending order according to the numerical values ​​of the node degrees. At step S704, the quantile of the node degree list is calculated. At step S705, the node degree list is divided into M intervals according to the quantile of the node degree list, where M is a positive integer less than N. At step S706, the interval where the node ranked X2 after sorting is located is found, where X2∈[N*0.05, N*0.1] and X2 takes a positive integer. At step S707, the upper limit value of the interval is determined as the third judgment threshold. It should be noted that the method for determining the third judgment threshold and the method for determining the first judgment threshold both utilize the concept of quantile / distribution interval in statistics. Regarding the same parts, they will not be repeated here, and some descriptions of the method for determining the first judgment threshold are also applicable to the method for determining the third judgment threshold.

[0071] It should be further explained that, when the similarity of the original image data is relatively high, the values ​​of the first judgment threshold, the second judgment threshold and the third judgment threshold may also be set by the operator based on experience.

[0072] When building a graph neural network model, the convolution layer is the core component for learning node features. In the convolution layer, each node performs feature propagation aggregation and update based on the connection relationship between itself and its neighboring nodes, thereby obtaining a new feature expression. Therefore, the convolution layer of any graph neural network can be represented by the following nonlinear function:

[0073] H l+1 =f(H, A)

[0074] In the formula, H 0 =X, the feature X of the node in the graph is used as the input of the first layer, X∈R N*D , N is the number of nodes in the graph, D is the dimension of each node feature vector, and A is the adjacency matrix.

[0075] In the training process of graph neural networks, the number of nodes and edges is an important factor that determines the computational complexity. The adjacency matrix A as input directly reflects the data scale of the graph. This paper effectively reduces the number of non-zero elements in A by screening and eliminating redundant nodes, thereby reducing the complexity of the matrix. Specifically, as can be seen from the previous convolutional layer formula, the adjacency matrix A directly affects the feature transformation calculation process. The more non-zero values ​​in A, the more complex the node connection relationship, and the more feature aggregation calculations that need to be performed. This paper effectively reduces the number of parameters in A by screening redundant nodes for the input adjacency matrix A, simplifies the feature propagation calculation process, and reduces the computational burden of model training, thereby significantly accelerating the training process of the graph neural network model and improving the overall operating efficiency.

[0076] Although the specific operation methods of different models are different, the method of screening and eliminating redundant nodes directly simplifies the input data scale, which can directly reduce the amount of calculation and thus speed up the training process. The positive impact of this method on reducing time consumption has also been verified through experimental data.

[0077] Corresponding to the aforementioned application function implementation method embodiment, the present disclosure embodiment also provides a method as follows: Figure 8 Electronic equipment shown. Figure 8 An exemplary structural block diagram of an electronic device 800 according to some embodiments of the present application is shown.

[0078] Figure 8 The electronic device 800 shown includes: a processor 810; and a memory 820, wherein the memory 820 stores executable program instructions. When the program instructions are executed by the processor 810, the electronic device implements any of the methods described above.

[0079] exist Figure 8 In the electronic device 800, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the electronic device 800 may also include components related to this embodiment. Figure 8 . The processor 810 can control the operation of the electronic device 800. For example, the processor 810 controls the operation of the electronic device 800 by executing the program stored in the memory 820 on the electronic device 800. The processor 810 can be implemented by a central processing unit (CPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the electronic device 800. However, the present disclosure is not limited to this. In this embodiment, the processor 810 can be implemented in any appropriate manner. For example, the processor 810 can take the form of a computer-readable medium, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc., such as a microprocessor or a processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor.

[0080] The memory 820 may be used to store hardware of various data and instructions processed in the electronic device 800. For example, the memory 820 may store processed data and data to be processed in the electronic device 800. The memory 820 may store data sets that have been processed or to be processed by the processor 810. In addition, the memory 820 may store applications, drivers, etc. to be driven by the electronic device 800. For example: the memory 820 may store various programs related to model building and constraint setting to be executed by the processor 810. The memory 820 may be a DRAM, but the present disclosure is not limited thereto. The memory 820 may include at least one of a volatile memory or a non-volatile memory. The non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), etc. The volatile memory may include a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous DRAM (SDRAM), a PRAM, an MRAM, an RRAM, a ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 820 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (XD) card, caches, or a memory stick.

[0081] In summary, the specific functions implemented by the memory 820 and the processor 810 of the electronic device 800 provided in the embodiments of this specification can be explained in comparison with the aforementioned embodiments in this specification, and can achieve the technical effects of the aforementioned embodiments, and will not be repeated here.

[0082] Furthermore, the present disclosure may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which computer program instructions (or computer program, or computer instruction code) are stored. When the computer program instructions (or computer program, or computer instruction code) are executed by a processor of an electronic device (or electronic device, server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present disclosure.

[0083] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for accelerating the training of a graph neural network, characterized in that: include: Acquire original graph data stored in a database, wherein the original graph data includes a plurality of nodes and edges; Finding redundant nodes in the original graph data according to a preset node index similarity determination rule; Find the redundant nodes that meet the filtering rules according to the preset redundant node filtering rules; Removing redundant nodes that meet the filtering rules to obtain optimized graph data; and The obtained optimized graph data replaces the original graph data as training input data for the graph neural network model.

2. The method according to claim 1, characterized in that The method of finding redundant nodes in the original graph data according to the preset node index similarity determination rule includes: Obtaining redundancy indicators of all nodes in the original graph data; Determine a first judgment threshold according to the obtained redundancy indexes of all nodes; A node whose redundancy index is greater than a first judgment threshold is determined as the redundant node.

3. The method according to claim 2, characterized in that The step of obtaining the redundancy index of all nodes in the original graph data includes performing the following steps for each node in the original graph data: Calculating the neighborhood heterogeneity of the node, wherein the neighborhood heterogeneity is determined based on the distribution of different types of nodes among neighboring nodes of the node; Calculate the probability of a link existing between the node and other random nodes in the graph, where the link existence probability includes the probability of a connecting edge existing between the two nodes; The redundancy index of the node is calculated based on the neighborhood heterogeneity of the node and the link existence probability of the node pair containing the node.

4. The method according to claim 3, characterized in that: For each node in the original graph data, the redundancy index WNH of the node is calculated using the following calculation formula based on the neighborhood heterogeneity of the node and the probability of the link between the node and all other nodes: v : In the formula, v represents the node in the graph, D v represents the degree of node v, N(v) represents the set of neighbor nodes of node v, and p vu represents the probability of the link between node v and node u, c v and c u represents the one-hot encoding of node v and node u labels, ‖ ‖2 represents the two-norm calculation.

5. The method according to claim 2, characterized in that: Determining the first judgment threshold according to the obtained redundancy indexes of all nodes includes: Obtain sample data of redundancy indicators of N nodes; Sort all nodes in the sample data in descending order; Calculate the quantile of the redundancy index sample data; Dividing the redundancy index sample data into L intervals according to the quantile, where L is a positive integer less than N; Find the interval where the node ranked X1 is located after sorting, where X1∈[N*0.05, N*0.1] and X1 is a positive integer; An upper limit value of the interval is determined as a first judgment threshold.

6. The method according to claim 3, characterized in that The calculation of the probability of existence of a link between a node pair formed by the node and other random nodes in the graph includes: Construct a node pair link binary classification model; Obtaining a node feature matrix and an adjacency matrix of the original graph data, wherein the node feature matrix represents a feature vector of each node in the original graph data; and the adjacency matrix represents a connection relationship and a connection status between nodes in the original graph data; The node feature matrix and adjacency matrix of the original graph data are input into the link binary classification model to obtain the link existence probability of the node pairs in the original data graph.

7. The method according to any one of claims 1 to 6, characterized in that: The step of finding a redundant node that satisfies the filtering rule according to the preset redundant node filtering rule includes: Obtaining redundancy indicators of all nodes in the original graph data; Obtaining the degrees of all nodes in the original graph data, wherein the degrees include the number of edges connected to the nodes; Determine a second judgment threshold according to the redundancy index of all nodes in the original graph data; Determine a third judgment threshold according to the degrees of all nodes in the original graph data; The redundant nodes whose redundancy index is greater than the second judgment threshold and whose degree is greater than the third judgment threshold are the redundant nodes that meet the filtering rule.

8. The method according to claim 7, characterized in that The step of determining the third judgment threshold according to the degrees of all nodes in the original graph data includes: Obtaining an adjacency matrix of the original graph data; Calculate the column sum of all columns in the adjacency matrix to obtain a node degree list including the degrees of N nodes; Sorting the node degree list in descending order according to the values ​​of the node degrees; Calculating the quantile of the node degree list; Dividing the node degree list into M intervals according to the quantile of the node degree list, where M is a positive integer less than N; Find the interval where the node ranked X2 after sorting is located, where X2∈[N*0.05, N*0.1] and X2 is a positive integer; The upper limit value of the interval is determined as the third judgment threshold.

9. An electronic device, characterized in that: include: processor; A memory storing program instructions for a method for accelerating the training of a graph neural network, wherein when the program instructions are executed by the processor, the device implements the method described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions for a method for accelerating the training of a graph neural network are stored thereon, and when the computer-readable instructions are executed by one or more processors, the method described in any one of claims 1-8 is implemented.

Images