Graph neural network training method, device, equipment and readable storage medium

Abstract

The present application relates to a graph neural network training method, apparatus, device, and readable storage medium. The method comprises: obtaining multiple training edge sets of a graph structure and a graph neural network to be trained; determining the neighbor node set of each node in each edge of each training edge set; obtaining common neighbor encoding information by determining the neighbor position encoding information and structure position encoding information of each neighbor node in the neighbor node set, and expanding the common neighbors between nodes; training the graph neural network based on the structure position encoding information and the common neighbor encoding information, and terminating the training to obtain a trained graph neural network when preset training conditions are met; obtaining a graph structure to be processed, inputting the graph structure to be processed into the trained graph neural network, and obtaining a prediction result for the graph structure to be processed. This method can improve the accuracy of prediction.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a graph neural network training method, apparatus, device and readable storage medium. Background Art

[0002] Graph Neural Networks (GNNs) are algorithms that use neural networks to learn from graph-structured data, extracting and discovering features and patterns within it, and meeting the demands of graph learning tasks such as clustering, classification, prediction, segmentation, and generation. GNNs are widely used in diverse fields, such as circuit design, traffic flow prediction, route planning, and social impact prediction.

[0003] In related technologies, the training method of graph neural networks limits the ability to obtain more graph features, resulting in low prediction accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a graph neural network training method, device, computer equipment, computer-readable storage medium and computer program product that can improve prediction accuracy in response to the above technical problems.

[0005] In a first aspect, the present application provides a graph neural network training method, comprising:

[0006] Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. The graph structure is determined based on the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, which are used to represent the business information between the business nodes.

[0007] For each training edge set, determine the neighbor node set of each node in each edge;

[0008] Determine neighbor position coding information of each neighbor node in the neighbor node set and structural position coding information of each neighbor node in the graph structure;

[0009] Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between nodes based on the number of occurrences and neighbor position coding information, and obtain the common neighbor coding information;

[0010] The graph neural network is trained based on the structural position encoding information and the common neighbor encoding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0011] Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the graph structure to be processed.

[0012] In one embodiment, the business scenario includes at least one of a traffic prediction scenario, a course selection scenario, and a social network analysis scenario.

[0013] In the case where the business scenario is a traffic prediction scenario, the graph structure is determined based on the scheduled operation route information of the transportation tool. The business nodes of the business scenario include the target objects associated with the scheduled operation route information, and the business information between the business nodes includes the scheduled operation routes between the target objects.

[0014] In the scenario where the business scenario is a course selection scenario, the graph structure is determined based on the access behavior data of the selecting object and the selected object. The business nodes of the business scenario include the selecting object and the selected object, and the business information between the business nodes includes the access behavior between the selecting object and the selected object.

[0015] When the business scenario is a social network analysis scenario, the graph structure is determined by the interactive behavior data in the social network. The business nodes of the business scenario include the target objects that generate the interactive behavior data, and the business information between the business nodes includes the interactive behavior between the target objects.

[0016] In one embodiment, determining the neighbor position encoding information of each neighbor node in the neighbor node set and the structural position encoding information of each neighbor node in the graph structure includes:

[0017] According to the time sequence information of each neighbor node in the neighbor node set, each neighbor node is sorted in chronological order, and the sorted neighbor nodes are encoded to obtain the neighbor position encoding information of each neighbor node;

[0018] Determine the number of paths between each neighbor node and the node;

[0019] The structural position encoding information of the neighbor nodes in the graph structure is determined according to the number of paths.

[0020] In one embodiment, for each training edge set, determining a set of neighbor nodes for each node in each edge includes:

[0021] For each training edge set, the neighbor nodes of each node in each edge are sampled in parallel to obtain the first-order neighbor nodes and high-order neighbor nodes of each node;

[0022] The neighbor node set of each node is obtained based on the first-order neighbor nodes and high-order neighbor nodes.

[0023] In one embodiment, the nodes include a source node and a target node, the number of occurrences of each neighbor node in a set of N neighbor nodes is determined, and common neighbors between the nodes are expanded based on the number of occurrences and neighbor position encoding information to obtain common neighbor encoding information, including:

[0024] Determine a common neighbor set of a first neighbor node set of the source node and a second neighbor node set of the target node;

[0025] For each neighbor node in the first neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding first two-dimensional vector;

[0026] For each neighbor node in the second neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding second two-dimensional vector;

[0027] Performing Fourier transform on the first two-dimensional vector and the second two-dimensional vector respectively to obtain a first embedded representation and a second embedded representation corresponding to each;

[0028] According to the first embedding representation, the second embedding representation and the neighbor position encoding information, information of each common neighbor in the common neighbor set is determined to obtain common neighbor encoding information.

[0029] In one embodiment, training a graph neural network based on structural position encoding information and common neighbor encoding information includes:

[0030] Obtain the original features of each node, the edge feature encoding information and time encoding information of each edge;

[0031] The structural position encoding information is concatenated with the corresponding original features to obtain the node encoding information of each node;

[0032] The node coding information, edge feature coding information, time coding information and common neighbor coding information are concatenated and mapped to obtain mapping features;

[0033] Use mapping features to train the Transformer of the graph neural network.

[0034] In one embodiment, obtaining multiple training edge sets of a graph structure includes:

[0035] If the current training of the graph neural network is the first training, all edge sets corresponding to the graph structure are divided based on the normal distribution to obtain multiple training edge sets;

[0036] For each training from the start of the second training to the end of the i-th training, obtain the previous training results of the graph neural network;

[0037] Update the weight of each edge in the graph structure according to the last training result to obtain the updated weight of each edge;

[0038] According to the updated weight of each edge, the target edge is extracted from all edge sets corresponding to the graph structure to obtain multiple training edge sets.

[0039] In one embodiment, the method further comprises:

[0040] Processing a pre-stored graph structure to obtain a graph data structure storing connection relationships between nodes in the graph structure;

[0041] Split the graph data structure to obtain all edge sets of the graph structure.

[0042] In a second aspect, the present application further provides a graph neural network training device, comprising:

[0043] The data acquisition module is used to obtain multiple training edge sets of the graph structure and the graph neural network to be trained. The graph structure is determined according to the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, which are used to represent the business information between the business nodes.

[0044] A neighbor node processing module is used to determine the neighbor node set of each node in each edge for each training edge set;

[0045] A first encoding module is used to determine the neighbor position encoding information of each neighbor node in the neighbor node set and the structural position encoding information of each neighbor node in the graph structure;

[0046] The second encoding module is used to determine the number of occurrences of each neighbor node in the N neighbor node sets, and expand the common neighbors between the nodes according to the number of occurrences and the neighbor position encoding information to obtain the common neighbor encoding information;

[0047] The training module is used to train the graph neural network based on the structural position encoding information and the common neighbor encoding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network;

[0048] The prediction module is used to obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the target operation route information in the graph structure to be processed.

[0049] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: obtaining multiple training edge sets of a graph structure and a graph neural network to be trained, obtaining multiple training edge sets of a graph structure and a graph neural network to be trained, wherein the graph structure is determined according to a business scenario, the nodes of the graph structure correspond to the business nodes of the business scenario, the nodes are connected by edges, and the edges are used to represent the business information between the business nodes;

[0050] For each training edge set, determine the neighbor node set of each node in each edge;

[0051] Determine neighbor position coding information of each neighbor node in the neighbor node set and structural position coding information of each neighbor node in the graph structure;

[0052] Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between nodes based on the number of occurrences and neighbor position coding information, and obtain the common neighbor coding information;

[0053] The graph neural network is trained based on the structural position encoding information and the common neighbor encoding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0054] Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the graph structure to be processed.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0056] Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. The graph structure is determined according to the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, which are used to represent the business information between the business nodes.

[0057] For each training edge set, determine the neighbor node set of each node in each edge;

[0058] Determine neighbor position coding information of each neighbor node in the neighbor node set and structural position coding information of each neighbor node in the graph structure;

[0059] Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between nodes based on the number of occurrences and neighbor position coding information, and obtain the common neighbor coding information;

[0060] The graph neural network is trained based on the structural position encoding information and the common neighbor encoding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0061] Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the graph structure to be processed.

[0062] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0063] Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. The graph structure is determined according to the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, which are used to represent the business information between the business nodes.

[0064] For each training edge set, determine the neighbor node set of each node in each edge;

[0065] Determine neighbor position coding information of each neighbor node in the neighbor node set and structural position coding information of each neighbor node in the graph structure;

[0066] Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between nodes based on the number of occurrences and neighbor position coding information, and obtain the common neighbor coding information;

[0067] The graph neural network is trained based on the structural position encoding information and the common neighbor encoding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0068] Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the graph structure to be processed.

[0069] The above-mentioned graph neural network training method, device, computer equipment, computer-readable storage medium and computer program product obtain a graph structure determined based on a business scenario, and split the graph structure into multiple training edge sets for training the graph neural network according to the characteristics of the graph structure and the dependencies between nodes. That is, a small batch of training samples is used to train the graph neural network to be trained. On this basis, by considering the position of each edge node in the training edge set in the graph structure and the position of the node in the neighboring node, the corresponding neighbor position coding information and structure position coding information are obtained. The neighbor position coding information and structure position coding information are added on the basis of considering time coding, and the enhanced input features In order to further improve the structure representation ability of graph neural networks, the common neighbors between nodes are expanded to capture more frequency domain information and obtain common neighbor encoding information. In this way, the information of input features is enhanced to train the temporal graph neural network, which solves the missing structural information in the model and improves the representation ability of the temporal graph neural network for structural sensitivity, thereby obtaining a trained graph neural network. When processing the graph structure to be processed based on the trained graph neural network, it can better capture the changes of nodes over time and time-sensitive features, and can more comprehensively determine the rationality of business information in the graph structure to be processed, thereby improving the accuracy of business information prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 FIG is an application environment diagram of a neural network training method in one embodiment;

[0072] Figure 2 FIG is a flow chart of a neural network training method according to an embodiment;

[0073] Figure 3 1 is a schematic diagram of a flow chart of a graph neural network to be trained in one embodiment;

[0074] Figure 4 206 is a flow chart of step 206 in one embodiment;

[0075] Figure 5 A schematic diagram of a process for determining common neighbor coding information in one embodiment;

[0076] Figure 6 Schematic diagram of the time cost process in one embodiment;

[0077] Figure 7 1 is a flow chart of a neighbor sampling method according to an embodiment;

[0078] Figure 8 A schematic flow chart of a detailed flow chart of a feature processing stage in one embodiment;

[0079] Figure 9 A flowchart of a method for determining a training sample in one embodiment is shown;

[0080] Figure 10 FIG is a flow chart of a neural network training method according to another embodiment;

[0081] Figure 11 FIG is a structural block diagram of a neural network training device in one embodiment;

[0082] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0084] Graph neural networks can be widely used in different fields, such as circuit design, traffic flow prediction, traffic route planning, and social impact prediction. For example, in traffic scenarios, route conflicts can be understood as mutual interference or potential collisions between vehicles, pedestrians, or other traffic participants during road use. If there is a route conflict, there will be safety hazards and waste of resources, so it is necessary to ensure traffic safety and traffic resource utilization. In related technologies, traditional optimization algorithms are used to ensure the reliability of routes, thereby ensuring traffic safety and traffic resource utilization.

[0085] However, for complex and dynamically changing traffic scenarios, the optimization algorithms of related technologies cannot accurately predict the rationality of operating routes, resulting in low utilization of traffic resources. Therefore, to address the low prediction accuracy of existing graph neural networks, it is proposed to improve prediction accuracy by training neural networks. While temporal graphs are widely used for prediction in various fields, initial exploration of the Temporal Graph Transformer (TGT) in temporal networks has been limited. Related dynamic graph neural networks trained using a layered aggregation approach for graph neural networks present challenges such as oversmoothing and overcompression. Another neural network architecture, the Transformer, has demonstrated excellent performance in modeling long-term dependencies, making it a promising solution for representation learning in temporal networks. Its powerful Transformer model has attracted emerging efforts in temporal networks, which represent node dynamic embeddings by training them to encode one-hop neighbors. However, due to the dramatic increase in computational complexity caused by neighborhood explosion, these Temporal Graph Transformers (TGTs) often employ oversimplified neighborhood and position information designs. They only consider temporal encodings without distinguishing between a node's position in the graph structure and its set of neighbors. Second, they ignore high-order neighbor information, which limits the model's ability to capture more graph features and leads to low prediction accuracy.

[0086] Therefore, in order to solve the technical problem of low prediction accuracy, a graph neural network training method is proposed. By obtaining the graph structure, multiple training edge sets for training the graph neural network are determined based on the graph structure. For each training edge set, the neighbor node set of each node in each edge is determined. The position of each neighbor node in the graph structure and the position information in the neighbor node set are taken into consideration, and the input features are enhanced to train the graph neural network, thereby enhancing the model representation ability and improving the accuracy of training. When processing the graph structure to be processed based on the trained graph neural network, it can better capture the changes of nodes over time and time-sensitive features, and can more comprehensively determine the accuracy of the prediction of the edges to be processed in the graph structure to be processed. In traffic prediction scenarios, the rationality of the target operation route information can be targeted to achieve reasonable operation route planning, ensuring driving safety and improving the utilization of traffic resources.

[0087] The graph neural network training method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal obtains multiple training edge sets of the graph structure and the graph neural network to be trained from the server; for each training edge set, the neighbor node set of each node in each edge is determined; the neighbor position coding information of each neighbor node in the neighbor node set and the structural position coding information of each neighbor node in the graph structure are determined; the number of occurrences of each neighbor node in the N neighbor node sets is determined, and the common neighbors between the nodes are expanded according to the number of occurrences and the neighbor position coding information to obtain the common neighbor coding information; the graph neural network is trained according to the structural position coding information and the common neighbor coding information, and when the preset training conditions are met, the training is terminated to obtain the trained graph neural network; the graph structure to be processed is obtained, and the graph structure to be processed is input into the trained graph neural network to obtain the prediction result of the target operation route information in the graph structure to be processed.

[0088] The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, projectors, and the like. Portable wearable devices may include smart watches, smart bracelets, head-mounted devices, and the like. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0089] In an exemplary embodiment, Figure 2 As shown in the figure, a graph neural network training method is provided, which is applied to Figure 1 The terminal in FIG is taken as an example to illustrate, including the following steps 202 to 212. Among them:

[0090] Step 202: Obtain multiple training edge sets of the graph structure and the graph neural network to be trained. The graph structure is determined according to the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, and the edges are used to represent the business information between the business nodes. The graph neural network to be trained can be a pre-built Transformer-based fast perception time series graph neural network, called QSFormer. The schematic diagram of the graph neural network to be trained is as follows: Figure 3 As shown, it includes a dynamic graph input module, a feature processing module and a Transformer module. The dynamic graph input module is used to generate small batch samples for training. The small batch samples can be generated using an adaptive method to generate small batches that are beneficial to training. Being beneficial to training can be understood as being able to accelerate the convergence speed of model training and improve training efficiency.

[0091] The feature processing module can be used to accept a mini-batch as input, process the mini-batch on the edge unit, extract high-order neighbors, and encode features for the source node u and target node v of the edge (u, v, t) at time t, and finally connect all the features to obtain the feature encoding Z_(u, v)^t. That is, the feature processing module includes at least a padded parallel neighbor sampler, an encoder, and a mapper. The encoder may include a position-differentiated encoder, an extended common neighbor encoder, etc. Among them, the encoder can be used to encode the features of nodes u and v and their neighborhoods, edges (u, v, t) and related timestamps into node, edge, time and extended common neighbor encodings, which are represented as node encoding information Edge encoding information Time code information Extended common neighbor encoding information Where * represents the source node u or the target node v. The time coding function for Δt′=tt′ is given as follows, and the implementation method of the time coding information is obtained:

[0092]

[0093] where ω1,…,ω dt is a trainable parameter, d t is the dimension of time encoding. The mapper can be the encoded features that are then patched, mapped and concat into a unified feature encoding Represents the comprehensive state of the edges input to the Transformer module.

[0094] The Transformer module can be an output encoding that receives feature processing As input, and through multi-layer multi-head attention Multi-Head Attention, feed forward network Feed Forward training, etc., finally, it is passed through an average pooling layer AvgPool to get the final embedding, which is split into the embedding representation of the source node u and the target node v of the edge (u, v, t) and That is, the Transformer module includes normalization layer, multi-head attention layer, feedforward network and average pooling, among which the multi-head attention layer allows the model to focus on different embedding segments and capture the complex dependencies in the graph; the feedforward network is after normalization and residual connection, and the attention output is further refined through the feedforward network; the average pooling layer is the last step of the Transformer involving pooling output to form the representation of the source node u and the target node v of the edge, which can be expressed as This is crucial for downstream tasks such as link prediction. In the link prediction task, we evaluate similarity between them to determine whether there is a link between u and v at time t.

[0095] The graph structure can be understood as a dynamic graph. The graph structure is determined based on the business scenario. The nodes of the graph structure correspond to the business nodes of the business scenario. The nodes are connected by edges, which are used to represent the business information between the business nodes. The business scenario includes at least one of the following: traffic prediction scenario, course selection scenario, and social network analysis scenario. This embodiment uses the traffic prediction scenario as an example. It should be noted that different application scenarios correspond to different graph structures, and the specific meanings of the nodes in the graph structure and the meanings of the edges between nodes are also different.

[0096] Optionally, in a traffic forecasting scenario, the graph structure is determined based on the scheduled route information of a vehicle. The business nodes of the business scenario include target objects associated with the scheduled route information, and the business information between the business nodes includes the scheduled routes between the target objects. That is, the graph structure is determined based on the scheduled route information of a vehicle. The scheduled route information may include route information between multiple regions within a preset time period. The scheduled route information may include the scheduled route, the scheduled route's operating time, the target objects associated with the scheduled route, and associated information related to the target objects, including the airport type, the region identifier of the region where the airport is located, the airport identifier, and the geographic location of the region where the airport is located.

[0097] There are at least two target objects, which can be cross-regional airports. Each node corresponds to one target object. If the target object is an airport, the node features include the airport type, the region identifier of the area where the airport is located, the airport identifier, and the geographic location of the area where the airport is located. Edge features corresponding to the edges between nodes include the number of flights and the flight time corresponding to each flight. It should be noted that there can be a transit airport between the source and destination airports. Therefore, the route from the source airport to the destination airport will include at least one path, that is, at least one edge.

[0098] Optionally, in a scenario where the business scenario is a course selection scenario, the graph structure is determined based on the access behavior data of the selecting object and the selected object. The business nodes of the business scenario include the selecting object and the selected object, and the business information between the business nodes includes the access behavior between the selecting object and the selected object. That is, for a large open network course scenario, the graph structure includes two types of nodes. The business nodes corresponding to the nodes can be selecting objects (for example, students) and / or selected objects (for example, course units). The edges between the nodes represent the students' access behavior to specific content units. Each link is associated with a multidimensional feature, which can also be called an N-dimensional feature. N can be, but is not limited to, 4, or 3 or 2. Among them, the association of multidimensional features can be determined by selecting N behavioral information from the learning behavior information. The selection method can be determined according to actual needs and is not limited here. Among them, learning behavior information may include learning start time (start_time_DI), learning end time (last_event_DI), number of course interactions (nevents), number of learning access days (ndays_act), number of video plays (nplay_video), number of learning chapters (nchapters), number of posts (nforumposts), etc., among which, the number of course interactions refers to the number of times the learner logs in to the learning account; the number of posts refers to the number of times the learner speaks in discussion areas and forums.

[0099] Optionally, in the case where the business scenario is a social network analysis scenario, the graph structure is determined by the interactive behavior data in the social network, the business nodes of the business scenario include the target objects that generate the interactive behavior data, and the business information between the business nodes includes the interactive behavior between the target objects, that is, for the social network scenario, the interactive behavior data includes the directed reply network information based on the social network, and the business nodes corresponding to the nodes of the graph structure can be but are not limited to different users and / or products, and the business information between the business nodes represented by the edges connecting the nodes may include interactive behaviors between users, for example, a reply from one user to another.

[0100] Optionally, the relevant data used to determine the graph structure may be obtained from data pre-stored in a server, and matching target data may be determined from the server according to an identifier carried in a data acquisition instruction.

[0101] Exemplarily, multiple training edge sets of a graph structure determined based on scheduled route information of a vehicle and a graph neural network to be trained are obtained from a server. Each edge in the training edge set has two nodes, which can be referred to as a source node and a target node.

[0102] Furthermore, multiple training edge sets of the graph structure can be obtained by processing a pre-stored graph structure to obtain a graph data structure that stores the connection relationships between nodes in the graph structure; splitting the graph data structure to obtain all edge sets of the graph structure, and generating all edge sets based on a normal distribution.

[0103] Step 204: for each training edge set, determine the neighbor node set of each node in each edge.

[0104] Neighbor nodes include first-order neighbors and higher-order neighbors. First-order neighbors can be understood as neighbor nodes directly connected to the node, while higher-order neighbors can be understood as neighbor nodes reachable via multi-hop paths. High-order neighbors provide richer contextual information, helping the model capture complex relationships between nodes. In other words, embedding features calculated by considering higher-order neighbors contain more information than those calculated by considering only low-order neighbors. Furthermore, from a graph structure perspective, the inclusion of higher-order neighbors allows the model to perceive a more comprehensive graph structure. Each node can serve as the root node for all neighbor nodes in its corresponding neighbor node set.

[0105] For example, there are dependencies between nodes in a graph structure. Based on obtaining multiple training edge sets of the graph structure, each node in each training edge set is analyzed based on the dependencies between the nodes to obtain a set of neighboring nodes for each node. Determining the set of neighboring nodes for each node can be accomplished using existing methods and will not be elaborated upon here.

[0106] Step 206 : Determine the neighbor position coding information of each neighbor node in the neighbor node set and the structural position coding information of each neighbor node in the graph structure.

[0107] It should be noted that the transformer module does not distinguish between token positions and requires additional position information. However, in dynamic graph transformer research, only time encoding is often used to provide the transformer with node timing information, while ignoring the node's position in the neighbor set and its position in the graph structure. This results in a lack of input feature information. In order to enable the model to perceive a more comprehensive graph structure and improve prediction accuracy, it is necessary to consider the position of each neighbor node in the neighbor node set and its position in the graph structure.

[0108] Neighbor position encoding information is used to determine the node's position in the neighbor node set (also called the extended neighborhood set). ), combined with the Transformer's attention mechanism, indirectly provides insights into the underlying graph structure. This encoding scheme generates a unique code for each node, making it possible to distinguish identical nodes that appear in the same neighborhood set. The identical nodes that appear in the same neighborhood set can be understood as taking airport A and airport B as an example. When there are multiple edges between airports A and B, the node corresponding to airport B is a neighbor node of the node corresponding to airport A. Then, in the neighbor node set of the node corresponding to airport A, the node corresponding to airport B will appear multiple times. The neighbor position encoding information can be implemented by sorting each neighbor node in each neighbor node set in chronological order and determining the number of the sorted neighbor nodes.

[0109] Structural position encoding information is used to identify the number of hops between neighboring nodes and the root node, directly reflecting the graph structure. Hop count refers to the number of edges required to go from one node to another in the graph structure, and can also be understood as the number of paths. Since the Transformer model only processes sequential input, high-order neighbors are flattened into a sequence. To identify hop counts from this sequence, this method uses a one-hot code to encode the structural position and obtain the corresponding structural position encoding information.

[0110] Step 208: Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between the nodes according to the number of occurrences and the neighbor position coding information, and obtain common neighbor coding information.

[0111] It should be noted that the common neighbor coded information can also be called the extended common neighbor coded information. The original common neighbors are the intersection of the first-order neighbor sets of two nodes. Based on the introduction of higher-order neighbors, the common neighbors need to be expanded to the intersection of the higher-order neighbor sets of two nodes. In other words, the expanded common neighbor set includes the intersection of the first-order neighbor sets of two nodes and the intersection of the higher-order neighbor sets of two nodes. N is a positive integer, which can be, but is not limited to, 2.

[0112] Optionally, information about the extended common neighbors that enumerates the frequency of occurrence of each neighbor in N neighbor node sets is obtained by encoding. Further, assuming that N is 2, the common neighbor set of the first neighbor node set of the source node and the second neighbor node set of the target node is determined; for each neighbor node in the first neighbor node set of the source node, the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node is determined to obtain a corresponding first two-dimensional vector; for each neighbor node in the second neighbor node set of the source node, the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node is determined to obtain a corresponding second two-dimensional vector; Fourier transform is performed on the first two-dimensional vector and the second two-dimensional vector respectively to obtain their respective corresponding first embedded representations and second embedded representations; based on the first embedded representation, the second embedded representation and the neighbor position encoding information, the information of each common neighbor in the common neighbor set is determined to obtain common neighbor encoding information.

[0113] Step 210: Train the graph neural network based on the structural position coding information and the common neighbor coding information. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0114] Among them, the preset training conditions can be understood as model convergence conditions.

[0115] Optionally, for each edge in the training edge set, the source node u and target node v of the edge (u, v, t) at time t are encoded features to obtain the corresponding time coding information, edge coding information, and node coding information of the source node / target node, and all features such as the time coding information, edge coding information, node coding information of the source node / target node, structural position coding information, and common neighbor coding information are connected to obtain splicing features. The obtained splicing features are used to train the graph neural network. When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0116] Step 212: Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain a prediction result of the graph structure to be processed.

[0117] The structure to be processed includes a newly added edge structure, which may be a new route added between Airport A and Airport B at time t.

[0118] Optionally, the graph structure to be processed is input into a trained graph neural network, and the predicted structure of the edges to be processed in the graph structure to be processed can be predicted based on the trained graph neural network, that is, it can be judged whether the newly added operation route is reasonable. Compared with the existing method, the correlation information between the graph structures composed of the flight operation routes can be fully considered, thereby improving the accuracy of the prediction of the newly added operation routes, thereby avoiding the waste of transportation resources and ensuring the safety of flights. Furthermore, the graph neural network trained in the above manner can accurately determine the relationship between users when predicting the graph structure to be processed in the social network analysis scenario. In addition, data can be pushed to users, such as friend push, message push, etc., thereby improving the accuracy of data push. In the course selection scenario, when making predictions based on the graph structure to be processed, the matching degree between students and course units, as well as the attention of course units, etc. can be accurately determined, thereby optimizing the allocation of resources.

[0119] In the above-mentioned graph neural network training method, by obtaining the graph structure representing the scheduled operation route information of the transportation vehicle, the graph structure is split into multiple training edge sets for training the graph neural network according to the characteristics of the graph structure and the dependency relationship between the nodes, that is, the graph neural network to be trained is trained using small batches of training samples. On this basis, by considering the position of each edge node in the training edge set in the graph structure and the position of the node in the neighboring nodes, the corresponding neighbor position coding information and structure position coding information are obtained. The neighbor position coding information and structure position coding information are added on the basis of considering time coding, and the information amount of the input features is enhanced. In order to further improve the representation of the structure by the graph neural network, The representation capability is improved by expanding the common neighbors between nodes, capturing more frequency domain information, and obtaining the common neighbor encoding information. In this way, the information of the input features is enhanced to train the time-series graph neural network, solve the missing structural information in the model, and improve the representation capability of the time-series graph neural network for structural sensitivity, thereby obtaining a trained graph neural network. When processing the graph structure to be processed based on the trained graph neural network, it can better capture the changes of nodes over time and time-sensitive features, and can more comprehensively determine the rationality of the target operation route information in the graph structure to be processed, realize the reasonable planning of the operation route, ensure driving safety and improve the utilization rate of traffic resources.

[0120] Existing TGT models mainly focus on temporal encoding, but they often ignore the key role of graph structure. This oversight may limit the model's ability to capture the structural differences between adjacent nodes, thereby affecting the accuracy of training. Therefore, in an exemplary embodiment, Figure 4 As shown, step 206 includes steps 402 to 406. Among them:

[0121] Step 402 : sort each neighbor node in chronological order according to the time sequence information of each neighbor node in the neighbor node set, encode the sorted neighbor nodes, and obtain neighbor position encoding information of each neighbor node.

[0122] Among them, the timing information can be understood as the interaction time between the neighbor node and the root node. This interaction time can be the flight time of a flight, and the flight time can be the take-off time and / or landing time. Consider a sorted neighbor list in and The list represents the neighbors of a node v at time t, and each neighbor u i With a time t i Associated, indicating that v and u i The interaction time between neighbors. The neighbor position identity encoding of each neighbor is defined as follows:

[0123]

[0124] Step 404: Determine the number of paths between each neighbor node and the root node.

[0125] The number of paths between each neighbor node and the root node can be determined based on the dependency relationship between the nodes. The number of paths can also be called the number of hops.

[0126] Step 406: Determine the structural position coding information of the neighbor node in the graph structure according to the number of paths.

[0127] Where k is used to represent the number of hops, and the structure position identification code is defined as follows:

[0128]

[0129] Among them, p represents the path between the source node u and the target node v, e is the edge, u i Represents a neighbor node.

[0130] In the above embodiment, by combining location and structure information to encode neighbor nodes in the graph, the model can perceive a more comprehensive graph structure, thereby improving the model's predictive performance on graph data. Furthermore, by effectively encoding neighbor node information, the amount of data the model needs to process during training and inference can be reduced, thereby improving the algorithm's computational efficiency and reducing the consumption of computing resources.

[0131] It should be noted that after expanding the high-order neighbors, the common neighbor code needs to be expanded. In an exemplary embodiment, the nodes include a source node and a target node, such as Figure 5As shown, a method for determining common neighbor coding information is provided, including steps 502 to 510, wherein:

[0132] Step 502: Determine a common neighbor set of a first neighbor node set of a source node and a second neighbor node set of a target node.

[0133] It should be noted that for an interaction (u, v, t), the original definition of the neighbor node set of node u at time t is For all nodes v that are directly connected to u via edges and whose time is no later than t: In this embodiment, the multi-hop neighbors of the source node u at time t are defined as the extended neighbor set The expanded neighbor set of the target node The method of determining is the same as that of the source node, which will not be repeated here. Then the common neighbor set of the first neighbor node set of the source node and the second neighbor node set of the target node can be expressed as: the extended common neighbor set The endpoint extends the neighbor set.

[0134] Step 504 : for each neighbor node in the first neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding first two-dimensional vector.

[0135] Step 506 : For each neighbor node in the second neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding second two-dimensional vector.

[0136] Optionally, the extended neighbor set for the source node For each neighbor in and to obtain the first two-dimensional vector Similarly, for For each neighbor in and The number of occurrences in the corresponding second two-dimensional vector

[0137] Step 508: Perform Fourier transform on the first two-dimensional vector and the second two-dimensional vector respectively to obtain a first embedded representation and a second embedded representation corresponding to each.

[0138] Among them, since the Fourier transform can provide additional attention to frequency domain information to enhance the representation ability of the structure and reduce the time consumption of encoding common neighbors, a non-learning Fourier transform method is defined for the first two-dimensional vector and the second two-dimensional vector to obtain their respective first embedding representation and second embedding representation.

[0139] Defining a non-learning Fourier transform method for the first two-dimensional vector and the second two-dimensional vector can be expressed as follows:

[0140]

[0141] Among them, freq(x) represents the The frequency count of the xth element of freq Indicates the size of the frequency vector dimension. Where * can be u, v. Apply the CoE function to and To obtain the first embedding representation for each dimension of and the second embedding representation

[0142] Step 510: Determine information of each common neighbor in the common neighbor set based on the first embedded representation, the second embedded representation, and the neighbor position encoding information to obtain common neighbor encoding information.

[0143] Among them, determine the common neighbor coding information, that is, determine the high-order neighbor coding that distinguishes the final position This can be achieved by:

[0144]

[0145] Among them, * can be u, v, t is time, and NE represents the neighbor location encoding information.

[0146] In the above embodiment, a non-learning Fourier transform is used to capture more frequency domain information, thereby further improving the representation capability of the structure. Moreover, this method does not require learning, and it can avoid back-propagation calculations and improve training efficiency.

[0147] It is understandable that if Figure 6As shown, it is a flow chart of the time cost in one embodiment, including loadfeature, encodeCo, neighbor sample, transformer and construct patches, wherein encodeCo represents common neighbor encoding and neighbor sample represents neighbor sampling. It can be seen that the time cost ring diagram of the training on DyGFormer shows that the time cost of neighbor sampling and the time cost of encoding common neighbors account for a considerable part of its time cost. In order to directly reduce the time cost caused by adding high-order neighbors, the learnable MLP layer (MultilayerPerceptron) for encoding common neighbors is replaced with a non-learning Fourier transform encoding method to reduce the time cost of encoding common neighbors. In order to speed up neighbor sampling, a padded parallel sampler is proposed to efficiently perform neighbor sampling. In an exemplary embodiment, as shown in FIG. Figure 7 As shown, a neighbor sampling method is provided, including steps 702 to 704, wherein:

[0148] Step 702 : For each training edge set, sample the neighbor nodes of each node in each edge in parallel to obtain the first-order neighbor nodes and high-order neighbor nodes of each node.

[0149] Among them, the search method can be to first initialize a zero-filled array neigh for sampling results, fill unfilled positions with zeros, and optimize the array for parallel operation and memory block copying.

[0150] Step 704: Obtain a neighbor node set of each node according to the first-order neighbor nodes and the high-order neighbor nodes.

[0151] For example, the sampling principle of filling the parallel neighbor sampler can be: input a target node list Expansion Factor A graph G in adjacency format,

[0152] Initialize a shape The zero tensor neigh;

[0153] For each expansion factor i from 1 to l: and in parallel for each node j from 1 to b:

[0154]

[0155] end←Binary search

[0156] start←max(0,end-m i );

[0157] Memory block copy Output the sampled neighbors neigh. In this way, the neighbor node set of each node can be obtained.

[0158] Optionally, for each training edge set, its corresponding graph is structured in an adjacency format, neighbors are sorted by time, and neighbor information such as node ID and timestamp is accessed in O(1). The size of the corresponding neighbor array of each node can be preset, and binary search is used to quickly find the earliest neighbor before a specific timestamp. Furthermore, memory block copying is used to transfer data from the earliest valid neighbor to the required number of neighbors in neigh. If the existing neighbors are insufficient, this method copies all available neighbors to neigh, which can be understood as the remaining positions of the array neigh are represented by zeros. This method can be specifically implemented by using C++ to implement thread-level parallelism and direct memory operations.

[0159] In this embodiment, a high-order neighbor sampling technique is used to fill in parallel acceleration, and a binary search is used to quickly find the earliest neighbor before a specific timestamp, achieving a time complexity of O(logn). In addition, a memory-friendly memory block copy operation is used to maximize CPU parallelism and reduce memory access time.

[0160] The above respectively discusses the specific implementation steps of filling parallelism and encoding. The following is a detailed flow chart of the feature processing stage in an exemplary embodiment. Figure 8 As shown, the high-order neighbors of the node are illustrated by taking the second order as an example. Feature processing includes three parts: efficiency optimization, perception enhancement and mapping processing. For each training edge set Batch, each edge can be represented as (u, v, t). By filling the parallel sampler, the 2-hop neighbors of u and v, that is, the neighbor node set, are obtained, and the neighbor position encoding information of each neighbor node in the neighbor node set and the structural position encoding information of each neighbor node in the graph structure are determined; the number of occurrences of each neighbor node in the two neighbor node sets is determined, and the common neighbors between the nodes are expanded according to the number of occurrences and the neighbor position encoding information to obtain the common neighbor encoding information; and the source node u and the target node v of the edge (u, v, t) at time t are encoded features, all features are connected, and through mapping processing, the feature encoding Z_(u, v)^t is obtained, that is, the encoded features are then patched, mapped and concat into a unified feature encoding. Represents the comprehensive state of the edges input to the Transformer module. This approach improves the model's perception of graph structure and node positions by increasing the amount of information in the input features.

[0161] The increase in the amount of information brings about an increase in the amount of calculation, which leads to efficiency problems. In order to reduce the efficiency problems caused by the increase in the amount of information, in an exemplary embodiment, Figure 9As shown, a method for determining a training sample is provided, including steps 902 to 908, wherein:

[0162] Step 902: If the current training of the graph neural network is the first training, all edge sets corresponding to the graph structure are divided based on the normal distribution to obtain multiple training edge sets.

[0163] Among them, multiple training edge sets can be understood as being generated based on small batches. It should be noted that during the first training, all the obtained training edge sets are used to train the graph neural network to be trained, and corresponding training results will be obtained.

[0164] Step 904: For each training from the start of the second training to the end of the i-th training, obtain the previous training result of the graph neural network.

[0165] Step 906: Update the weight of each edge in the graph structure according to the last training result to obtain the updated weight of each edge.

[0166] The updating method may be based on a preset function. For example, each positive sample e(u i ,v i ,t i )∈E B The importance weight w i =P(e) is updated according to the following function:

[0167]

[0168] in, represents the embedding of the source node u of the edge, represents the embedding of the target node v of the edge, γ is a hyperparameter representing the amplitude of the uniform distribution mixed with the adaptive importance distribution, and Pred is the prediction head of the model, which consists of two layers of MLP and is used to finally give the probability that there is an edge connection between two points.

[0169] Step 908 : Extract target edges from all edge sets corresponding to the graph structure according to the updated weight of each edge, and obtain multiple training edge sets.

[0170] For example, a list of importance weights is determined To evaluate the importance level of each edge in the dynamic graph, the list is initialized to a uniform distribution. E_train is the set of training edges. Then, a batch of training edges E are randomly selected according to the corresponding importance score ratio. B After forward propagation, each positive sample e(u i ,v i ,t i )∈E B The importance weight w i=P(e) is updated according to the following function:

[0171] Among them, among them, represents the embedding of the source node u of the edge, represents the embedding of the edge's target node v. γ is a hyperparameter representing the magnitude of a uniform distribution mixed with an adaptive importance distribution. Pred is the model's prediction head, consisting of two MLP layers, which ultimately determines the probability of an edge connection between two points. The initial mini-batch is generated based on a normal distribution. To accelerate convergence, selecting high-confidence positive samples effectively improves accuracy. It is understandable that, considering the cross-entropy loss function, the gradient of the loss with respect to the log-odds is inversely proportional to the log-odds. Larger gradient updates for a sample indicate that the sample is informative but more likely an outlier in the data distribution. To balance the noise and diversity of the training samples, this method can achieve this by adjusting the value of γ. A larger value of γ causes the mini-batch selector to favor samples with greater noise, thereby improving training performance. Since log-odds are typically small, the result is cubed to increase the variance in importance scores between different edges.

[0172] To be applied to Figure 3 As an example, after forward propagation, each positive sample e(u i ,v i ,t i )∈E B The importance weight w i =P(e) is updated according to the above function, and a batch of training edges E are randomly selected according to the corresponding importance score ratio B .

[0173] In the above embodiment, adaptive mini-batches of training samples are used to reduce the number of training cycles, accelerate model convergence, and thus improve training efficiency and computer resource utilization. This approach can eliminate marginal samples that may have poor training effects or even negative effects, thereby accelerating convergence and reducing training time.

[0174] Optionally, in an exemplary embodiment, the graph neural network is trained based on the structural position encoding information and the common neighbor encoding information, including: obtaining the original features of each node, the edge feature encoding information of each edge, and the time encoding information; concatenating the structural position encoding information with the corresponding original features to obtain the node encoding information of each node; concatenating the node encoding information, edge feature encoding information, time encoding information, and common neighbor encoding information and mapping them to obtain mapping features; and using the mapping features to train the Transformer of the graph neural network. The original features can be understood as the node encoding information, and the edge feature encoding information and time encoding information can be implemented in the above-mentioned manner, which will not be elaborated here.

[0175] Alternatively, in an exemplary embodiment, Figure 10 As shown in the figure, a graph neural network training method is provided, which is applied to Figure 1 The terminal in FIG is taken as an example to illustrate, including the following steps 1002 to 1020. Among them:

[0176] Step 1002: Obtain the graph structure and the graph neural network to be trained.

[0177] The graph structure is determined based on the scheduled route information of the transportation vehicle. The nodes of the graph structure correspond to the target objects associated with the scheduled route information. The nodes are connected by edges, and the edges are used to represent the scheduled routes between the target objects.

[0178] Step 1004: If the current training of the graph neural network is the first training, all edge sets corresponding to the graph structure are divided based on the normal distribution to obtain multiple training edge sets.

[0179] Step 1006: Determine the neighbor position coding information of each neighbor node in the neighbor node set and the structural position coding information of each neighbor node in the graph structure.

[0180] Step 1008 : Determine the number of occurrences of each neighbor node in the N neighbor node sets, expand the common neighbors between the nodes according to the number of occurrences and the neighbor position coding information, and obtain common neighbor coding information.

[0181] Step 1010: Train the graph neural network based on the structural position encoding information and the common neighbor encoding information to obtain a training result.

[0182] Step 1012: If the model has not converged, for each training from the start of the second training to the end of the i-th training, for each training edge set, determine the neighbor node set of each node in each edge.

[0183] Step 1014: Update the weight of each edge in the graph structure according to the last training result to obtain the updated weight of each edge.

[0184] Step 1016 , extracting target edges from all edge sets corresponding to the graph structure according to the updated weight of each edge, to obtain multiple training edge sets.

[0185] Step 1018: When the preset training conditions are met, the training is terminated to obtain a trained graph neural network.

[0186] Step 1020: Obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the target operation route information in the graph structure to be processed.

[0187] It should be noted that the specific implementation method of this embodiment can be but is not limited to the above-mentioned limitation method, which will not be described in detail here.

[0188] In the above embodiment, by constructing position-differentiated coding and adding high-order neighbors, the information content of the input features is increased to enhance the model's perception of the graph structure and node positions. On this basis, from the perspective of reducing the single training time, it is proposed to fill the parallel sampler to reduce the sampling time, and optimize the common neighbor coding to reduce the time for encoding common neighbors, and utilize an adaptive small batch generation method based on the training results to accelerate convergence and improve training efficiency. In this way, a trained graph neural network is obtained, which in turn improves the accuracy of the model's prediction in practical applications and improves the utilization rate of traffic resources.

[0189] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0190] Based on the same inventive concept, embodiments of the present application also provide a graph neural network training device for implementing the aforementioned graph neural network training method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more graph neural network training device embodiments provided below can be found in the limitations of the graph neural network training method above and will not be repeated here.

[0191] In an exemplary embodiment, Figure 11 As shown, a graph neural network training device is provided, including: a data acquisition module 1102, a neighbor node processing module 1104, a first encoding module 1106, a second encoding module 1108, a training module 1110 and a prediction module 1112, wherein:

[0192] The data acquisition module 1102 is used to obtain multiple training edge sets of a graph structure and a graph neural network to be trained. The graph structure is determined based on the scheduled operating route information of the vehicle. The nodes of the graph structure correspond to the target objects associated with the scheduled operating route information. The nodes are connected by edges, and the edges are used to represent the scheduled operating routes between the target objects.

[0193] The neighbor node processing module 1104 is configured to determine, for each training edge set, a neighbor node set of each node in each edge.

[0194] The first encoding module 1106 is configured to determine the neighbor position encoding information of each neighbor node in the neighbor node set and the structural position encoding information of each neighbor node in the graph structure.

[0195] The second encoding module 1108 is used to determine the number of occurrences of each neighbor node in the N neighbor node sets, and expand the common neighbors between the nodes according to the number of occurrences and the neighbor position encoding information to obtain common neighbor encoding information.

[0196] The training module 1110 is used to train the graph neural network based on the structural position coding information and the common neighbor coding information, and to terminate the training to obtain a trained graph neural network when the preset training conditions are met.

[0197] The prediction module 1112 is used to obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the graph structure to be processed.

[0198] The above-mentioned graph neural network training device, through the graph structure, splits the graph structure into multiple training edge sets for training the graph neural network according to the characteristics of the graph structure and the dependencies between the nodes, that is, uses small batches of training samples to train the graph neural network to be trained. On this basis, by considering the position of each edge node in the training edge set in the graph structure and the position of the node in the neighboring nodes, the corresponding neighbor position coding information and structure position coding information are obtained. On the basis of considering time coding, the neighbor position coding information and structure position coding information are added to enhance the information content of the input features. In order to further improve the representation ability of the graph neural network for the structure, by expanding the common neighbors between the nodes, more frequency domain information is captured, and common neighbor coding information is obtained. In this way, the information of the input features is enhanced to train the temporal graph neural network, solve the missing structural information in the model, and improve the representation ability of the temporal graph neural network for structural sensitivity, thereby obtaining a trained graph neural network. Based on the trained graph neural network, when processing the graph structure to be processed, it can better capture the changes of nodes over time and time-sensitive characteristics, and can more comprehensively determine the rationality of the business information in the graph structure to be processed, thereby improving the accuracy and reliability of the prediction.

[0199] Optionally, in an exemplary embodiment, the business scenario includes at least one of a traffic prediction scenario, a course selection scenario, and a social network analysis scenario.

[0200] In the case where the business scenario is a traffic prediction scenario, the graph structure is determined based on the scheduled operation route information of the transportation tool. The business nodes of the business scenario include the target objects associated with the scheduled operation route information, and the business information between the business nodes includes the scheduled operation routes between the target objects.

[0201] In the scenario where the business scenario is a course selection scenario, the graph structure is determined based on the access behavior data of the selecting object and the selected object. The business nodes of the business scenario include the selecting object and the selected object, and the business information between the business nodes includes the access behavior between the selecting object and the selected object.

[0202] When the business scenario is a social network analysis scenario, the graph structure is determined by the interactive behavior data in the social network. The business nodes of the business scenario include the target objects that generate the interactive behavior data, and the business information between the business nodes includes the interactive behavior between the target objects.

[0203] Optionally, in an exemplary embodiment, the first encoding module includes a neighbor position encoding module and a structure position encoding module, wherein the neighbor position encoding module is used to sort each neighbor node in chronological order according to the timing information of each neighbor node in the neighbor node set, encode the sorted neighbor nodes, and obtain the neighbor position encoding information of each neighbor node.

[0204] The structural position encoding module is used to determine the number of paths between each neighbor node and the node, and determine the structural position encoding information of the neighbor node in the graph structure according to the number of paths.

[0205] Optionally, in an exemplary embodiment, the apparatus further comprises a sampling module configured to sample neighbor nodes of each node in each edge in parallel for each training edge set, to obtain first-order neighbor nodes and high-order neighbor nodes of each node;

[0206] Optionally, in an exemplary embodiment, the neighbor node processing module 1104 is further configured to obtain a neighbor node set of each node according to the first-order neighbor nodes and the high-order neighbor nodes.

[0207] The second encoding module 1108 is further configured to determine a common neighbor set of the first neighbor node set of the source node and the second neighbor node set of the target node;

[0208] For each neighbor node in the first neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding first two-dimensional vector;

[0209] For each neighbor node in the second neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding second two-dimensional vector;

[0210] Performing Fourier transform on the first two-dimensional vector and the second two-dimensional vector respectively to obtain a first embedded representation and a second embedded representation corresponding to each;

[0211] According to the first embedding representation, the second embedding representation and the neighbor position encoding information, information of each common neighbor in the common neighbor set is determined to obtain common neighbor encoding information.

[0212] Optionally, in an exemplary embodiment, the apparatus further includes a data acquisition module configured to acquire original features of each node, edge feature coding information and time coding information of each edge;

[0213] Optionally, in an exemplary embodiment, the apparatus further comprises a splicing module for splicing the structural position coding information with the corresponding original features to obtain node coding information of each node;

[0214] Optionally, in an exemplary embodiment, the apparatus further comprises a mapping module configured to concatenate and map the node coding information, the edge feature coding information, the time coding information, and the common neighbor coding information to obtain a mapping feature;

[0215] Optionally, in an exemplary embodiment, the training module 1110 is further used to train the Transformer of the graph neural network using mapping features.

[0216] Optionally, in an exemplary embodiment, the apparatus further includes a training sample generation module, the training sample generation module being configured to, if the current training of the graph neural network is the first training, divide all edge sets corresponding to the graph structure based on a normal distribution to obtain multiple training edge sets;

[0217] For each training from the start of the second training to the end of the i-th training, obtain the previous training results of the graph neural network;

[0218] Update the weight of each edge in the graph structure according to the last training result to obtain the updated weight of each edge;

[0219] According to the updated weight of each edge, the target edge is extracted from all edge sets corresponding to the graph structure to obtain multiple training edge sets.

[0220] Optionally, in an exemplary embodiment, the above-mentioned device also includes a graph structure processing module, which is used to process the pre-stored graph structure to obtain a graph data structure that stores the connection relationship between nodes in the graph structure; and split the graph data structure to obtain all edge sets of the graph structure.

[0221] Each module in the graph neural network training device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0222] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 12As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a graph neural network training method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0223] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0224] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0225] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0226] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0227] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0228] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable logic unit (PLC), a data processing logic unit based on quantum computing, an artificial intelligence (AI) processor, and the like.

[0229] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0230] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A graph neural network training method, characterized in that: The method comprises: Obtain multiple training edge sets of a graph structure and a graph neural network to be trained, wherein the graph structure is determined based on a business scenario, the nodes of the graph structure correspond to business nodes of the business scenario, the nodes are connected by edges, and the edges are used to represent business information between the business nodes; if the business scenario is a traffic prediction scenario, the graph structure is determined by scheduled operation route information of a vehicle, the nodes are airports, and the node features of the nodes include at least the airport type, the regional identifier of the area where the airport is located, the airport identifier, and the geographical location of the area where the airport is located, and the edges are operation routes between airports; For each of the training edge sets, determining a neighbor node set of each node in each edge; Determining neighbor position coding information of each neighbor node in the neighbor node set and structural position coding information of each neighbor node in the graph structure; Determine the number of occurrences of each neighbor node in the N neighbor node sets, and expand common neighbors between the nodes according to the number of occurrences and neighbor position coding information to obtain common neighbor coding information; Training the graph neural network according to the structural position encoding information and the common neighbor encoding information, and ending the training to obtain a trained graph neural network when a preset training condition is met; Obtaining a graph structure to be processed, inputting the graph structure to be processed into the trained graph neural network, and obtaining a prediction result of target operation route information in the graph structure to be processed; The determining of the neighbor position coding information of each neighbor node in the neighbor node set and the structural position coding information of each neighbor node in the graph structure includes: According to the time sequence information of each neighbor node in the neighbor node set, each of the neighbor nodes is sorted in chronological order, and the sorted neighbor nodes are encoded to obtain neighbor position encoding information of each neighbor node; Determine the number of paths between each of the neighboring nodes and the node; The structural position coding information of the neighbor node in the graph structure is determined according to the number of paths.

2. The method according to claim 1, characterized in that The step of determining, for each training edge set, a neighbor node set of each node in each edge includes: For each of the training edge sets, sampling the neighbor nodes of each node in each edge in parallel to obtain the first-order neighbor nodes and high-order neighbor nodes of each node; A neighbor node set of each node is obtained according to the first-order neighbor nodes and the high-order neighbor nodes.

3. The method according to claim 1, characterized in that The nodes include a source node and a target node, and determining the number of occurrences of each neighbor node in the N neighbor node sets, and expanding common neighbors between the nodes according to the number of occurrences and neighbor position encoding information to obtain common neighbor encoding information, including: Determine a common neighbor set of a first neighbor node set of the source node and a second neighbor node set of the target node; For each neighbor node in the first neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding first two-dimensional vector; For each neighbor node in the second neighbor node set of the source node, determine the number of times the neighbor node appears in the first neighbor node set and the second neighbor node set of the target node, and obtain a corresponding second two-dimensional vector; Performing Fourier transform on the first two-dimensional vector and the second two-dimensional vector respectively to obtain respective corresponding first embedded representations and second embedded representations; According to the first embedded representation, the second embedded representation and the neighbor position encoding information, information of each common neighbor in the common neighbor set is determined to obtain common neighbor encoding information.

4. The method according to claim 1, wherein The training of the graph neural network according to the structural position encoding information and the common neighbor encoding information includes: Obtain the original features of each node, the edge feature encoding information and time encoding information of each edge; Concatenate the structural position coding information with the corresponding original features to obtain node coding information of each node; The node coding information, the edge feature coding information, the time coding information and the common neighbor coding information are concatenated and mapped to obtain mapping features; The mapping features are used to train the Transformer of the graph neural network.

5. The method according to any one of claims 1 to 4, characterized in that The step of obtaining multiple training edge sets of the graph structure includes: If the current training of the graph neural network is the first training, all edge sets corresponding to the graph structure are divided based on a normal distribution to obtain multiple training edge sets; For each training from the start of the second training to the end of the i-th training, obtain the previous training result of the graph neural network; Updating the weight of each edge in the graph structure according to the last training result to obtain an updated weight for each edge; According to the updated weight of each edge, target edges are extracted from all edge sets corresponding to the graph structure to obtain multiple training edge sets.

6. The method according to claim 5, characterized in that The method further comprises: Processing a pre-stored graph structure to obtain a graph data structure storing connection relationships between nodes in the graph structure; The graph data structure is split to obtain all edge sets of the graph structure.

7. A graph neural network training device, characterized in that: The device comprises: A data acquisition module is configured to acquire multiple training edge sets of a graph structure and a graph neural network to be trained, wherein the graph structure is determined based on a business scenario, the nodes of the graph structure correspond to business nodes of the business scenario, the nodes are connected by edges, and the edges are used to represent business information between the business nodes; when the business scenario is a traffic prediction scenario, the graph structure is determined by information about scheduled operation routes of transportation vehicles, the nodes are airports, and the node features of the nodes include at least the type of airport, the regional identifier of the area where the airport is located, the airport identifier, and the geographical location of the area where the airport is located, and the edges are operation routes between airports; a neighbor node processing module is configured to determine, for each of the training edge sets, a neighbor node set for each node in each edge; a first encoding module, configured to determine the neighbor position encoding information of each neighbor node in the neighbor node set and the structural position encoding information of each neighbor node in the graph structure; wherein the first encoding module includes a neighbor position encoding module and a structural position encoding module, wherein the neighbor position encoding module is configured to sort each neighbor node in chronological order according to the timing information of each neighbor node in the neighbor node set, encode the sorted neighbor nodes, and obtain the neighbor position encoding information of each neighbor node; and the structural position encoding module is configured to determine the number of paths between each neighbor node and the node; and determine the structural position encoding information of the neighbor node in the graph structure according to the number of paths; A second encoding module is configured to determine the number of occurrences of each neighbor node in the N neighbor node sets, and expand the common neighbors between the nodes according to the number of occurrences and the neighbor position encoding information to obtain common neighbor encoding information; A training module, configured to train the graph neural network according to the structural position encoding information and the common neighbor encoding information, and terminate the training to obtain a trained graph neural network when preset training conditions are met; The prediction module is used to obtain the graph structure to be processed, input the graph structure to be processed into the trained graph neural network, and obtain the prediction result of the target operation route information in the graph structure to be processed.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images