Training method and system of graph neural network model

Abstract

The invention provides a graph neural network model training method and system, and the method comprises the steps: obtaining n training data sets, enabling n to be an integer greater than 2, enabling one training data set to correspond to one environment, enabling one environment to correspond to one graph data, enabling the training data sets to be obtained through the construction based on the graph data, and carrying out the training of a graph neural network model according to the n training data sets, the prediction performance of the basic graph neural network model for the connection relation in each environment is determined, a target graph neural network model is obtained through training with the goal of minimizing the prediction performance deviation of multiple sets of environments, the prediction performance deviation of one set of environment represents the difference between the prediction performance of two environments, and the prediction performance deviation of the two sets of environments represents the prediction performance deviation of the two sets of environments. The target graph neural network model is used for generating corresponding graph data in the corresponding environment. Therefore, the expression of the target graph neural network model in different environments is close to the optimal state as much as possible.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and particularly to a method and system for training a graph neural network model. Background Art

[0002] Graph data is one of the basic data structures for describing relationships between entities. In recent years, graph neural networks (GNNs) have proven to be very effective methods for improving performance in many real-world applications, including scenarios such as recommendation, advertising, and financial anomaly detection.

[0003] There is a common problem in these scenarios. Due to noise, selection bias, and other inherent properties, the training and test graphs are always biased. For example, the transfer relationships in modern Internet more reflect the fund flow patterns of young people, while the transfer relationships in traditional banks more reflect those of middle-aged and elderly people.

[0004] This bias in graph distribution contradicts the basic independent and identically distributed assumption (I.I.D. assumption) of the graph neural network model. Directly using a graph neural network model trained in one environment to process graph data in a new environment with a shifted graph structure distribution may lead to a significant decline in model performance.

[0005] It should be noted that the content of the above related technologies is only information known to the inventor personally, and does not mean that the above information has entered the public domain before the filing date of this specification, nor does it mean that it can become the prior art of this specification. Summary of the Invention

[0006] This specification provides a method and system for training a graph neural network model to avoid at least one of the above technical problems.

[0007] In a first aspect, this specification provides a method for training a graph neural network model, including:

[0008] Obtain n training data sets, where n is an integer greater than 2. One training data set corresponds to one environment, one environment corresponds to one graph data, and the training data set is constructed based on the graph data;

[0009] According to the n training data sets, determine the prediction performance of the basic graph neural network model for connection relationships in each environment; and

[0010] Train to obtain a target graph neural network model with the goal of minimizing the prediction performance bias of multiple groups of environments, where the prediction performance bias of a group of environments characterizes the difference between the prediction performances of two environments, and the target graph neural network model is used to generate the corresponding graph data in the corresponding environment.

[0011] In a second aspect, this specification provides a training system for a graph neural network model, including:

[0012] At least one storage medium storing at least one instruction set for training a graph neural network model;

[0013] At least one processor communicatively connected to the at least one storage medium, wherein when the at least one processor runs, it reads the at least one instruction set and executes the method described in the first aspect according to the instructions of the at least one instruction set.

[0014] In a third aspect, this specification provides a computer-readable non-transitory storage medium, wherein at least one instruction set is stored in the computer-readable non-transitory storage medium, and the at least one instruction set is executed by at least one processor to implement the method described in the first aspect.

[0015] As can be seen from the above technical solutions, the training method and system for the graph neural network model provided in this specification determine the prediction performance of the connection relationships corresponding to each environment by using a basic graph neural network model based on the training data corresponding to each environment, and train a target graph neural network model with the goal of minimizing the prediction performance deviation of multiple groups of environments. So that the target graph neural network model can maintain consistent prediction performance in each environment, that is, reduce the difference in prediction performance between different environments. This means that, no matter in which environment, the performance of the target graph neural network model is as close as possible to the optimal state. Therefore, the generalization ability and general applicability of the target graph neural network model can be improved, and the accuracy and reliability of the application of the target graph neural network model can be improved.

[0016] Other functions of the training method and system for the graph neural network model provided in this specification will be partially listed in the following description. The creative aspects of the training method and system for the graph neural network model provided in this specification can be fully explained by practicing or using the methods, devices, and combinations described in the detailed examples below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic diagram of the application scenario of the training method for the graph neural network model provided in the embodiments of this specification;

[0019] Figure 2 It is a schematic structural diagram of a training system for a graph neural network model provided by an embodiment of this specification;

[0020] Figure 3 It is a schematic flowchart of a training method for a graph neural network model provided by an embodiment of this specification;

[0021] Figure 4 It is a schematic diagram of the principle of a training method for a graph neural network model provided by an embodiment of this specification;

[0022] Figure 5 It is a schematic flowchart of a training method for a graph neural network model provided by another embodiment of this specification;

[0023] Figure 6 It is a schematic diagram of the principle of a training method for a graph neural network model provided by an embodiment of this specification. Detailed implementation manners

[0024] Here, exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0025] It should be understood that the terms "include" and "have" and any variations thereof in the embodiments of this specification are intended to cover but not be exclusive of inclusion. For example, a product or device including a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0026] The term "and / or" in the embodiments of this specification describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0027] The term "plural" in the embodiments of this specification means two or more, and other quantifiers are similar thereto.

[0028] The terms "first", "second", "third", "initial", "target", etc. in this specification are used to distinguish similar or like objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be interchanged under appropriate circumstances, for example, it is possible to implement in an order other than those given in the illustrations or descriptions of the embodiments of this specification.

[0029] The term "unit / module" used in this specification refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to that element.

[0030] To avoid at least one of the technical problems mentioned in the above background art, this specification presents a technically creative concept: First, obtain the graph data corresponding to at least 3 environments respectively, and construct the corresponding training data according to the graph data corresponding to each environment. Then, use multiple training data sets as the input data of the basic graph neural network model, so that the basic graph neural network model determines the prediction results of the edges with connection relationships between nodes in each environment. Then, based on the obtained prediction results corresponding to each environment, with the goal of minimizing the prediction performance deviation between two or more groups of environments, and to narrow the prediction performance deviation between different environments as much as possible, train to obtain the target graph neural network model.

[0031] Among them, graph data can be understood as a data structure used to represent objects (nodes or vertices) and their connecting edges.

[0032] Exemplarily, a node (Node / Vertex) is the basic unit in graph data, representing an entity or object. For example, in a social network, each user can be regarded as a node.

[0033] An edge is a line segment connecting two nodes, representing the connection relationship (or called the association relationship) between nodes. For example, in a social network, if two users are friends, there is an edge between them.

[0034] The technical solution provided in this specification is implemented based on the above technical concept. From the description of the above technical concept, it can be seen that in the technical solution provided in this specification, by aiming to minimize the prediction performance deviation between two or more environments, the prediction performance deviation between different environments can be minimized as much as possible. Therefore, using a graph neural network model trained in a certain environment to process graph data in a new environment with graph structure distribution shift will not cause a significant decline in the model performance. That is, through the technical solution provided in this specification, a trained graph neural network model can be stably migrated and deployed in different environments, and generate relatively stable and highly reliable graph data in the corresponding environments.

[0035] For the convenience of readers to understand this specification, the application scenarios of this specification will be introduced below.

[0036] The technical solution provided in this specification is applicable to scenarios where a graph neural network model needs to be trained. For example, the technical solution provided in this specification can be applied to scenarios such as social network analysis, recommendation systems, financial risk control, and bioinformatics that require the use of graph neural network models.

[0037] Among them, social network analysis includes: using graph models to analyze interpersonal networks, identifying key figures, community discovery, etc. Recommendation systems include: constructing a relationship graph between items based on user behavior, and recommending relevant products or content through graph algorithms, etc. Financial risk control includes: detecting fraud or assessing credit risks by analyzing transaction networks, etc. Bioinformatics includes: studying biological processes such as protein interaction networks and gene regulatory networks, etc.

[0038] Taking the scenario of the above recommendation system as an example:

[0039] Through the technical solution provided in this specification, a graph neural network model deployed in a recommendation system can be trained. Based on this graph neural network model, the recommendation system can generate graph data in the recommendation scenario, and based on this graph data, recommend movies that friends with connection relationships like to watch each other, etc.

[0040] For the description of the application of the technical solution provided in this specification to other scenarios, reference can be made to the description of the recommendation scenario in the above example, and will not be listed one by one here.

[0041] Figure 1 It is a schematic diagram of the application scenario of the training method of the graph neural network model in the embodiments of this specification (hereinafter referred to as the training method). Among them, the training method of this specification can be applied to Figure 1 the scenario 100 as shown. As Figure 1 shown, the scenario 100 may include a target user 101, a client 102, a server 103, and a network 104.

[0042] The target user 101 can be the user who triggers the training of the graph neural network model. For example, the target user 101 can perform a target operation on the client 102 to trigger the training of the graph neural network model.

[0043] The client 102 can be an electronic device that provides an interaction function to the target user 101. For example, the client 102 can provide an interaction interface to the target user 101, and the target user 101 can perform interaction operations on the interaction page. In some embodiments, in response to detecting an operation for training the graph neural network model triggered by the target user 101, the client 102 executes the training method described in this specification. At this time, the client 102 may store data or instructions for executing the training method described in this specification and may execute or be used to execute the data or instructions. In some embodiments, the client 102 may include a hardware device with data information processing capabilities and necessary programs for driving the hardware device to work to execute the training method described in this specification.

[0044] In some embodiments, the client 102 may include a mobile device, a tablet computer, a laptop computer, a built-in device of a motor vehicle, or the like, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home device may include a smart TV, a desktop computer, etc., or any combination. In some embodiments, the smart mobile device may include a smart phone, a personal digital assistant, a gaming device, a navigation device, etc., or any combination thereof. In some embodiments, the built-in device in a motor vehicle may include an in-vehicle computer, an in-vehicle TV, etc.

[0045] In some embodiments, the client 102 may be installed with one or more applications (APPs). The APP can provide the target user 101 with the ability to interact with the outside world through the network 104 and an interface. The APP includes, but is not limited to: web browser type APP programs, search type APP programs, chat type APP programs, shopping type APP programs, video type APP programs, financial management type APP programs, instant messaging tools, email clients, social platform software, and so on.

[0046] As Figure 1 shown, the client 102 can be communicatively connected to the server 103. Among them, the server 103 can be communicatively connected to one client 102 or multiple clients 102. In some embodiments, the client 102 can interact with the server 103 through the network 104 to receive or send messages, etc.

[0047] Server 103 can be a server that provides various services. For example, server 103 can be a cloud server or a local server. Server 103 can be communicatively connected to a client 102 and receive data sent by the client 102, or can be communicatively connected to multiple clients 102 and receive data sent by each of the clients 102 respectively.

[0048] In some embodiments, the training method described in this specification can be executed on server 103. At this time, server 103 can store data or instructions for executing the training method described in this specification, and can execute or be used to execute the data or instructions. Server 103 can include a hardware device with data information processing capabilities and necessary programs for driving the hardware device to work.

[0049] Network 104 is a medium for providing a communication connection between client 102 and server 103. Network 104 can facilitate the exchange of information or data. As Figure 1 shown, client 102 and server 103 can be respectively connected to network 104 and transmit information or data to each other through network 104.

[0050] In some embodiments, network 104 can be any type of wired or wireless network, or a combination thereof. For example, network 104 can include a cable network, a wired network, an optical fiber network, a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network TM, a short-range wireless network (ZigBee TM), a near field communication (NFC) network, or a similar network.

[0051] In some embodiments, network 104 can include one or more network access points. For example, network 104 can include a wired or wireless network access point, such as a base station or an Internet exchange point, through which one or more components of client 102 and server 103 can be connected to network 104 to exchange data or information.

[0052] It is worth noting that Figure 1The numbers of the client 102, the server 103, and the network 104 in [the system] are merely illustrative. According to implementation requirements, there can be any number of clients 102, servers 103, and networks 104. Moreover, the training method provided in this specification can be executed entirely on the client 102, entirely on the server 103, or partially on the client 102 and partially on the server 103.

[0053] That is to say, Figure 1 and the above description regarding Figure 1 is only used to exemplarily illustrate the application scenarios to which the training method in this specification may apply, and should not be construed as a limitation on the application scenarios.

[0054] Figure 2 FIG. shows a hardware structure diagram of a training system 200 provided according to an embodiment of this specification. The training system 200 can execute the training method described in this specification. The training method is introduced in other parts of this specification. When the training method is executed on the client 102, the training system 200 can be the client 102. When the training method is executed on the server 103, the training system 200 can be the server 103. When the training method is partially executed on the client 102 and partially executed on the server 103, the training system 200 can be a system including the client 102 and the server 103.

[0055] As Figure 2 shown, the training system 200 can include at least one storage medium 203 and at least one processor 202. In some embodiments, the training system 200 can further include a communication port 204 and an internal communication bus 201. The training system 200 can further include I / O components 205.

[0056] The internal communication bus 201 can connect different system components. For example, the internal communication bus 201 can connect the storage medium 203, the processor 202, the communication port 204, and the I / O components 205.

[0057] The I / O components 205 support input / output between the training system 200 and other components.

[0058] The communication port 204 is used for data communication between the training system 200 and the outside world. For example, the communication port 204 can be used for data communication between the training system 200 and the network 104. The communication port 204 can be a wired communication port or a wireless communication port.

[0059] The storage medium 203 may include a data storage device. The data storage device may be a non-transitory storage medium or a transitory storage medium. For example, the data storage device may include one or more of a magnetic disk 2031, a read-only storage medium (ROM) 2032, or a random access storage medium (RAM) 2033. The storage medium 203 further includes at least one instruction set stored in the data storage device. The instruction set includes computer program code, and the computer program code may include programs, routines, objects, components, data structures, procedures, modules, etc. for executing the training method provided in this specification.

[0060] At least one processor 202 may be communicatively connected to at least one storage medium 203. The at least one processor 202 is configured to execute the above at least one instruction set. When the training system 200 runs, the at least one processor 202 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the training method provided in this specification. The processor 202 may execute all steps included in the training method. The processor 202 may be in the form of one or more processors. In some embodiments, the processor 202 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, etc., or any combination thereof.

[0061] For illustrative purposes only, only one processor 202 is shown in the training system 200 in the drawings. However, it should be noted that the training system 200 in this specification may further include multiple processors. Therefore, the operations and / or method steps disclosed in this specification may be executed by one processor or jointly executed by multiple processors. For example, if it is described in this specification that the processor 202 of the training system 200 executes step A and step B, it should be understood that step A and step B may also be jointly or separately executed by two different processors 202 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0062] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of the training method of the graph neural network model provided in an embodiment of this specification. Among them, Figure 3The execution entity of the training method shown can be a training system. For the description of the training system, reference can be made to the above examples and will not be elaborated here.

[0063] As Figure 3 shown, the method includes the following S301 to S303:

[0064] S301: Obtain n training data sets, where n is an integer greater than 2. One training data set corresponds to one environment, one environment corresponds to one graph data, and the training data set is constructed based on the graph data.

[0065] Exemplarily, n can be 3 or a larger value. That is, the number of training data sets can be 3, 4 or more. This embodiment does not make a limitation, and it can be determined by the training system based on requirements, historical records, experiments and other means.

[0066] For example, taking the accuracy requirement as an example, for a relatively high accuracy requirement, the training system can determine n as a relatively large number; conversely, for a relatively low accuracy requirement, the training system can determine n as a relatively small number.

[0067] The training data set can be understood as a data set for training a graph neural network model. The training data set can include training samples so that the training system can specifically train the graph neural network model based on the training samples.

[0068] Among them, the n training data sets are obtained by the training system from n environments. For example, for each of the n environments, the training system can obtain the graph data in the environment and construct the training data set corresponding to the environment based on the graph data. And so on, so as to obtain the graph data corresponding to each environment respectively, and further obtain the training data sets corresponding to each environment respectively.

[0069] For example, combined with Figure 4 it can be known that the graph data in environment 1 is graph data 1, and so on, until the graph data in environment n is graph data n. Correspondingly, the training system can construct training data set 1 based on graph data 1 in environment 1, and so on, the training system can construct training data set n based on graph data n in environment n.

[0070] Regarding the training system to obtain n training data sets, it can be implemented by the following examples:

[0071] In one example, the training system can be connected to a data acquisition device and receive the n training data sets sent by the data acquisition device.

[0072] In another example, the training system can provide a tool for loading data, and the user can transmit the n training data sets to the training system through this tool for loading data.

[0073] Among them, the tool for loading data can be an interface for connecting to an external device, such as an interface for connecting to other storage devices, and data transmitted by the external device can be obtained through this interface; the tool for loading data can also be a display device. For example, the training system can output an interface for the data loading function on the display device, and the user can import n training data sets into the training system through this interface.

[0074] S302: Determine the prediction performance of the connection relationship of the basic graph neural network model in each environment according to the n training data sets.

[0075] The basic graph neural network model can be understood as a graph neural network model trained by the training system based on the n training data sets. That is, the graph neural network model to be trained.

[0076] The prediction performance can also be called the prediction effect. The prediction performance of the basic graph neural network model in a certain environment can characterize the performance of the basic graph neural network model in predicting the connection relationship between nodes in the graph data of this environment, such as prediction accuracy, etc.

[0077] Exemplarily, this step can be understood as that for each training data set in the n training data sets, the training system can predict the prediction performance of the basic graph neural network model based on this training data set. Among them, the prediction performance can specifically be the prediction performance of the connection relationship between nodes in the graph data in the environment corresponding to this training data set.

[0078] Continue to refer to Figure 4 , the training system can use the basic graph neural network to determine the prediction performance 1 corresponding to environment 1 according to training data set 1. And so on, the training system can use the basic graph neural network to determine the prediction performance n corresponding to environment n according to training data set n.

[0079] S303: Train to obtain a target graph neural network model with the goal of minimizing the prediction performance deviation of multiple groups of environments. Among them, the prediction performance deviation of a group of environments characterizes the difference between the prediction performances of two environments, and the target graph neural network model is used to generate corresponding graph data in the corresponding environment.

[0080] The prediction performance deviation of a group of environments can be understood as the difference in the performance of the basic graph neural network model in determining prediction results in any two different environments.

[0081] For example, continue to combine the above example and Figure 4, the prediction effect of the basic graph neural network model in environment 1 may be different from that in environment n. The difference between these two effects can be understood as the prediction performance deviation 1-n of a group of environments composed of environment 1 and environment n.

[0082] Similarly, the prediction effect of the basic graph neural network model in environment 2 may be different from that in environment n. The difference between these two effects can be understood as the prediction performance deviation 2-n of a group of environments composed of environment 2 and environment n.

[0083] Correspondingly, the training system adjusts the parameters of the basic graph neural network model with the goal of minimizing the difference between the prediction performance deviation 1-n and the prediction performance deviation 2-n, so as to finally obtain the target graph neural network model.

[0084] Therefore, relatively speaking, the target graph neural network model can maintain consistent prediction performance in each environment, that is, reduce the difference in prediction performance between different environments. This means that, no matter in which environment, the performance of the target graph neural network model is as close to the best state as possible. Therefore, the generalization ability and general applicability of the target graph neural network model can be improved, and the accuracy and reliability of the application of the target graph neural network model can be improved.

[0085] Correspondingly, when generating graph data in the corresponding environment based on the target graph neural network model, the effectiveness and reliability of the generated graph data can be improved.

[0086] To facilitate readers to further deeply understand the technical solution provided in this specification, the technical solution provided in this specification will be elaborated in detail in combination with Figure 5 and Figure 6 here. Figure 5 is a schematic flowchart of the training method of the graph neural network model provided in another embodiment of this specification. As Figure 5 shown, this method includes the following S501 to S503:

[0087] S501: Obtain the graph data corresponding to each environment from n environments, and construct the corresponding training dataset according to the n graph data.

[0088] It can be understood that, in order to avoid cumbersome statements, for the same or similar technical features in this embodiment and the above embodiment, this embodiment will not be elaborated again.

[0089] For example, regarding the implementation principle of S501, reference can be made to the relevant description of S301 above, which will not be elaborated here.

[0090] For the convenience of readers' further understanding, this embodiment takes n = 3 as an example to demonstratively elaborate on the technical solution provided in this specification.

[0091] As Figure 6 shown, the number of environments is 3, and they are respectively: E k , E i , E j . One environment includes one graph data.

[0092] In some embodiments, for any training data set, any training data set includes positive sample subgraph structure pairs and negative sample subgraph structure pairs.

[0093] The positive sample subgraph structure pair includes a subgraph structure with the source node as the central node and nodes having a connection relationship with the source node, and also includes a subgraph structure with the target node as the central node and nodes having a connection relationship with the target node.

[0094] The negative sample subgraph structure pair includes a subgraph structure with the source node as the central node and nodes having no connection relationship with the source node, and also includes a subgraph structure with the target node as the central node and nodes having no connection relationship with the target node.

[0095] The source node and the target node are respectively nodes in the same nodes of n training data sets. The source node can also be called the starting node, and the target node can also be called the ending node.

[0096] It can be understood that the source nodes in the positive sample subgraph structure and the negative sample subgraph structure may be the same or different. The target nodes in the positive sample subgraph structure and the negative sample subgraph structure may be the same or different.

[0097] Exemplarily, for any training data set among n training data sets, this training data set includes positive and negative sample pairs, and specifically positive and negative sample subgraph structure pairs (positive sample subgraph structure pairs and negative sample subgraph structure pairs).

[0098] That is to say, the input of the basic graph neural network model is in the form of subgraph structure pairs. The subgraph structure can be understood as a graph data composed of at least some nodes and edges in the graph data.

[0099] For example, taking the positive sample subgraph structure pair as an example, the input of the basic graph neural network model includes a subgraph structure pair composed of a subgraph structure with the source node as the central node and nodes having a connection relationship with the source node, and a subgraph structure with the target node as the central node and nodes having a connection relationship with the target node.

[0100] As Figure 6 shown, taking environment E kFor example, in a pair of positive sample subgraph structures, the subgraph structure with the source node as the central node and the nodes connected to the source node is represented by a dot, and the pair of subgraph structures composed of the subgraph structure with the target node as the central node and the nodes connected to the target node is also represented by a dot.

[0101] Similarly, in a pair of negative sample subgraph structures, the subgraph structure with the source node as the central node and the nodes not connected to the source node is represented by a dot, and the pair of subgraph structures composed of the subgraph structure with the target node as the central node and the nodes not connected to the target node is also represented by a dot.

[0102] Relatively speaking, in this embodiment, the subgraph structure usually includes multiple nodes and edges associated with the central node. By using the subgraph structure to determine the input of the basic graph neural network model, the training system can enable the graph neural network model to make predictions based on the information of the central node itself and the information associated with it, thereby improving the accuracy and reliability of the prediction. In addition, by combining positive and negative sample pairs for training, the training system can further improve the learning effect of the basic graph neural network model, thereby further improving the effectiveness and reliability of the training.

[0103] In some embodiments, for the training system to obtain positive sample subgraph structure pairs and negative sample subgraph structure pairs, the following steps 1 to 4 may be included:

[0104] Step 1: Obtain the graph data corresponding to any training data set.

[0105] Exemplarily, continuing to combine the above example and Figure 6 , the training system can obtain the graph data in environment E k .

[0106] Step 2: Use random walk to generate multiple paths for each node in the graph data corresponding to any training data.

[0107] Among them, random walk can be understood as a probability-based path generation method. It starts from a certain starting node in the graph data and randomly selects the next node to visit according to certain rules (such as based on the edge weights or the distribution of neighbor nodes, etc.), gradually forming a path.

[0108] Continuing to combine the above example and Figure 6 , after obtaining the graph data in environment E k , the training system can use random walk to generate corresponding multiple paths for each node in this graph data.

[0109] For example, the graph data includes a source node and a target node. The training system can generate multiple corresponding paths for the source node based on random walk, and can also generate multiple corresponding paths for the target node based on random walk.

[0110] Step 3: Determine a positive sample subgraph structure pair according to two nodes within a first preset distance in the multiple paths, where the first preset distance is 3-hop neighbors.

[0111] Exemplarily, the training system can regard any two nodes within a distance of ω or less in the path as positive sample data, and determine a subgraph structure pair according to the positive sample data. And ω = 3.

[0112] For example, combining the above example Figure 6 , after obtaining the paths based on random walk, the training system can obtain positive samples based on the first preset distance, and construct a positive sample subgraph structure pair based on the positive samples.

[0113] Relatively speaking, when the training system determines the positive sample subgraph structure pair based on ω = 3, the positive sample subgraph structure pair can include both first-order neighbors and high-order neighbors, so as to improve the richness of information in subsequent training.

[0114] Step 4: Determine a negative sample subgraph structure pair according to two nodes that are 2-hop neighbors or 3-hop neighbors and not directly connected in the multiple paths.

[0115] Combining the above example Figure 6 , the training system can randomly sample corresponding neighbor nodes and 2 / 3-hop non-directly connected neighbors to obtain negative samples, and construct a negative sample subgraph structure pair based on the negative samples, so that the negative sample subgraph structure pair can accurately represent the information without a connection relationship, thereby improving the effectiveness and reliability of the contrastive learning of positive and negative sample subgraph structures in subsequent training.

[0116] S502: Use n training data sets as input data for the basic graph neural network model to determine the prediction result of the edge where there is a connection relationship between the source node and the target node in each environment, and the prediction performance is characterized by the prediction result.

[0117] Similarly, for the content already described in the above example in S502, such as prediction performance, etc., reference can be made to the above example and will not be elaborated here.

[0118] The prediction result can be understood as the result that there is a representation between the source node and the target node and there is a connection relationship between the source node and the target node.

[0119] Exemplarily, this step can be understood as follows. For each training dataset among the n training datasets, the training system can input the training dataset into the basic graph neural network model. The basic graph neural network model can use the training dataset as input data and predict the possibility of an edge existing between the source node and the target node in the environment corresponding to the training dataset, obtaining a prediction result.

[0120] That is, the training system can use the basic graph neural network model to predict each training dataset corresponding to its respective environment to obtain the prediction result corresponding to each environment.

[0121] For example, in combination with the above example, the training system can use the basic graph neural network to determine the prediction result 1 corresponding to environment 1 according to training dataset 1. The prediction result 1 represents the possibility of an edge existing between the source node and the target node in graph data 1 of environment 1. By analogy, the training system can use the basic graph neural network to determine the prediction result n corresponding to environment n according to training dataset n. The prediction result n represents the possibility of an edge existing between the source node and the target node in graph data n of environment n.

[0122] Based on the above example analysis, the input data of the basic graph neural network model can be a positive / negative sample subgraph structure pair. That is, the input data is in the form of a subgraph structure, rather than a single node.

[0123] Correspondingly, in some embodiments, S502 may include the following steps 21 and 22:

[0124] Step 21: For any training dataset, based on the basic graph neural network model, determine the vector representation (Embedding) corresponding to each node in the arbitrary training dataset, where the vector representation corresponding to any node among the nodes is used to represent the aggregation feature of the neighbor information of the arbitrary node in the current layer and the information of the arbitrary node in different layers.

[0125] Continuing to combine the above example and Figure 6 , the basic graph neural network model includes an encoder (Encoder). The input data of the encoder ( Figure 6 "GNN Encoder" shown in k ) is the training dataset, such as the positive sample subgraph structure pair and the negative sample subgraph structure pair corresponding to environment E.

[0126] Taking the positive sample subgraph structure pair as an example, the positive sample subgraph structure pair includes a subgraph structure with the source node as the central node and the nodes connected to the source node. For this subgraph structure, the encoder can aggregate the neighbor information of the source node in the current layer and the information of the source node in different layers to obtain the aggregated information (i.e., the aggregation feature).

[0127] In some embodiments, the vector representation of source node i at layer l can be represented by Equation 1, Equation 1:

[0128]

[0129] where W (l-1) is the weight matrix of the (l - 1)th layer, N i ∪{i} is the set of neighbors of source node i plus source node i itself, is the attention mechanism, used to measure the correlation between source node i and node j, is the vector representation of node j at the (l - 1)th layer, and tanh is the activation function for non - linear transformation.

[0130] The final vector representation h i (i.e., the aggregated feature) of source node i can be represented by Equation 2, Equation 2:

[0131]

[0132] where is the vector representation of source node i across all layers, is the function for integrating information from different layers, and Φ is the set of parameters for adjusting the integration process.

[0133] Step 22: Determine the prediction result of any training data set according to the vector representation corresponding to each node.

[0134] Correspondingly, the basic graph neural network model can make a prediction based on the vector representation determined in Step 21 to obtain the corresponding prediction result.

[0135] Combining the above analysis of Step 21 and Step 22, it can be seen that in this embodiment, the training system aggregates the neighbor information of the current layer and the information across different layers to obtain the aggregated feature, and determines the prediction result based on the aggregated feature. This is equivalent to fully combining the full information of the node itself and the information of related nodes for prediction. Therefore, the accuracy and reliability of the prediction can be improved.

[0136] Combining the above example analysis, it can be seen that the input data of the basic graph neural network model is specifically the positive / negative sample sub - graph structure pair to determine the prediction result through contrastive learning.

[0137] Correspondingly, in some embodiments, S502 may include the following Steps 31 and 32:

[0138] Step 31: Based on the basic graph neural network model, respectively determine the first similarity between connected nodes in the positive sample sub - graph structure and the second similarity between unconnected nodes in the negative sample sub - graph structure.

[0139] Combined with the above example, the basic graph neural network model can obtain the aggregated features of the source node and the target node in the positive / negative sample subgraph structure.

[0140] Correspondingly, taking the sample subgraph structure as an example, the basic graph neural network model can determine the similarity (i.e., the first similarity) between the source node and the target node based on the aggregated features of the corresponding source node and target node.

[0141] Similarly, the basic graph neural network model can also determine the similarity (i.e., the second similarity) between the source node and the target node based on the aggregated features of the source node and the target node in the negative sample subgraph structure.

[0142] Step 32: Determine the ratio between the first similarity and the second similarity as the prediction result of any training dataset.

[0143] Correspondingly, the training system can enable the basic graph neural network model to learn through contrastive learning: the similarity between nodes with a connection relationship becomes larger and larger, while the similarity between nodes without a connection relationship becomes smaller and smaller. Thereby improving the effectiveness and reliability of the training of the basic graph neural network model.

[0144] S503: Train to obtain the target graph neural network model with the goal of minimizing the difference in the deviation between multiple groups of prediction results.

[0145] In some embodiments, the deviation between a group of prediction results is the absolute value of the difference between the prediction results of any two environments; the prediction result of a training dataset represents the prediction probability of an edge with a connection relationship between the source node and the target node in the corresponding environment. S503 may include: determining the absolute value of the difference between at least two groups of prediction probabilities, and training to obtain the target graph neural network model with the goal of minimizing the difference between the determined at least two absolute values.

[0146] Continuing to combine the above example, for environment E k the corresponding prediction probability is P k , for environment E i the corresponding prediction probability is P i , for environment E j the corresponding prediction probability is P j . There are a total of 3 groups of deviations between the prediction results, and they are respectively |P k - P i |, |P k - P j |, |P i - P j |.

[0147] Correspondingly, S503 can be understood as the training system minimizing the difference between |P k -P i | and |P k -P j | as the goal to train the target graph neural network model.

[0148] Based on the above analysis, in this embodiment, the training system first determines the absolute values of the differences between multiple sets of prediction probabilities, and then trains with the goal of minimizing the differences between multiple sets of absolute values. This can enable the basic graph neural network model to learn the ability of prediction performance in different environments. Thus, the deployment of the target graph neural network model in different environments can be generalized, and the general applicability of the target graph neural network model in different environments can be improved.

[0149] In some embodiments, S503 may include the following steps 41 to 43:

[0150] Step 41: Determine the graph structure learning loss function between multiple prediction results and the preset true value respectively.

[0151] Exemplarily, for each of the n prediction results, the training system can determine the loss function between this prediction result and the preset true value (in order to be distinguished from other loss functions, it can be called the graph structure learning loss function).

[0152] Continuing to combine the above example and Figure 6 and environment E k The corresponding prediction result is The preset true value is The training system can calculate the corresponding graph structure learning loss function based on Equation 3 Equation 3:

[0153]

[0154] Among them, s is the source node and t is the target node.

[0155] Step 42: For any set of training data sets, determine the cross-environment loss function between the prediction results of different training data sets in any set.

[0156] Among them, the cross-environment loss function is used to characterize the cross-entropy loss of the prediction results of the edge existing between the source node and the target node in two different environments. Correspondingly, the cross-environment loss function can also be called the stability loss function to characterize the loss function of the prediction stability of the basic graph neural network model in different environments.

[0157] Continuing to combine the above example and Figure 6 and environment E k and environment Ej The cross-entropy loss between can be represented by Equation 4, Equation 4:

[0158]

[0159] where is the preset true value in environment E j in the preset true value.

[0160] The training system can calculate the cross-environment loss function based on Equation 7, Equation 7:

[0161]

[0162] Step 43: With the goal of minimizing the multi-group graph structure learning loss function and the cross-environment loss function, train to obtain the target graph neural network model.

[0163] For example, after calculating the loss functions of two dimensions (the graph structure learning loss function and the cross-environment loss function), the training system can minimize the loss functions of these two dimensions to train the basic graph neural network model, so as to obtain the target graph neural network model.

[0164] Combined with the above analysis of Steps 41 to 43, it can be seen that in this embodiment, the training system trains with the goal of minimizing the loss functions of two dimensions, which can not only enable the basic graph neural network to learn the information in the graph structure dimension (such as being more accurate in connection relationships), but also enable the basic graph neural network model to learn that the performance deviation of predicting connection relationships in different environments is as small as possible.

[0165] In some embodiments, Step 43 may include the following Steps 431 and 432:

[0166] Step 431: Construct the sum of functions between the multi-group graph structure learning loss function and the cross-environment loss function as the goal.

[0167] Step 432: According to the sum of functions and the preset regularization term, construct the total objective function.

[0168] Exemplarily, the training system can determine the total objective function L based on Equation 8 SRG , Equation 8:

[0169]

[0170] where N is the total number of environments, λ and a are hyperparameters, and L reg is the preset regularization term.

[0171] Step 433: With the goal of minimizing the total objective function, train to obtain the target graph neural network model.

[0172] Combined with the above example, the training system can minimize L SRG as the goal, adjust the parameters of the basic graph neural network model, such as the parameters of the encoder in the basic graph neural network model, to train the target graph neural network model.

[0173] Combined with the above analysis of steps 431 to 433, it can be seen that in this embodiment, the training system determines the final objective function by combining the loss function of the graph structure learning dimension, the loss function of the prediction stability dimension of the basic graph neural network model in different environments, and the regularization term, and trains the target graph neural network with the goal of minimizing this objective function. It can not only enable the target graph neural network model to have the ability of graph structure learning, such as the ability to accurately predict the connection relationship between corresponding nodes in the corresponding environment; but also enable the target graph neural network model to have the ability to predict the stability of the connection relationship in different environments; and the regularization term can be used to avoid overfitting to improve the generalization ability of the target graph neural network model.

[0174] It should be noted that the above examples are only used to exemplarily illustrate the possible implementation manners of the training method in this specification, and should not be construed as a limitation on the implementation manners of the training method in this specification. Exemplarily, on the basis of the above technical concept, some of the above technical features can be combined to obtain a new embodiment; new technical features can also be added on the basis of the above example to obtain a new embodiment; some technical features can also be reduced on the basis of the above example to obtain a new embodiment; some of the technical features in the above example can be replaced with other technical features; or the order of some of the technical features in the above example can be adjusted to obtain a new embodiment, etc., which will not be listed one by one here.

[0175] According to the above technical concept, this specification also provides a computer-readable non-transitory storage medium, in which at least one instruction set is stored, and when the at least one instruction set is executed by a processor, the steps of the training method described in this specification are implemented.

[0176] In some possible embodiments, aspects of this specification can also be implemented in the form of a program product, which includes program code. When the program product runs on the training system 200, the program code is used to cause the training system 200 to execute the steps of the training method described in this specification. The program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) that includes program code and can run on the training system 200. However, the program product of this specification is not limited to this. In this specification, a readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system. The program product can be any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted with any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above. The program code for performing the operations of this specification can be written in any combination of one or more programming languages, which include object-oriented programming languages - such as Java, C++, etc., and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the training system 200, partially on the training system 200, executed as an independent software package, partially on the training system 200 and partially on a remote training system, or entirely on the remote training system 200.

[0177] The above description has been made of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require a particular order or a sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0178] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and is not necessarily limiting. Although not explicitly stated herein, those skilled in the art will understand that this specification is intended to encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0179] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" mean that the specific features, structures, or characteristics described in connection with that embodiment may be included in at least one embodiment of this specification. Thus, it should be emphasized and understood that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Additionally, the specific features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0180] It should be understood that in the foregoing description of the embodiments of this specification, for the purpose of helping to understand a feature and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, drawing, or its description. However, this does not mean that the combination of these features is necessary, and those skilled in the art, when reading this specification, may well mark out some of the devices as separate embodiments for understanding. That is to say, the embodiments in this specification may also be understood as the integration of multiple sub - embodiments. And the content of each sub - embodiment is also valid when it has fewer features than all the features of a single foregoing disclosed embodiment.

[0181] Every patent, patent application, publication of patent applications, and other materials cited herein, such as articles, books, specifications, publications, documents, references, etc. (excluding any historical prosecution files associated therewith), are hereby incorporated by reference for all purposes relevant hereto, e.g., in the specification and claims of this application. However, in the event of any inconsistency or conflict between the description, definition, and / or terminology of such materials and those used in this application, the description, definition, and / or terminology used in this application shall prevail.

[0182] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Accordingly, the embodiments disclosed in this specification are presented by way of example only and not limitation. Those skilled in the art may implement the application of this specification by adopting alternative configurations based on the embodiments in this specification. Thus, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A training method for a graph neural network model, comprising: Obtaining n training data sets, where n is an integer greater than 2, one training data set corresponds to one environment, one environment corresponds to one graph data, and the training data set is constructed based on the graph data; Determining the prediction performance of the basic graph neural network model for connection relationships in each environment according to the n training data sets; and Training a target graph neural network model with the goal of minimizing the prediction performance deviation of multiple groups of environments, where the prediction performance deviation of a group of environments characterizes the difference between the prediction performances of two environments, and the target graph neural network model is used to generate corresponding graph data in corresponding environments.

2. The method according to claim 1, wherein, The determining the prediction performance of the basic graph neural network model for connection relationships in each environment according to the n training data sets includes: Using the n training data sets as the input data of the basic graph neural network model to determine the prediction results of the edges with connection relationships between source nodes and target nodes in each environment, where the prediction performance is characterized by the prediction results; And, the training a target graph neural network model with the goal of minimizing the prediction performance deviation of multiple groups of environments includes: training the target graph neural network model with the goal of minimizing the difference in the deviation between multiple groups of prediction results.

3. The method according to claim 2, wherein The deviation between a group of prediction results is the absolute value of the difference in prediction results between any two environments; the prediction result of a training data set characterizes the prediction probability of the edges with connection relationships between source nodes and target nodes in the corresponding environment; the training a target graph neural network model with the goal of minimizing the difference in the deviation between multiple groups of prediction results includes: Determining the absolute value of the difference in at least two groups of prediction probabilities; and Training the target graph neural network model with the goal of minimizing the difference between the determined at least two absolute values.

4. The method according to claim 2, wherein, The training a target graph neural network model with the goal of minimizing the difference in the deviation between multiple groups of prediction results includes: Determining the graph structure learning loss function between the multiple groups of prediction results and a preset true value; For any group of training data sets, determining the cross-environment loss function between the prediction results of different training data sets in the any group, where the cross-environment loss function is used to characterize the cross-entropy loss of the prediction results of the edges existing between source nodes and target nodes in two different environments; and Training the target graph neural network model with the goal of minimizing multiple groups of the graph structure learning loss function and the cross-environment loss function.

5. The method according to claim 4, wherein, The training a target graph neural network model with the goal of minimizing multiple groups of the graph structure learning loss function and the cross-environment loss function includes: Constructing a function sum between multiple groups of the graph structure learning loss function and the cross-environment loss function as the goal; Constructing a total objective function according to the function sum and a preset regularization term; and Training the target graph neural network model with the goal of minimizing the total objective function.

6. The method according to any one of claims 1 to 5, wherein, Determining the prediction performance of the basic graph neural network model for connection relationships in each environment according to the n training data sets includes: For any training data set, based on the basic graph neural network model, determining the vector representation corresponding to each node in the any training data set, where the vector representation corresponding to any node in the nodes is used to characterize the neighbor information of the any node in the current layer and the aggregated features of the information of the any node in different layers; and, Determining the prediction performance of the any training data set according to the vector representations corresponding to the respective nodes.

7. The method according to any one of claims 2 to 5, wherein For any training data set, the any training data set includes positive sample sub-graph structure pairs and negative sample sub-graph structure pairs; The positive sample sub-graph structure pair includes a sub-graph structure with a source node as the central node and nodes connected to the source node, and also includes a sub-graph structure with a target node as the central node and nodes connected to the target node; The negative sample sub-graph structure pair includes a sub-graph structure with a source node as the central node and nodes not connected to the source node, and also includes a sub-graph structure with a target node as the central node and nodes not connected to the target node.

8. The method according to claim 7, wherein The method further includes: Obtaining the graph data corresponding to the any training data set; Using random walk to generate multiple paths for each node in the graph data corresponding to the any training data; Determining the positive sample sub-graph structure pair according to two nodes within a first preset distance in the multiple paths, where the first preset distance is 3-hop neighbors; and Determining the negative sample sub-graph structure pair according to two nodes that are 2-hop neighbors or 3-hop neighbors and not directly connected in the multiple paths.

9. The method according to claim 7, wherein Taking the n training data sets as the input data of the basic graph neural network model to determine the prediction result of the edge where there is a connection relationship between the source node and the target node in each environment includes: Based on the basic graph neural network model, respectively determining the first similarity between the connected nodes in the positive sample sub-graph structure and the second similarity between the unconnected nodes in the negative sample sub-graph structure; and Determining the ratio between the first similarity and the second similarity as the prediction result of the any training data set.

10. A training system for a graph neural network model, including: At least one storage medium storing at least one instruction set for training a graph neural network model; At least one processor communicatively connected to the at least one storage medium, where when the at least one processor runs, it reads the at least one instruction set and executes the method according to any one of claims 1 to 9 according to the instruction of the at least one instruction set.

Images