Generalized graph neural network training method and device based on personalized environment learning
By distinguishing and decoupling the stable information and noise information in the graph data, and modeling the environmental noise information in each graph data, the problem of performance degradation of the graph neural network when distribution differences exist is solved, and the test environment and the training environment are realized. Adaptive performance preservation in the presence of distribution differences.
Patent Information
- Application Number
- CN202411444982.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-01-21
AI Technical Summary
The prior art is difficult to maintain the predictive performance of graph neural networks when there is a distribution difference between the test environment and the training environment, resulting in a significant decline in model performance.
By obtaining the stable subgraph representation vector and the noise subgraph representation vector of the graph data, they are respectively input into the graph neural network to be trained for processing, the predicted probability of the noise subgraph is calculated and its influence is removed, and the parameters of the graph neural network are updated to focus on stable information.
It effectively eliminates spurious correlations in representation vectors, prevents graph neural networks from making predictions based on noise information and spurious correlations between labels, and ensures that when distribution differences exist, graph neural networks make decisions based on stable information and maintain prediction performance.
Smart Images
Figure CN119337926A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of graph data technology, and in particular to a generalizable graph neural network training method and device based on personalized environment learning. Background Art
[0002] Graph data (graph structured data) is a data structure composed of nodes and edges, which is mainly used to represent complex or irregular data relationships. There are many types of graph data in real life, such as social networks, transportation networks, and biological networks. However, these graph data are usually not directly used in most deep learning algorithms, and their vectorized representation needs to be calculated in order to better solve tasks related to graph data.
[0003] In related technologies, vectorized representations are usually only fitted for training environments, lacking the ability to adapt to unknown, complex test environments that may have distribution changes. However, real-world graph data often comes from complex distributions. When the training data is insufficient to fully reflect the true distribution of the data, distribution differences will arise between the training and test data, resulting in a significant decrease in model performance, greatly limiting the effectiveness of vectorized representations in practical applications. Summary of the invention
[0004] In view of the above problems, the embodiments of the present application provide a generalizable graph neural network training method and device based on personalized environment learning, so as to overcome the above problems or at least partially solve the above problems.
[0005] In a first aspect of an embodiment of the present application, a generalizable graph neural network training method based on personalized environment learning is disclosed, the method comprising: Obtaining a stable subgraph characterization vector and a noise subgraph characterization vector of the graph data, wherein the stable subgraph characterization vector includes stability information in the graph data, and the noise subgraph characterization vector includes noise information in the graph data; Inputting the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result; Calculating the prediction probability of the noise subgraph according to the noise subgraph representation vector and the label corresponding to the graph data; Calculating the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result; According to the predicted probability of the stable subgraph, the parameters of the graph neural network to be trained are updated.
[0006] Optionally, obtaining a stable subgraph characterization vector and a noise subgraph characterization vector of the graph data includes: Performing sub-graph segmentation processing on the graph data to obtain a stable sub-graph and a noise sub-graph; The stable subgraph and the noise subgraph are characterized by two graph neural networks respectively, so as to obtain a stable subgraph characterization vector and a noise subgraph characterization vector.
[0007] Optionally, performing sub-graph segmentation processing on the graph data to obtain a stable sub-graph and a noise sub-graph includes: Aggregating neighbor information of each node in the graph data to obtain neighbor information; Update the current node according to the neighbor information to obtain a node representation; The similarity between nodes is calculated according to the node representation, and the stable subgraph and the noise subgraph are determined according to the similarity calculation result.
[0008] Optionally, determining a stable subgraph and a noise subgraph according to a similarity calculation result includes: Determine the edges between the node pairs with the highest similarity above the threshold value in the similarity calculation result as the edges of the stable subgraph; Determine the edges between the node pairs other than the node pairs with the first percentage threshold having high similarity in the similarity calculation results as the edges of the noise subgraph; A stable subgraph is generated according to the edges of the stable subgraph, and a noise subgraph is generated according to the edges of the noise subgraph.
[0009] Optionally, calculating the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result includes: Inputting the prediction probability of the noise subgraph and the prediction result into the inverse weighted cross entropy function for calculation to obtain the prediction probability of the stable subgraph; Wherein, in the inverse weighted cross entropy function, the prediction probability of the noise subgraph is weighted in an inverse manner.
[0010] Optionally, the method further comprises: Selecting batch graph data from the training data set, wherein the batch graph data includes a plurality of graph data; Calculating in parallel the prediction probability of the stable subgraph corresponding to each graph data in the batch of graph data; According to the predicted probability of the stable subgraph, the parameters of the graph neural network to be trained are updated, including: According to the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data, the parameters of the graph neural network to be trained are updated in sequence.
[0011] A second aspect of the embodiments of the present application discloses a graph data processing method, the method comprising: Get the graph data to be processed; Inputting the graph data to be processed into a pre-trained graph neural network for processing to obtain a representation vector corresponding to the graph data to be processed; the graph neural network is trained by the generalizable graph neural network training method based on personalized environment learning described in the first aspect of the embodiment of the present application; The representation vector corresponding to the graph data to be processed is input into the corresponding task network for processing to obtain the task result.
[0012] In a third aspect of the embodiments of the present application, a generalizable graph neural network training device based on personalized environment learning is disclosed, the device comprising: A first acquisition module is used to acquire a stable subgraph representation vector and a noise subgraph representation vector of the graph data, wherein the stable subgraph representation vector includes stability information in the graph data, and the noise subgraph representation vector includes noise information in the graph data; A first input module, used for inputting the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result; A first calculation module, configured to calculate a prediction probability of a noise subgraph according to the noise subgraph characterization vector and a label corresponding to the graph data; A second calculation module, used to calculate the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result; The first updating module is used to update the parameters of the graph neural network to be trained according to the predicted probability of the stable subgraph.
[0013] According to a fourth aspect of an embodiment of the present application, an electronic device is disclosed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of training a generalizable graph neural network based on personalized environment learning as described in the first aspect of an embodiment of the present application are implemented, or the steps of the graph data processing method as described in the second aspect of an embodiment of the present application are implemented.
[0014] In a fifth aspect of an embodiment of the present application, a computer-readable storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, the steps of training a generalizable graph neural network based on personalized environment learning described in the first aspect of the embodiment of the present application are implemented, or the steps of the graph data processing method described in the second aspect of the embodiment of the present application are implemented.
[0015] The embodiments of the present application include the following advantages: In an embodiment of the present application, a stable subgraph representation vector and a noise subgraph representation vector of graph data are obtained, wherein the stable subgraph representation vector contains stability information in the graph data, and the noise subgraph representation vector contains noise information in the graph data; the stable subgraph representation vector and the noise subgraph representation vector are input into a graph neural network to be trained for processing to obtain a prediction result; the prediction probability of the noise subgraph is calculated based on the noise subgraph representation vector and the label corresponding to the graph data; the prediction probability of the stable subgraph is calculated based on the prediction probability of the noise subgraph and the prediction result; and the parameters of the graph neural network to be trained are updated based on the prediction probability of the stable subgraph.
[0016] In this way, by distinguishing, separating and decoupling the stable information and noise information in the graph structure, modeling the environmental noise information in each different graph data, and obtaining the prediction probability of the noise subgraph, the influence of the environmental noise information is removed from the prediction result according to the prediction probability of the noise subgraph to eliminate the pseudo-correlation in the representation vector. This can prevent the graph neural network from making predictions based on the pseudo-correlation between noise information and labels, ensure that the graph neural network makes decisions based on stable information, and adaptively maintain the prediction performance of the graph neural network when there is a distribution difference between the test environment and the training environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0018] Figure 1 It is a flowchart of the steps of a generalizable graph neural network training method based on personalized environment learning provided by an embodiment of the present application; Figure 2 It is an overall architecture diagram of a generalizable graph neural network training method based on personalized environment learning provided in an embodiment of the present application; Figure 3 It is a flowchart of the steps of a graph data processing method provided by an embodiment of the present application; Figure 4 It is a structural schematic diagram of a generalizable graph neural network training device based on personalized environment learning provided in an embodiment of the present application; Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0020] In the related technology, the methods for obtaining vectorized representation mainly include: representation learning of graph neural networks, graph neural networks with generalized number of nodes, and the expressive power of graph neural networks. (1) Representation learning of graph neural networks: Graph neural networks have attracted widespread attention due to their powerful graph representation capabilities and have achieved good results in many tasks. Such methods mainly adopt a neighbor aggregation mechanism to iteratively update node representations using neighbor information; then, the node representations are aggregated into full graph representations through a graph pooling method. However, when the distribution of the test environment and the training environment are inconsistent, the effect of such methods will be significantly reduced. Therefore, their application scenarios are limited and difficult to use in actual scenarios of out-of-distribution generalization. (2) Graph neural networks with generalized number of nodes: The prediction results of such methods do not depend on the number of nodes in the graph data. The model trained on a small graph can be transferred to a large graph and perform well on the large graph. However, such methods only adapt to the situation where the number of nodes changes and cannot cope with other types of distribution differences. Therefore, they also cannot meet the needs of graph representation learning with out-of-distribution generalization in real scenarios. (3) The expressive power of graph neural networks: This method analyzes the ability of graph neural networks to solve tasks from a theoretical perspective and provides theoretical guarantees when the distribution of the training set and the test set are similar. However, when the training and test environments are inconsistent, this type of method is difficult to apply, resulting in its inability to accurately describe the graph structure data in an unknown test environment, which in turn affects the performance of downstream tasks.
[0021] Therefore, the vectorized representations obtained by existing methods are usually only fitted for the training environment, and lack the ability to adapt to unknown, complex test environments that may undergo distribution changes. However, real-world graph data often comes from complex distributions. When the training data is not sufficient to fully reflect the true distribution of the data, distribution differences will arise between the training and test data. In this case, although traditional methods can achieve good results on the training data set and terminate training, in actual tests, due to distribution migration, the model performance will drop significantly, greatly limiting its effectiveness in practical applications.
[0022] In order to overcome the limitations of related technologies, an embodiment of the present application provides a generalizable graph neural network training method based on personalized environmental learning. Compared with related technologies, which mainly focus on fitting the graph structure representation of training data and generate a unified vector from an overall perspective, it is unable to capture the multiple hidden factors behind the graph data, and fails to fully consider the true characteristics of the graph data from the perspective of multiple hidden factors. The difference of the embodiment of the present application is that the stable information and noise information in the graph data are distinguished, separated and decoupled, and the environmental noise information in each different graph data is modeled to obtain the prediction probability of the noise subgraph, thereby removing the influence of the environmental noise information from the prediction result according to the prediction probability of the noise subgraph to eliminate the pseudo-correlation in the representation vector. In this way, it is possible to prevent the graph neural network from making predictions based on the pseudo-correlation between the noise information and the label, ensure that the graph neural network makes decisions based on stable information, and adaptively maintain the prediction performance of the graph neural network when there is a distribution difference between the test environment and the training environment.
[0023] The following is a detailed description of the generalizable graph neural network training method based on personalized environment learning provided by an embodiment of the present application in conjunction with the accompanying drawings.
[0024] Reference Figure 1 As shown, Figure 1 1 is a flowchart of the steps of a generalizable graph neural network training method based on personalized environment learning provided by an embodiment of the present application. Figure 1 As shown, the generalizable graph neural network training method based on personalized environment learning may include steps S110 to S150: Step S110: obtaining a stable subgraph characterization vector and a noise subgraph characterization vector of the graph data, wherein the stable subgraph characterization vector includes stability information in the graph data, and the noise subgraph characterization vector includes noise information in the graph data.
[0025] Among them, the graph data contains stable information and noise information; the stable subgraph representation vector refers to the representation vector corresponding to each node in the stable subgraph, and the stable subgraph refers to the subgraph composed of nodes whose graph data contains stable information; the noise subgraph representation vector refers to the representation vector corresponding to each node in the noise subgraph, and the noise subgraph refers to the subgraph composed of nodes whose graph data contains noise information.
[0026] Since the association between noise information and stable information may lead to performance degradation when the test environment and the training environment are inconsistent, the stable information and noise information in the graph data are decoupled and modeled to obtain a stable subgraph representation vector containing stable information and a noise subgraph representation vector containing noise information.
[0027] Step S120: input the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result.
[0028] Among them, the prediction result is a prediction result based on the combined effect of noise information and stable information. The stable subgraph representation vector and the noise subgraph representation vector are input into the graph neural network to be trained for processing, including: according to the stable subgraph representation vector and the noise subgraph representation vector, the average value of the corresponding nodes is calculated to obtain a summarized graph vectorization representation, and the summarized graph vectorization representation contains both stable information and noise information in the graph data, so as to generate a prediction result according to the summarized graph vectorization representation.
[0029] Step S130: Calculate the prediction probability of the noise subgraph according to the noise subgraph characterization vector and the label corresponding to the graph data.
[0030] Among them, the prediction probability of the noise subgraph refers to the prediction ability of the noise subgraph representation vector for labels, that is, the prediction probability of the noise subgraph represents the prediction ability based on noise information. A classifier is used to learn the prediction ability of the noise subgraph representation vector for labels according to the generalized cross entropy loss function, that is, the noise subgraph representation vector and the label corresponding to the graph data are input into the generalized cross entropy loss function for calculation to obtain the prediction probability of the noise subgraph. During the training process, the same processing is performed on each graph data (that is, the prediction probability of the noise subgraph corresponding to each graph data is calculated), so that different distribution environments can be simulated.
[0031] Step S140: Calculate the prediction probability of the stable sub-graph according to the prediction probability of the noise sub-graph and the prediction result.
[0032] In the embodiment of the present application, since the prediction result is a prediction result based on the combined effect of noise information and stable information, the prediction probability of the noise subgraph represents the prediction ability based on the noise information, and therefore, according to the prediction probability of the noise subgraph, the influence of the noise information is removed from the prediction result, thereby stabilizing the prediction probability of the subgraph. The prediction probability of the stable subgraph refers to the prediction ability of the stable subgraph representation vector for labels.
[0033] Step S150: updating the parameters of the graph neural network to be trained according to the predicted probability of the stable subgraph.
[0034] In the embodiment of the present application, the parameters of the graph neural network to be trained are updated by the prediction probability of the stable subgraph, which can encourage the stable subgraph to realize that the information captured by its corresponding graph neural network focuses more on the real and predictive information of the graph data itself (i.e., stable information), while ignoring the noise information in the graph data. Since the stable subgraph and the noise subgraph of each graph data are different, the prediction probability of the stable subgraph used to update the parameters of the graph neural network to be trained varies with the prediction probability of the noise subgraph, and finally realizes the personalized reweighted training of the stable subgraph.
[0035] Through the above process, the stable information and noise information in the graph structure are distinguished, separated and decoupled, and the environmental noise information in each different graph data is modeled to obtain the prediction probability of the noise subgraph, so as to remove the influence of the environmental noise information from the prediction result according to the prediction probability of the noise subgraph to eliminate the pseudo-correlation in the representation vector. This can prevent the graph neural network from making predictions based on the pseudo-correlation between noise information and labels, ensure that the graph neural network makes decisions based on stable information, and adaptively maintain the prediction performance of the graph neural network when there is a distribution difference between the test environment and the training environment.
[0036] In combination with the above embodiments, in one implementation, the present application embodiment also provides a generalizable graph neural network training method based on personalized environment learning. In this method, the "obtaining a stable subgraph representation vector and a noise subgraph representation vector of the graph data" in the above step S110 can specifically include the following sub-steps S110-1 to S110-2: Step S110 - 1 : performing sub-graph segmentation processing on the graph data to obtain a stable sub-graph and a noise sub-graph.
[0037] Step S110 - 2 : Characterize the stable subgraph and the noise subgraph respectively through two graph neural networks to obtain a stable subgraph characterization vector and a noise subgraph characterization vector.
[0038] In an embodiment of the present application, a stable subgraph refers to a subgraph composed of nodes in the graph data that contain stable information, and a noise subgraph refers to a subgraph composed of nodes in the graph data that contain noise information. The graph data can be segmented according to the representation of each node in the graph data to obtain a stable subgraph and a noise subgraph.
[0039] After obtaining the stable subgraph and the noise subgraph, two separate graph neural networks are used to characterize the stable subgraph and the noise subgraph respectively; for example, the stable subgraph is characterized by using the first graph neural network to obtain a stable subgraph characterization vector. The noise subgraph is characterized by using the second graph neural network to obtain a noise subgraph characterization vector.
[0040] Furthermore, the graph data is subjected to sub-graph segmentation processing to obtain stable sub-graphs and noise sub-graphs, including: aggregating neighbor information of each node in the graph data to obtain neighbor information; updating the current node according to the neighbor information to obtain a node representation; calculating the similarity between nodes according to the node representation, and determining the stable sub-graph and the noise sub-graph according to the similarity calculation result.
[0041] The more similar two nodes are, the more similar they are, indicating that they contain similar information. Based on the similarity calculation results, the edges between pairs of nodes with high similarity are defined as edges of stable subgraphs, and the remaining edges are classified as edges of noise subgraphs. Therefore, different stable subgraphs and noise subgraphs can be obtained for each graph data.
[0042] In one embodiment, a stable subgraph and a noise subgraph are determined based on similarity calculation results, including: determining the edges between node pairs with a top percentage threshold of high similarity in the similarity calculation results as edges of the stable subgraph; determining the edges between node pairs other than the node pairs with a top percentage threshold of high similarity in the similarity calculation results as edges of the noise subgraph; generating a stable subgraph based on the edges of the stable subgraph, and generating a noise subgraph based on the edges of the noise subgraph.
[0043] Among them, the first percentage threshold is flexibly set according to the actual scenario. For example, if the first percentage threshold is 50%, according to the similarity calculation results, the edges between the first 50% of the node pairs with high similarity are determined as the edges of the stable subgraph; the edges between the remaining 50% of the node pairs are determined as the edges of the noise subgraph. If the first percentage threshold is 40%, according to the similarity calculation results, the edges between the first 40% of the node pairs with high similarity are determined as the edges of the stable subgraph; the edges between the remaining 60% of the node pairs are determined as the edges of the noise subgraph.
[0044] In one embodiment, a stable subgraph and a noise subgraph are determined based on similarity calculation results, including: determining the edges between node pairs whose similarities are greater than a similarity threshold in the similarity calculation results as edges of the stable subgraph, determining the edges between node pairs whose similarities are not greater than the similarity threshold as edges of the noise subgraph, generating a stable subgraph based on the edges of the stable subgraph, and generating a noise subgraph based on the edges of the noise subgraph.
[0045] The similarity threshold is flexibly set according to the actual scenario. For example, when the similarity threshold is 0.6, according to the similarity calculation result, the edges between the node pairs with similarity greater than 0.6 are determined as the edges of the stable subgraph, and the edges between the node pairs with similarity less than 0.6 are determined as the edges of the noise subgraph.
[0046] Through the above process, the stable information and the noise information in the graph structure are segmented to obtain a stable subgraph containing the stable information in the graph data and a noise subgraph containing the noise information in the graph data, so that subsequent modeling can be performed based on the stable information and the noise respectively.
[0047] In combination with the above embodiments, in one implementation, the present application embodiment further provides a generalizable graph neural network training method based on personalized environment learning. In this method, the "calculating the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result" in the above step S140 may specifically include: The prediction probability of the noise subgraph and the prediction result are input into the inverse weighted cross entropy function for calculation to obtain the prediction probability of the stable subgraph; wherein, in the inverse weighted cross entropy function, the prediction probability of the noise subgraph is weighted in an inverse manner.
[0048] For example, the inverse weighted cross entropy function can be expressed as: , in, represents the predicted probability of a stable subgraph, represents the predicted probability of the noise subgraph, Represents the prediction result, represents the cross entropy function.
[0049] Furthermore, the function It can be expressed as : , in, Indicates the label corresponding to the graph data, represents a constant between 0 and 1. In some embodiments The value of is 0.7.
[0050] In this way, by inversely weighting the prediction probability of the noise subgraph, the stable subgraph is trained using the inverse weighted cross entropy function to encourage the stable subgraph to focus more on the real and predictive information (stable information) of the graph data itself, while ignoring the noise information of the graph data.
[0051] In combination with the above embodiments, in one implementation, the present application embodiment further provides a generalizable graph neural network training method based on personalized environment learning. In this method, in addition to the above steps, the following steps are also included: Step B1: Select batch graph data from the training data set, where the batch graph data includes multiple graph data.
[0052] Step B2: Calculate in parallel the prediction probability of the stable subgraph corresponding to each graph data in the batch graph data.
[0053] Step B3: According to the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data, the parameters of the graph neural network to be trained are updated in sequence.
[0054] In an embodiment of the present application, at each training, a batch of graph data is selected from the training data set, and the graph neural network to be trained is trained based on the batch graph data. Specifically, the graph data in the batch graph data are processed in parallel, and each graph data is processed in the manner of steps S110 to S140 described above, so as to obtain the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data, and according to the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data, the parameters of the graph neural network to be trained are updated in turn, and after the training end conditions are met, a trained graph neural network is obtained, and the trained graph neural network has a generalized output under distribution migration.
[0055] In this way, training the graph neural network in batches can improve the training efficiency of the neural network. For example, for a training data set containing 1,000 graph data, 100 graph data are used as a batch. During each training, the 100 graph data in the batch are trained simultaneously, so only 10 times are needed to complete the training.
[0056] For example, Figure 2 This is an overall architecture diagram of a generalizable graph neural network training method based on personalized environment learning provided by an embodiment of the present application. Specifically, the training neural network is trained in a batch training optimization manner. For each graph data in the batch graph data: first, the stable subgraph is personalized segmented through the graph data to obtain a stable subgraph and a noise subgraph.
[0057] Next, personalized information modeling of environmental noise is performed. The stable subgraph and the noise subgraph are characterized by two graph neural networks to obtain the stable subgraph representation vector and the noise subgraph representation vector respectively. The stable subgraph representation vector and the noise subgraph representation vector are input into the graph neural network to be trained for processing to obtain the prediction result. The prediction probability of the noise subgraph is calculated according to the labels corresponding to the noise subgraph representation vector and the graph data through the generalized cross entropy loss function to learn the prediction ability of the noise subgraph representation vector for labels. The same processing is performed on each graph data to simulate different distribution environments.
[0058] Afterwards, personalized reweighted training of the stable subgraph is performed. According to the prediction probability and prediction result of the noise subgraph, the prediction probability of the stable subgraph is calculated, so as to update the parameters of the graph neural network to be trained according to the prediction probability of the stable subgraph, and obtain the trained graph neural network. In this way, the stable subgraph can be encouraged to realize that the information captured by its corresponding graph neural network focuses more on the real and predictive information of the graph data itself (i.e., stable information), while ignoring the noise information in the graph data. Since the stable subgraph and noise subgraph of each graph data are different, the prediction probability of the stable subgraph used to update the parameters of the graph neural network to be trained varies with the prediction probability of the noise subgraph, and finally the personalized reweighted training of the stable subgraph is realized; the trained graph neural network has a generalized output under distribution transfer.
[0059] Through the generalizable graph neural network training method based on personalized environment learning in the embodiment of the present application, it is possible to provide support for different downstream tasks and practical applications. For example, in drug analysis, the model can be trained on a small number of annotated molecular graphs, and then applied to a larger scale, unlabeled drug data with a distribution different from the training set for classification. In addition, in social network analysis, the environment-adaptive graph representation can provide stable generalization results in the case of dynamic changes, evolution, and migration of data distribution. In addition, this method plays an important role in fields such as human-computer interaction and computer-aided systems.
[0060] The present application also provides a method for processing graph data. Figure 3 As shown, Figure 3 This is a flowchart of a method for processing graph data provided by an embodiment of the present application. Figure 3 As shown, the graph data processing method may include steps S310 to S330: Step S310: Obtaining graph data to be processed; Step S320: inputting the graph data to be processed into a pre-trained graph neural network for processing to obtain a representation vector corresponding to the graph data to be processed; the graph neural network is trained by the generalizable graph neural network training method based on personalized environment learning described in an embodiment of the present application; Step S330: input the representation vector corresponding to the graph data to be processed into the corresponding task network for processing to obtain the task result.
[0061] In the embodiment of the present application, after training, the graph neural network can be used to process downstream tasks. For the graph data set (graph data to be processed) in the downstream task, the graph neural network can generate a representation vector corresponding to each graph data to be processed, so that the downstream task network generates the corresponding task result according to the representation vector corresponding to the graph data to be processed. Among them, the task network can be a classification network, a regression network, etc.
[0062] Since the graph neural network trained by the generalizable graph neural network training method based on personalized environment learning can explicitly characterize the false correlation strength of the graph data, and adaptively generate a stable graph structure representation (i.e., the representation vector corresponding to the graph data to be processed) on the out-of-distribution graph data, reliable task results can be obtained by processing the graph data of the graph data processing method of the embodiment of the present application.
[0063] The present application also provides a generalizable graph neural network training device based on personalized environment learning, referring to Figure 4 As shown, Figure 4 : is a schematic diagram of a generalizable graph neural network training device based on personalized environment learning provided by an embodiment of the present application, the device comprising: A first acquisition module 410 is used to acquire a stable subgraph representation vector and a noise subgraph representation vector of the graph data, wherein the stable subgraph representation vector includes stability information in the graph data, and the noise subgraph representation vector includes noise information in the graph data; A first input module 420, configured to input the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result; A first calculation module 430, configured to calculate a prediction probability of a noise subgraph according to the noise subgraph characterization vector and a label corresponding to the graph data; A second calculation module 440, configured to calculate a prediction probability of a stable subgraph according to the prediction probability of the noise subgraph and the prediction result; The first updating module 450 is used to update the parameters of the graph neural network to be trained according to the predicted probability of the stable subgraph.
[0064] In an optional embodiment, the first acquisition module includes: A sub-graph segmentation module, used for performing sub-graph segmentation processing on the graph data to obtain a stable sub-graph and a noise sub-graph; The characterization processing module is used to perform characterization processing on the stable subgraph and the noise subgraph respectively through two graph neural networks to obtain a stable subgraph characterization vector and a noise subgraph characterization vector.
[0065] In an optional embodiment, the sub-image segmentation module includes: An aggregation module, used to aggregate neighbor information of each node in the graph data to obtain neighbor information; A second updating module, used to update the current node according to the neighbor information to obtain a node representation; The third calculation module is used to calculate the similarity between nodes according to the node representation, and determine the stable subgraph and the noise subgraph according to the similarity calculation result.
[0066] In an optional embodiment, the third calculation module includes: A first determination module is used to determine the edges between the node pairs with a high similarity before the percentage threshold in the similarity calculation result as the edges of the stable subgraph; A second determination module is used to determine the edges between the node pairs other than the node pairs with a high similarity above the first percentage threshold in the similarity calculation result as the edges of the noise subgraph; The subgraph generation module is used to generate a stable subgraph according to the edge of the stable subgraph, and to generate a noise subgraph according to the edge of the noise subgraph.
[0067] In an optional embodiment, the first calculation module is specifically used to: input the predicted probability of the noise subgraph and the prediction result into an inverse weighted cross entropy function for calculation to obtain the predicted probability of the stable subgraph; wherein, in the inverse weighted cross entropy function, the predicted probability of the noise subgraph is weighted in an inverse manner.
[0068] In an optional embodiment, the device further includes: A selection module, used for selecting batch graph data from a training data set, wherein the batch graph data includes a plurality of graph data; A fourth calculation module, used for parallel calculation of the prediction probability of the stable subgraph corresponding to each graph data in the batch graph data; The third updating module is used to update the parameters of the graph neural network to be trained in sequence according to the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data.
[0069] The present application also provides an electronic device, referring to Figure 5 , Figure 5 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes: a memory 510 and a processor 520. The memory 510 and the processor 520 are connected via a bus communication. A computer program is stored in the memory 510. The computer program can be run on the processor 520, thereby implementing the steps of the generalizable graph neural network training method based on personalized environment learning described in the embodiment of the present application, or implementing the steps of the graph data processing method described in the embodiment of the present application.
[0070] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the generalizable graph neural network training method based on personalized environment learning described in the embodiment of the present application are implemented, or the steps of the graph data processing method described in the embodiment of the present application are implemented.
[0071] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the generalizable graph neural network training method based on personalized environment learning described in the embodiment of the present application, or implements the steps of the graph data processing method described in the embodiment of the present application.
[0072] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0073] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0076] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
[0077] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0078] The above is a detailed introduction to a generalizable graph neural network training method and device based on personalized environment learning provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A generalizable graph neural network training method based on personalized environment learning, characterized in that: include: Obtaining a stable subgraph characterization vector and a noise subgraph characterization vector of the graph data, wherein the stable subgraph characterization vector includes stability information in the graph data, and the noise subgraph characterization vector includes noise information in the graph data; Inputting the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result; Calculating the prediction probability of the noise subgraph according to the noise subgraph representation vector and the label corresponding to the graph data; Calculating the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result; According to the predicted probability of the stable subgraph, the parameters of the graph neural network to be trained are updated.
2. The method according to claim 1, characterized in that Obtaining stable subgraph representation vectors and noise subgraph representation vectors of graph data, including: Performing sub-graph segmentation processing on the graph data to obtain a stable sub-graph and a noise sub-graph; The stable subgraph and the noise subgraph are characterized by two graph neural networks respectively, so as to obtain a stable subgraph characterization vector and a noise subgraph characterization vector.
3. The method according to claim 2, characterized in that The graph data is subjected to sub-graph segmentation processing to obtain a stable sub-graph and a noise sub-graph, including: Aggregating neighbor information of each node in the graph data to obtain neighbor information; Update the current node according to the neighbor information to obtain a node representation; The similarity between nodes is calculated according to the node representation, and the stable subgraph and the noise subgraph are determined according to the similarity calculation result.
4. The method according to claim 3, characterized in that According to the similarity calculation results, the stable subgraph and the noise subgraph are determined, including: Determine the edges between the node pairs with the highest similarity above the threshold value in the similarity calculation result as the edges of the stable subgraph; Determine the edges between the node pairs other than the node pairs with the first percentage threshold having high similarity in the similarity calculation results as the edges of the noise subgraph; A stable subgraph is generated according to the edges of the stable subgraph, and a noise subgraph is generated according to the edges of the noise subgraph.
5. The method according to claim 1, characterized in that Calculating the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result, including: Inputting the prediction probability of the noise subgraph and the prediction result into the inverse weighted cross entropy function for calculation to obtain the prediction probability of the stable subgraph; Wherein, in the inverse weighted cross entropy function, the prediction probability of the noise subgraph is weighted in an inverse manner.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Selecting batch graph data from the training data set, wherein the batch graph data includes a plurality of graph data; Calculating in parallel the prediction probability of the stable subgraph corresponding to each graph data in the batch of graph data; According to the predicted probability of the stable subgraph, the parameters of the graph neural network to be trained are updated, including: According to the predicted probability of the stable subgraph corresponding to each graph data in the batch graph data, the parameters of the graph neural network to be trained are updated in sequence.
7. A graph data processing method, characterized in that: The method comprises: Get the graph data to be processed; Inputting the graph data to be processed into a pre-trained graph neural network for processing to obtain a representation vector corresponding to the graph data to be processed; the graph neural network is trained by the generalizable graph neural network training method based on personalized environment learning described in any one of claims 1 to 6; The representation vector corresponding to the graph data to be processed is input into the corresponding task network for processing to obtain the task result.
8. A generalizable graph neural network training device based on personalized environment learning, characterized in that: include: A first acquisition module is used to acquire a stable subgraph representation vector and a noise subgraph representation vector of the graph data, wherein the stable subgraph representation vector includes stability information in the graph data, and the noise subgraph representation vector includes noise information in the graph data; A first input module, used for inputting the stable subgraph representation vector and the noise subgraph representation vector into the graph neural network to be trained for processing to obtain a prediction result; A first calculation module, configured to calculate a prediction probability of a noise subgraph according to the noise subgraph characterization vector and a label corresponding to the graph data; A second calculation module, used to calculate the prediction probability of the stable subgraph according to the prediction probability of the noise subgraph and the prediction result; The first updating module is used to update the parameters of the graph neural network to be trained according to the predicted probability of the stable subgraph.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the generalizable graph neural network training method based on personalized environment learning as described in any one of claims 1 to 6, or implements the steps of the graph data processing method as described in claim 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the generalizable graph neural network training method based on personalized environment learning as described in any one of claims 1 to 6, or implements the steps of the graph data processing method as described in claim 7.