Graph Neural Network Training with Pseudo-Labels and Information Gain
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph neural networks (GNNs) performance significantly deteriorates with a scarcity of labeled data, as they heavily rely on the quantity of labeled data, leading to decreased accuracy in service processing tasks.
Innovation Solution
A training method that iteratively updates the GNN model by expanding labeled data using unlabeled data, where pseudo classification labels are allocated based on high-confidence predictions, and information gain is calculated to reduce the difference in training losses between original and expanded data distributions, thereby improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If GNN is trained using only original labeled data, then training simplicity is maintained, but model performance and prediction accuracy deteriorate due to insufficient labeled data
Solution Approach 1:
The system performs self-training by automatically generating pseudo-labels for unlabeled data through iterative training. The GNN model generates predictions on unlabeled nodes, selects high-confidence predictions as pseudo-labels, and uses these pseudo-labels to expand the training set, enabling the model to improve itself without external intervention
Solution Approach 2:
Pseudo-labels serve as an intermediary between labeled and unlabeled data. The system introduces pseudo-labels as a intermediate training target that bridges the gap between limited labeled data and abundant unlabeled data, allowing the model to leverage unlabeled data while maintaining training signal quality through confidence thresholding
2Quantity of substance
If unlabeled data is expanded into labeled data using pseudo-labels, then the quantity of training data increases, but the difference between training loss distributions of original and expanded data increases
Solution Approach 1:
The system implements feedback through iterative training where the model's predictions on unlabeled data are evaluated, high-confidence predictions are selected as pseudo-labels, and these pseudo-labels are fed back into the training set. This feedback loop allows the model to gradually adapt to the expanded data distribution while maintaining quality control through confidence thresholds
Solution Approach 2:
The system applies partial action by selectively using only high-confidence pseudo-labels (above a predetermined threshold) for training expansion, rather than using all unlabeled data. This selective approach ensures that only reliable pseudo-labels are incorporated, minimizing distribution shift while maximizing the benefit of unlabeled data
3Manufacturing precision
If information gain is introduced to reduce training loss difference, then training effect improves, but computational complexity increases
Solution Approach 1:
The system performs preliminary action by pre-calculating information gain values for unlabeled nodes before the main training process. By computing information gain in advance based on prediction confidence and data distribution, the system prepares weighting factors that guide subsequent training, reducing computational burden during iterative training while maintaining training precision
Data Source
AI summary
Implementations of the present specification provide a training method for a graph neural network, and relate to performing multiple rounds of iterative updating on a graph neural network based on a user relational graph, where any round of the multiple rounds includes: processing the user relational graph by using a current graph neural network, to obtain multiple classification prediction vectors corresponding to multiple user nodes in the user relational graph; allocating a corresponding pseudo classification label to a first quantity of unlabeled nodes in the multiple user nodes based on the multiple classification prediction vectors; determining, for each of the first quantity of unlabeled nodes, an information gain generated by training the current graph neural network by using the unlabeled node; and updating a model parameter in the current graph neural network according to a classification prediction vector and a real classification label that are corresponding to each labeled node in the multiple user nodes, and a classification prediction vector, a pseudo classification label, and an information gain that are corresponding to each unlabeled node.