LLM Edge Text Encoding for More Accurate Graph Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph neural networks struggle to effectively process and utilize textual information associated with edges in graphs, which can provide important context for improving prediction accuracy.
Innovation Solution
Incorporating a large language model to generate embeddings vectors for edge textual information, enabling graph neural networks to process and utilize this information, thereby enhancing the accuracy of graph neural network predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If graph neural networks process only numeric information, then the network structure remains simple, but prediction accuracy is limited due to loss of contextual information
Solution Approach 1:
The patent introduces an intermediary component (text embedding layer) that converts textual edge information into numeric embedding vectors. This mediator enables the graph neural network to process textual data without directly increasing the core network structure's complexity, thus improving prediction accuracy while maintaining structural simplicity.
Solution Approach 2:
The patent segments the data processing pipeline into separate components: text processing (using embedding models), graph construction, and neural network processing. By separating text handling from the core GNN architecture, the system can incorporate textual information without complicating the fundamental network structure.
2Loss of information
If textual information is added to graph edges, then more contextual information is available, but the graph neural network cannot directly process this text format
Solution Approach 1:
The patent replaces the mechanical limitation of direct text processing with a substitution approach: using pre-trained embedding models (language models) to convert text into numeric representations. This substitution enables the graph neural network to process textual information indirectly through embeddings, maintaining both information retention and processing compatibility.
Solution Approach 2:
The patent applies preliminary action by pre-processing textual edge information through embedding models before it enters the graph neural network. This preliminary conversion of text to embeddings occurs outside the main network, allowing the GNN to operate with compatible data formats while preserving full contextual information.
Data Source
AI summary
A method includes accessing a large language model (LLM) configured to encode textual information to generate encoded textual information, receiving a data set having a plurality of entities and a plurality of relationships among the entities, each relationship associated with respective textual information, determining a graph representative of the data set, the graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each node representative of a respective entity of the plurality of entities and each edge representative of the relationship between the entities connected by the edge, wherein the determining includes, for each edge, applying the LLM to the associated textual information to generate encoded edge textual information and adding the encoded edge textual information to the edge in the graph, whereby an enhanced graph is generated, and training a graph neural network model (GNN) based on the enhanced graph.