GraphQL Schema Chunking for Accurate Low-Latency Query Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language model (LLM) based solutions struggle with large schemas due to size limitations, leading to slow inference speeds, high latency, and increased hallucination, especially when retrieving data from large GraphQL SuperGraphs with thousands of entities and fields.
Innovation Solution
A system and method that involves chunking a GraphQL schema into smaller units, embedding these chunks into a vector database, and using natural language inputs to dynamically generate queries, reducing the size of the schema to improve performance and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full schemas are provided as context to LLMs, then the LLM can have deep understanding of relationships and connections, but inference speed slows down and latency increases
Solution Approach 1:
The patent segments the full GraphQL schema into smaller, relevant subschemas based on the user query. Instead of providing the entire schema to the LLM, the system identifies and extracts only the necessary portions (types, fields, and relationships) that are relevant to the specific query task, thereby reducing context size while maintaining understanding accuracy.
Solution Approach 2:
The system extracts only the essential schema components needed for the specific query task. By analyzing the user query and the full schema, it extracts relevant types, fields, and relationships, removing unnecessary portions before presenting to the LLM, thus reducing latency while preserving understanding.
2Loss of information
If full schemas are provided as context to LLMs, then complete information is available, but the LLM experiences hallucination and performance degradation
Solution Approach 1:
The patent applies local quality by providing different levels of schema detail to different parts of the LLM's context. The system identifies which specific types and fields are relevant to the query and provides detailed information about those while omitting or simplifying less relevant portions, creating a non-uniform context structure that improves reliability.
Solution Approach 2:
The schema context provided to the LLM is dynamic rather than static. The system adapts the schema content based on the specific user query, extracting and providing only the portions that are relevant to the current task. This dynamic adaptation prevents information overload and reduces hallucination while maintaining completeness for the specific query.
3Adaptability or versatility
If large schemas with thousands of entities and fields are used, then comprehensive data retrieval is possible, but developers have difficulty finding correct fields
Solution Approach 1:
The system performs preliminary action by pre-processing the full schema and organizing it into structured, queryable components. Before the LLM needs to generate queries, the system has already parsed the schema, identified relationships, and made the information accessible through structured representations, thereby easing field discovery while maintaining comprehensive retrieval capability.
Data Source
AI summary
A system and method are provided for enhanced schema discovery and query generation.