LLM Query Generation Using Vector-Chunked GraphQL Schemas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models face limitations due to maximum token limits, input and output size restrictions, and high computational resource requirements, making them unsuitable for handling large GraphQL schemas, which prevents them from generating and executing structured query language queries effectively.
Innovation Solution
The approach involves chunking a structured query language schema into subschemas, creating embeddings for these subschemas, and storing them in a vector store, allowing a large language model to generate and execute queries based on natural language input and the subschema context, thereby overcoming token limit constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the full GraphQL schema is provided as context to a language model, then the model can generate accurate structured query language queries, but the schema size exceeds the maximum token limit of the language model
Solution Approach 1:
The patent divides the full GraphQL schema into multiple subschemas by identifying type definitions and their related fields. Each subschema contains a specific type and its associated fields, which are then processed separately. This segmentation reduces the token count of each individual schema context while preserving the necessary information for accurate query generation.
Solution Approach 2:
The patent extracts only the relevant portions of the GraphQL schema needed for query generation. By using embeddings to identify and extract specific type definitions and their related fields, the system creates minimal viable subschemas that contain only the necessary information, reducing unnecessary token consumption while maintaining query generation accuracy.
2Ease of operation
If traditional large language models are used for query generation, then natural language processing capability is provided, but computational resource requirements and operational costs are excessively high
Solution Approach 1:
The patent introduces an intermediary system that bridges natural language queries and the GraphQL schema. This intermediary uses embeddings to convert both the queries and schema into vector representations, enabling semantic matching without requiring a full large language model. This approach maintains natural language processing capability while significantly reducing computational resource requirements.
Solution Approach 2:
The patent replaces the mechanical system of using a full large language model for schema processing with an embedding-based vector search system. Instead of relying on the computational power of a large language model to understand and process the entire schema, the system uses embeddings to efficiently match query intent with relevant schema portions, dramatically reducing computational overhead.
3Productivity
If the language model context is limited to fit within token constraints, then the model can operate within acceptable resource limits, but the ability to process complete schema information is compromised
Solution Approach 1:
The patent performs preliminary processing of the GraphQL schema by pre-computing embeddings for type definitions and their related fields. This preliminary action creates a structured representation that can be efficiently queried later, allowing the system to retrieve only the specific schema portions needed for each query without losing essential information while maintaining efficient processing.
Solution Approach 2:
The patent transforms the schema information from a traditional text-based representation into an embedding-based vector representation. This dimensionality change allows the system to capture semantic relationships and retrieve relevant schema information efficiently, maintaining schema context completeness while working within token constraints through vector search rather than text matching.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for executing structured query language queries having a schema associated therewith against an application programming interface using natural language. The schema can be chunked such that embeddings of the resulting chunks are stored in a vector store. Schemas (or subschemas) generated using on or more chunks of the vector store may be provided to a large language model along with a NL query to generate a structured query language query which may be executed against the application programming interface. This allows large language models to produce structured query language queries, such as GraphQL queries even if a GraphQL schema is too large to be provided to the model as context. Aspects disclosed herein also provide techniques for client code generation and client software development kit generation.