LLM Data Collection with Deterministic Calculation Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex computer systems require intricate user interfaces and hardware/software integration for tasks like tax calculations, leading to high maintenance costs and difficulties in handling free-form text and data model disambiguation.
Innovation Solution
The integration of large language models (LLMs) as data collection tools, allowing for data collection through conversation instead of tree structure-based UI systems, decoupling data collection from calculations, and using a deterministic calculation engine to process the data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complex calculation systems use traditional tree structure-based UI systems for data collection, then data can be collected systematically, but the system complexity and maintenance costs increase significantly
Solution Approach 1:
The patent introduces an LLM-based natural language interface as an intermediary between the user and the complex calculation system. Instead of requiring users to navigate complex tree structure UIs, the LLM translates natural language queries into structured data requests, maintaining data collection accuracy while dramatically simplifying the user interface complexity.
Solution Approach 2:
The patent replaces the mechanical tree structure-based UI navigation system with an LLM-based natural language processing system. This substitution eliminates the need for users to manually traverse complex hierarchical data structures, reducing system complexity while maintaining the ability to collect all necessary data points through conversational interfaces.
2Ease of operation
If LLMs are used for data collection through conversation, then user interaction becomes more natural and system complexity reduces, but LLMs may make arithmetic mistakes and consistency issues arise
Solution Approach 1:
The patent segments the system into two distinct components: an LLM-based natural language interface for data collection and a separate deterministic calculation engine for processing. This segmentation allows the LLM to handle only the conversational data collection aspect where natural language interaction is beneficial, while delegating all calculation and arithmetic operations to the deterministic engine, thereby maintaining both ease of operation and calculation accuracy.
Solution Approach 2:
The patent introduces a deterministic calculation engine as an intermediary between the LLM data collection component and the final results. This intermediary receives structured data from the LLM, performs all calculations using deterministic algorithms, and returns results, thereby eliminating arithmetic mistakes while preserving the natural interaction benefits of the LLM interface.
3Device complexity
If LLMs handle both data collection and calculations, then system integration is simplified, but performance and latency increase due to LLM computational overhead
Solution Approach 1:
The patent segments the processing workload by assigning data collection conversations to the LLM and all calculation operations to the deterministic engine. This segmentation enables parallel processing where the LLM collects data while the deterministic engine processes calculations independently, improving overall productivity and reducing latency compared to sequential LLM-only processing.
Solution Approach 2:
The patent uses the LLM to generate structured data representations that are copied to the deterministic calculation engine. This copying mechanism allows the deterministic engine to work with pre-processed, structured data without requiring the LLM to perform calculations, thereby improving processing speed while maintaining system integration simplicity.
4Stability of the object's composition
If traditional hard-coded data collection systems are used, then data collection follows a fixed structure, but the system cannot handle free-form text and data model disambiguation
Solution Approach 1:
The patent replaces the rigid, hard-coded data collection structure with an LLM-based natural language processing system. The LLM maintains structural integrity by outputting standardized data formats while simultaneously handling free-form text input and performing data model disambiguation through semantic understanding, thereby achieving both stability and adaptability.
Solution Approach 2:
The patent changes the parameter of data collection from fixed hierarchical structures to flexible natural language inputs processed by the LLM. The LLM dynamically adapts to different user inputs while maintaining consistent output structures, enabling the system to handle diverse free-form text while preserving data collection stability through standardized output formats.
Data Source
AI summary
A first large language model (LLM) instance may be instructed to request data while being prevented from performing calculations using the data. A second LLM instance may be instructed to provide a response to the request for data based on a known complete data set. The response may be translated into a machine-readable response in a format configured for processing by a calculation engine. The calculation engine may process the machine-readable response, thereby generating a calculation engine output. A mismatch between the calculation engine output and a known result obtained using the known complete data set may be identified, and the instruction to the first LLM may be modified in response.