Composable Asynchronous Tasks for LLM Token Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) face limitations such as token limits, output inconsistency, and difficulty in error recovery, which hinder their usage in software development and integration with traditional systems.
Innovation Solution
The implementation of composable asynchronous tasks that utilize LLMs and Generative Pretrained Transformers (GPTs) to decompose larger jobs into smaller, self-contained tasks, allowing for validation, error checking, and efficient error recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLMs are used to generate text output, then coherent and contextually relevant text can be produced, but token limits restrict the amount of input and output data that can be processed
Solution Approach 1:
The patent divides a large job into multiple smaller composable asynchronous tasks, each processing a subset of the data. This allows the system to handle large quantities of input/output data by processing it in manageable segments through multiple LLM interactions, overcoming the token limit constraint while maintaining text coherence within each segment.
Solution Approach 2:
The patent introduces an intermediary system that manages task decomposition, execution, and result aggregation. This intermediary layer coordinates multiple LLM calls across different tasks, enabling the processing of large data volumes by breaking them into smaller chunks that fit within token limits while maintaining overall contextual coherence.
2Productivity
If LLMs generate output quickly, then productivity is improved, but output consistency and adherence to best practices becomes difficult to ensure
Solution Approach 1:
The patent implements feedback mechanisms where each composable task validates its output against predefined criteria and best practices. The system aggregates results from multiple tasks and performs consistency checks, providing feedback loops that ensure output reliability while maintaining high productivity through parallel task execution.
Solution Approach 2:
The patent performs preliminary validation and consistency checks as part of the task execution framework before final output generation. By establishing validation rules and best practice criteria in advance, the system ensures output consistency is maintained throughout the rapid generation process without requiring slow post-processing verification.
3Adaptability or versatility
If LLMs process complex problems, then capability to handle extensive requirements is improved, but error recovery becomes difficult when incorrect output is produced
Solution Approach 1:
The patent segments complex problems into smaller composable tasks with well-defined inputs and outputs. This segmentation enables targeted error recovery by isolating failures to specific tasks rather than the entire system, making it easier to identify and correct errors while maintaining the ability to handle complex overall problems through task composition.
Solution Approach 2:
The patent implements mechanisms to discard incorrect task outputs and recover by re-executing only the failed tasks with corrected parameters. The composable task framework allows selective re-execution of specific tasks rather than restarting the entire process, improving error recovery ease while maintaining complex problem-solving capability.
4Adaptability or versatility
If LLM output is generated without standardization, then flexibility is maintained, but integration with traditional software systems becomes difficult
Solution Approach 1:
The patent creates a universal composable task framework that standardizes LLM output structures while maintaining flexibility in task implementation. The standardized task interfaces and output formats enable easy integration with traditional software systems, while the composable nature allows flexible adaptation to different problem domains through task composition rather than rigid system design.
Data Source
AI summary
Systems, devices, and techniques are disclosed for automation with composable asynchronous tasks. A prompt may be received at a computing device. Using a first large language model (LLM) composable asynchronous tasks may be determined from the prompt. One of the composable asynchronous tasks may use a second LLM. The composable asynchronous tasks may be performed. Performing one of the composable asynchronous tasks may include generating a first output with the second LLM based on the prompt and validating the first output of the LLM. Performing another of the composable asynchronous tasks may include generating a second output using the first output.