LLM Workflow Orchestration for Automatic External API Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language models (LLMs) are limited in their ability to handle queries for which they have not been trained and require cumbersome, expensive integration with external applications, necessitating expertise that many corporations lack.
Innovation Solution
A system and method for generating workflows using LLMs that can self-design and integrate external applications through building blocks, including semantic graphs, skills, and actions, allowing automatic API calls to external applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs are trained to handle more queries, then their capability improves, but training data size and cost increase
Solution Approach 1:
The patent segments the query handling capability into two parts: (1) a core LLM that handles general language understanding, and (2) external applications that handle specific domain queries. This allows the LLM to remain compact while gaining versatility through tool integration, avoiding the need to continuously expand training data.
Solution Approach 2:
The patent introduces an intermediary layer consisting of a parser, workflow generator, and tool executor that mediates between the LLM and external applications. This intermediary enables the LLM to leverage external knowledge sources without directly training on their data, thus improving versatility without proportionally increasing training data requirements.
2Adaptability or versatility
If LLMs integrate external applications through conventional means, then functionality improves, but integration complexity and cost increase
Solution Approach 1:
The patent creates a universal integration framework that can work with multiple different external applications through a common interface. The workflow generator and tool executor are designed to be application-agnostic, allowing the same LLM system to integrate with various external tools without requiring custom integration code for each application.
Solution Approach 2:
The patent changes the integration parameters from manual configuration to automated generation. The workflow generator automatically creates integration parameters and API calls based on the user's natural language query, eliminating the need for manual configuration and reducing integration complexity.
3Reliability
If LLMs integrate external applications with manual configuration, then integration reliability improves, but time and expertise requirements increase
Solution Approach 1:
The patent performs preliminary actions by pre-defining a structured workflow template that includes all necessary integration components. The workflow generator automatically populates this template with appropriate parameters before execution, ensuring reliability through structured design while reducing time by eliminating manual configuration steps.
Solution Approach 2:
The system performs self-service integration by automatically generating and executing the integration workflow without requiring manual intervention. The LLM parses the query, generates the appropriate workflow, and executes it with external applications autonomously, reducing both time and expertise requirements while maintaining reliability through the structured approach.
4Device complexity
If LLMs operate as standalone systems, then system simplicity improves, but functional limitations increase
Solution Approach 1:
The patent implements a nested architecture where the LLM is embedded within a larger system that includes the parser, workflow generator, and external application interfaces. This nested structure allows the simple LLM core to benefit from the versatility of external applications while maintaining its own simplicity.
Solution Approach 2:
The patent adds a new dimension to the system architecture by introducing a temporal workflow layer between the LLM and external applications. This workflow dimension allows the system to handle complex multi-step queries that require coordination between the LLM and multiple external tools, thereby increasing versatility without complicating the core LLM.
Data Source
AI summary
Systems and methods described receive user input or query at a user interface associated with a large language model (LLM) to perform a task. The user intent and persona are determined based on the user input and other user related data. An LLM automatically generates a workflow consisting of a plurality of steps for performing the task received. Each step of the workflow is mapped to a building block that is to be used for processing the step of the workflow. Selection of the building blocks is based on a plurality of factors, including relevancy, complexity, cost, and accuracy. One of the building blocks used is to perform an application programming interface (API) call to one or more external applications for executing a step of the workflow. Parameters for the API call may be obtained from a generated catalog. Results from the workflow may be customized based on the persona.