Copilot Microservice Task Distribution for Custom LLM Workflows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Large Language Models (LLMs) face challenges in customization due to computational burden, undesirable artifacts, and the difficulty in generating specialized training data, especially for small-scale applications, which require extensive human effort and resources.
Innovation Solution
A microservice network architecture for copilots is introduced, allowing for customized training and deployment of smaller LLMs that can be trained independently and optimized for specific tasks, with automated data generation techniques such as interviews and recorded workflow emulation to reduce human effort and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large LLM is extensively reprogrammed for customization, then the model can perform specialized tasks better, but the computational burden and time required become prohibitive
Solution Approach 1:
The patent divides the monolithic LLM into multiple smaller microservice models, each specialized for specific tasks. This segmentation allows independent training and deployment of individual models without requiring extensive reprogramming of a large model, thereby reducing training time while maintaining customization capability.
Solution Approach 2:
The patent creates multiple copies of smaller microservice models that can be trained in parallel on different specialized tasks. Instead of extensively reprogramming one large model, multiple smaller model copies are trained independently and deployed as microservices, significantly reducing the time and computational resources required for customization.
2Adaptability or versatility
If a large LLM is extensively reprogrammed for customization, then the model can perform specialized tasks better, but the computational resources required become prohibitive
Solution Approach 1:
The patent segments the large LLM into smaller microservice models that can be trained and deployed independently. This reduces the computational resources required for each training task, as smaller models require less memory and processing power compared to extensively reprogramming a large model.
Solution Approach 2:
The patent employs multiple copies of smaller microservice models that can be trained in parallel using distributed computing. This approach distributes the computational burden across multiple smaller units rather than concentrating resources on extensively reprogramming one large model, thereby reducing overall computational resource requirements.
3Manufacturing precision
If skilled personnel create specialized training data manually, then the training data quality improves, but the labor effort and time required increase significantly
Solution Approach 1:
The patent uses trained microservice models to automatically generate specialized training data by copying and adapting patterns from existing high-quality data sources. This automated data generation maintains quality standards while dramatically increasing productivity compared to manual creation by skilled personnel.
Solution Approach 2:
The patent implements self-service mechanisms where the microservice models automatically generate their own training data using their learned capabilities. The models can autonomously create specialized training examples without requiring continuous manual intervention, thereby maintaining quality while improving data generation efficiency.
4Adaptability or versatility
If multiple customized ML tools are deployed, then the organizational capability to handle diverse tasks improves, but the human effort required for training data generation increases greatly
Solution Approach 1:
The patent segments the organizational capability into multiple independent microservice models, each specialized for specific tasks. This allows parallel development and deployment of customized tools without requiring sequential manual training data generation for each model, thereby maintaining organizational versatility while reducing the time and human effort required.
Solution Approach 2:
The patent uses copying mechanisms where trained microservice models can be replicated and adapted for new specialized tasks. Instead of manually generating training data for each new customized tool, existing models serve as templates that can be copied and fine-tuned, dramatically reducing the human effort and time required to deploy multiple customized ML tools.
Data Source
AI summary
Recorded sessions of skilled personnel at work are used to train a first machine learning (ML) tool to extract workflow maps. These maps are used to train a second ML tool to emulate at least one workflow. The first ML tool is trained to predict an annotator's output. Either ML tool can be a copilot having a microservice network architecture. Further, complex sets of workflows can be subdivided for efficient support with small ML tools for (1) workflow map extraction and (2) emulation. Based on segment demarcation accompanying a recording, segments are assigned to specialized ML extraction tools for workflow map extraction. Extracted workflow maps are used to train workflow emulator(s). Similarly, specialized ML emulation tools can emulate respective tasks. A distribution microservice can identify a task to be emulated and can invoke the appropriate specialized ML tool. Similar microservice network architectures support both applications.