Multimodal Model Data Pipeline Generation via Action Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data pipeline generation systems rely on large language models (LLMs) that are limited by their training data, leading to inefficiencies, incompatibilities, and inconsistencies in code generation, which undermines reliability and agility in responding to technological changes.
Innovation Solution
A data pipeline generation system that uses a multimodal model (MM) to generate action labels and data source labels, which are then mapped to proprietary or platform-specific nodes, allowing for the creation of a project template that can be customized and executed without exposing underlying code to the MM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If large language models (LLMs) are used for data pipeline generation, then code generation capability is provided, but reliability and consistency deteriorate due to training data limitations
Solution Approach 1:
The patent introduces an intermediary layer between the user's natural language requests and the actual code generation. A project template with defined nodes and connections serves as a mediator that structures the pipeline generation process, ensuring consistent and reliable output while maintaining ease of operation through high-level abstractions.
Solution Approach 2:
The data pipeline is segmented into discrete nodes (each representing a specific operation) and connections (representing data flow). This segmentation allows the system to generate reliable, consistent code by composing standardized units rather than generating monolithic pipelines, addressing the reliability issue while preserving operational ease.
2Adaptability or versatility
If proprietary or platform-specific nodes are used, then adaptability to specific systems is improved, but compatibility with general models deteriorates
Solution Approach 1:
The project template defines a universal interface for data pipeline nodes that can work across different proprietary systems. Each node is designed with standardized inputs and outputs that can be mapped to various platform-specific implementations, allowing the same template to adapt to different proprietary systems without increasing model integration complexity.
3Measurement precision
If code is exposed to the multimodal model, then model training accuracy is improved, but data privacy and security deteriorate
Solution Approach 1:
The patent extracts the actual implementation code from the data pipeline definition. Only the high-level project template (nodes and connections) is exposed to the multimodal model for generation, while the actual proprietary code remains hidden and is only instantiated from the template during execution, thus maintaining data privacy while enabling accurate model training on abstracted representations.
4Productivity
If LLMs are used for pipeline generation, then development speed is improved, but responsiveness to technological changes deteriorates
Solution Approach 1:
The project template system is dynamic and can be easily updated to accommodate new technological changes. When new nodes or connections are needed, they can be added to the template without requiring retraining of the multimodal model, allowing the system to maintain high development speed while becoming increasingly agile to technological advancements.
Data Source
AI summary
Disclosed herein are system, method, and computer program product aspects for generating a data pipeline. A model prompt including a received natural language description and a prompt template is generated. The prompt template includes action labels and a processing example. Each action label indicates a respective data processing action, and the processing example includes a sample query and a sample answer comprising one or more sample action labels associated with a sample natural language description of a sample data pipeline. A multimodal model (MM) is queried with the model prompt. The MM response includes one or more action labels corresponding to the natural language description of the requested data pipeline in a format guided by the prompt template. A data pipeline project template can then be generated using one or more executable nodes corresponding to the action labels.