Distributed Vision Code Transformation for Parallel API Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current implementations of Large Language Model (LLM) code generation are limited to serial execution on a single computing device, which restricts their applicability in artificial intelligence applications due to varying hardware configurations.
Innovation Solution
A method and system for transforming serial code generated by LLMs into distributed code that can be executed on a container orchestration platform cluster, utilizing a trained model to identify code dependencies and incorporating program semantics for concurrent execution across multiple computing devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If serial code is generated by LLM for vision applications, then code generation is simplified and can be executed on a single computing device, but execution efficiency is limited and cannot utilize diverse hardware configurations
Solution Approach 1:
The patent segments the monolithic serial code into multiple independent tasks or microservices that can be executed in parallel across different computing devices. The code transformation process identifies independent API calls and separates them into concurrent execution units, allowing the system to leverage multiple hardware resources simultaneously while maintaining the original code generation simplicity.
2Device complexity
If code is executed on a single computing device, then device complexity is reduced, but adaptability to varying hardware configurations is limited
Solution Approach 1:
The patent creates a universal execution framework where the transformed code can run on any computing device within a cluster, regardless of specific hardware configuration. The system uses standardized service calls and API interfaces that can be mapped to different hardware backends, enabling the same code to adapt to varying GPU, CPU, or specialized AI accelerator configurations without requiring device-specific modifications.
3Ease of manufacture
If serial code execution is used, then implementation is straightforward, but throughput is limited and latency is increased
Solution Approach 1:
The patent performs preliminary analysis of the serial code to identify independent API calls and execution paths before transformation. The trained model pre-processes the code to detect parallelization opportunities, marking sections that can be executed concurrently. This preliminary action enables the transformed code to automatically utilize parallel execution paths without requiring complex runtime analysis, thereby reducing latency while maintaining implementation simplicity.
4Productivity
If serial code is transformed to distributed code, then throughput is maximized through parallel processing, but code complexity increases
Solution Approach 1:
The patent introduces an intermediary transformation layer that converts serial code into distributed code with standardized service calls and API interfaces. This intermediary representation uses a consistent syntax and structure for parallel execution, masking the underlying complexity of distributed system coordination. The trained model automatically generates this intermediary form, allowing high throughput without exposing the full complexity of distributed computing to the original code author.
Data Source
AI summary
Systems and methods for generating and executing distributed code. The systems and methods include receiving serial code generated by a large language model (LLM) for vision applications and analyzing the serial code with a trained model to identify code dependencies and detect independent application programming interface (API) calls. The systems and methods further include transforming the serial code by incorporating program semantics that enable concurrent execution of the independent API calls and generating distributed code configured for execution on a container orchestration platform cluster, wherein the distributed code includes service calls that can be understood and executed by a runtime system.