Parallel Execution Controller for Partitioned Data Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex machine-learning workloads often overwhelm single computing systems, and while grid computing increases resource availability, it does not reduce workload complexity, making it difficult to process with fewer, less sophisticated resources.
Innovation Solution
A parallel execution controller partitions complex data models into less complex sub-models that can be executed independently on less sophisticated network resources, allowing for aggregation of results to form the original model, thereby reducing processing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If grid computing is used to increase resource availability, then more computational resources are available to process complex workloads, but the workload complexity remains unchanged and requires sophisticated computational resources
Solution Approach 1:
The patent applies segmentation by dividing a complex data model into multiple independent sub-models that can be executed in parallel. Each sub-model processes a specific partition of the data, transforming a single complex computational task into multiple simpler tasks that can run simultaneously on less sophisticated resources, thereby reducing the complexity requirement while maintaining processing capability
2Device complexity
If a single computing system is used, then the system is simple to manage, but it struggles to meet the processing demands of complex machine-learning workloads
Solution Approach 1:
The system segments the complex workload into independent sub-models that can be distributed across multiple computing resources. This allows a single logical system to leverage multiple physical resources, significantly increasing processing capability while maintaining manageable system architecture through the parallel execution framework
Solution Approach 2:
The patent introduces parallel execution as an additional dimension for processing. By executing multiple sub-models simultaneously across different computing resources rather than sequentially on a single system, the processing capability is dramatically increased without proportionally increasing system management complexity
3Measurement precision
If complex data models are processed, then accurate results are obtained, but the processing requires sophisticated and numerous computational resources
Solution Approach 1:
The complex data model is segmented into multiple sub-models, each processing a specific data partition. This segmentation maintains results accuracy because each sub-model independently processes its designated portion with the same level of precision, while the overall complexity is distributed across simpler computational resources rather than concentrated in a single sophisticated system
Data Source
AI summary
This disclosure describes techniques that enable a parallel execution controller to execute partitioned segments of a data model in parallel and on different network resources. The resultant processed data from each partitioned segment may be combined to form an aggregated set of results data. The parallel execution controller may identify model attributes of a data model and determine whether the data model is partitionable, based on the model attributes. If the data model is partitionable, the parallel execution controller may partition the data model, initiate a parallel execution of the individual partitioned segments on available network resources, and combine the corresponding results data into an aggregated set of results data.