Dynamic Parallel Processing Framework for Server Load Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for parallel processing of subprocesses lack flexibility and efficiency, as they require manual configuration and static load distribution among servers, leading to suboptimal processing times and inability to dynamically adjust workload distribution.
Innovation Solution
A computer-implemented method and system for dynamic parallel processing that builds work packages from data objects according to a process-specific scheme and dynamically distributes them among servers based on a predefined execution order, using a parallel processing framework to optimize resource utilization and minimize idle time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual configuration is used for parallel processing, then process control is possible, but system complexity and processing time increase
Solution Approach 1:
The system automatically distributes work packages to servers based on predefined execution orders and process-specific schemes, eliminating the need for manual configuration and dispatching of each subprocess. The parallel processing framework self-manages the distribution and coordination of multiple subprocesses across available servers.
2Productivity
If static load distribution is used among servers, then configuration is simple, but processing efficiency and resource utilization deteriorate
Solution Approach 1:
The system dynamically distributes work packages to servers based on real-time availability and process-specific execution orders, rather than using static load distribution. The framework adapts to changing system conditions and optimizes resource allocation during runtime, improving processing efficiency while managing complexity through automated decision-making.
3Adaptability or versatility
If manual job definition is done for each job, then customization is possible, but time consumption and operational complexity increase
Solution Approach 1:
The parallel processing framework provides a universal mechanism for defining and executing multiple subprocesses simultaneously. Instead of manually defining each job separately, the framework accepts a process description and automatically generates and manages multiple customized subprocesses based on process-specific schemes, reducing time consumption while maintaining adaptability.
4Speed
If parallel processing is implemented without dynamic distribution, then implementation is simpler, but resource utilization and processing speed deteriorate
Solution Approach 1:
The parallel processing framework acts as an intermediary layer between the process definition and server execution. It receives process descriptions, builds work packages according to process-specific schemes, and dynamically distributes them to appropriate servers. This intermediary mechanism enables fast parallel processing while managing distribution complexity through centralized intelligent control.
Data Source
AI summary
Systems and methods are disclosed for dynamic parallel processing of a plurality of subprocesses of a process initiated by a parallel processing framework. The plurality of subprocesses may include processing of one or more data objects in a computer system including a plurality of servers for processing the plurality of data objects by the plurality of subprocesses. The method may include building work packages from the plurality of data objects according to a process-specific scheme, and dynamically distributing the work packages of the plurality of data objects among the plurality of servers according to a process-specific execution order.