Data Center Task Processing via Parallel Worker Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data center resource management is bottlenecked by sequential processing of tasks, limiting the scalability and efficiency of resource management in large-scale networks.
Innovation Solution
The implementation of a task data structure that allows multiple task data elements to be created and processed asynchronously by different worker processes, enabling parallel execution of tasks and workflows, thereby breaking down business logic into discrete, composable tasks that can be executed in parallel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential processing of tasks is used to manage data center resources, then system simplicity is maintained, but productivity and scalability are limited due to bottlenecked task handling
Solution Approach 1:
The patent segments the monolithic task processing system into multiple independent worker processes, each capable of handling tasks autonomously. The task queue is divided into discrete task data elements that can be independently processed. This segmentation enables parallel task execution across multiple workers, dramatically increasing throughput while maintaining manageable complexity through modular design
Solution Approach 2:
The patent transitions from one-dimensional sequential processing to multi-dimensional parallel processing by introducing multiple worker processes operating simultaneously. Tasks are distributed across the worker dimension, allowing concurrent execution. This dimensional expansion resolves the bottleneck without requiring complex coordination mechanisms
2Productivity
If parallel execution of tasks is implemented, then productivity and resource utilization are improved, but system complexity increases due to coordination requirements
Solution Approach 1:
The patent introduces a task queue as an intermediary data structure that mediates between task sources and worker processes. Workers asynchronously consume tasks from this shared queue without direct coordination or communication with each other. This intermediary eliminates the need for complex inter-worker coordination while enabling parallel execution, resolving the contradiction between productivity and complexity
3Loss of time
If sequential task handling is used, then ease of operation is maintained, but loss of time increases due to bottlenecked processing
Solution Approach 1:
The patent implements self-service through asynchronous worker processes that independently retrieve and execute tasks from the shared queue without requiring coordination or approval from other workers. Each worker autonomously manages its own task execution, eliminating waiting time and bottlenecks while maintaining operational simplicity through the self-organizing nature of the system
Data Source
AI summary
An example method includes receiving a resource management request associated with resources provided by at least one data center, creating, based on the resource management request, task data elements including at least first and second task data elements, adding the task data elements to a task data structure accessible at least by a first and second worker processes, removing, by the first worker process, a first task data element from the task data structure and initiate execution of a first task, removing, by the second worker process, a second task data element from the task data structure and initiate execution of a second task, wherein the second worker process executes at least a portion of the second task while the first worker process executes at least a portion of the first task in parallel, and sending, to the client computing device, a response to the resource management request.