Autobatch Buffer for Parallel Thread Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer processing systems face challenges in efficiently managing multiple thread requests in parallel, leading to increased memory and processor operations, which can result in decreased throughput and scalability issues.
Innovation Solution
The implementation of an autobatch buffer system that batches multiple thread requests as a set, storing them in linear buffers and submitting them in a single operation, with one thread designated as a control thread to manage atomic operations and buffer retirement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple thread requests are managed individually in parallel processing systems, then processing capability is improved, but memory consumption and processor management operations increase
Solution Approach 1:
The patent combines multiple individual thread requests into a single batch operation. Instead of managing each thread request separately with individual memory allocations and management operations, the system merges them into a unified batch that is processed as one unit. This merging reduces the total memory overhead and management operations while maintaining the parallel processing capability of handling multiple threads simultaneously.
2Adaptability or versatility
If multiple thread requests are managed individually, then processing flexibility is maintained, but processor time for management operations increases
Solution Approach 1:
The system merges multiple thread management operations into a single batch submission and retirement operation. By combining these operations, the processor executes fewer management instructions overall, reducing the time lost to management overhead while still maintaining flexibility through the batch interface that can accommodate variable numbers of threads and different thread configurations.
Solution Approach 2:
The batch operation is prepared in advance with all thread requests collected and organized before execution. This preliminary organization allows the system to submit multiple threads in a single operation rather than sequentially managing each thread, thereby reducing the total processor time required for management while maintaining adaptability.
3Ease of operation
If thread requests are processed sequentially for management, then control is simplified, but throughput decreases
Solution Approach 1:
The patent segments the batch operation into distinct phases: a control thread that manages the batch lifecycle (submission, execution monitoring, retirement) and worker threads that perform the actual processing tasks. This segmentation allows the control thread to maintain simple sequential management logic while worker threads execute in parallel, thereby preserving control simplicity while achieving high throughput through parallel execution.
Solution Approach 2:
The control thread acts as an intermediary between the batch submission interface and the worker threads. It receives the batch operation, prepares it for execution, monitors progress, and manages retirement. This intermediary structure simplifies control by centralizing management logic in one thread while enabling parallel execution of multiple worker threads, thus maintaining ease of operation while improving throughput.
Data Source
AI summary
In managing a multiple threaded processing system, it can be difficult to store and commit each thread. To reduce the computing resources needed to manage the multiple threads, a linear autobatch buffer process can be utilized. As thread requests are received, an appropriate autobatch buffer can be identified. If the autobatch buffer is full, an increment can be used to select a different autobatch buffer as the active buffer. Once each thread in the autobatch buffer are committed and completed, the buffer can be retired. One thread in the buffer is designated the control thread and is responsible for controlling and retiring the buffer so that the other threads do not expend resources checking on the buffer status.