Sorting Engine for Parallel Training Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems face bottlenecks in training data distribution due to slow single-stream distribution rates, and parallel distribution models fail to preserve the order of training data, leading to undesirable downstream effects.
Innovation Solution
A sorting engine with a shared buffer organizes batches of training data from multiple producers in a deterministic order, ensuring that batches are transmitted to the machine learning engine in a consistent sequence, using a multi-threaded retrieval engine and enqueue/dequeue modules to maintain order and manage concurrent data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If training data is distributed in parallel from multiple producers, then the distribution speed increases, but the order of training data distribution is not preserved
Solution Approach 1:
A sorting engine is introduced as an intermediary component between multiple training data producers and the machine learning system. The sorting engine receives data batches from multiple producers in parallel, sorts them according to their sequence numbers, and outputs them in the correct deterministic order, thus mediating between the parallel input streams and the ordered output requirement
Solution Approach 2:
The system segments the training data distribution into multiple independent streams from different producers, each handled by separate threads. Each thread retrieves data independently from its assigned producer, and the sorting engine reassembles them in the correct order, allowing parallel processing while maintaining global order
2Stability of the object's composition
If a single stream distribution model is used, then the order of training data is preserved, but the distribution rate is slow
Solution Approach 1:
The single distribution stream is segmented into multiple parallel streams, each handled by a separate thread retrieving data from different producers. This segmentation enables concurrent data retrieval while the sorting engine ensures the final output maintains the required deterministic order
Solution Approach 2:
The system uses a composite architecture combining multiple parallel data retrieval threads with a sorting engine. This composite structure integrates the speed benefits of parallel processing with the order preservation of sequential processing, achieving both high distribution rate and deterministic order
Data Source
AI summary
A sorting engine is an intermediary layer between a multi-threaded engine that obtains batches of training data from the producers in parallel and the underlying machine learning engine. The sorting engine includes a shared buffer that has various slots for storing batches of training data, where the slots are organized in a deterministic order associated with the producers. A batch of training data obtained by a thread from a given producer may be stored only in a corresponding slot in the shared buffer. Further, the sorting engine transmits the batch to the machine learning engine only when a previous batch in the deterministic order has been transmitted from the shared buffer to the machine learning engine.