Order Preserving Data Processing via Differential Delay Queuing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data processing systems, the recombination of data streams processed through multiple paths often results in out-of-order blocks due to differential delays, which is particularly challenging in 'any-to-any' applications where standard sorting algorithms are impractical.
Innovation Solution
The method involves queuing and reordering data blocks based on a determined maximum differential delay among processing paths, ensuring that data blocks are transmitted in the correct order by using priority queues and FIFO queues to manage and resequence data blocks accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data blocks are distributed across multiple processing paths to increase throughput, then processing speed is improved, but data blocks may arrive out of order due to differential delays
Solution Approach 1:
The patent applies preliminary action by attaching sequence numbers to data blocks before distribution across multiple processing paths. This pre-labeling enables the reordering mechanism to reconstruct the original sequence at the output, resolving the contradiction between parallel processing and order preservation without requiring complex real-time synchronization
Solution Approach 2:
The patent introduces an intermediary reordering mechanism that receives out-of-order data blocks from multiple processing paths and reconstructs them in the correct sequence using the attached sequence numbers. This intermediary component acts as a buffer between the parallel processing paths and the output, eliminating the direct conflict between throughput enhancement and order preservation
2Manufacturing precision
If standard sorting algorithms are used to restore order in any-to-any applications, then data block order can be restored, but the complexity increases and standard algorithms become impractical
Solution Approach 1:
By pre-attaching sequence numbers to data blocks before distribution, the patent eliminates the need for complex sorting algorithms at the reordering stage. The preliminary labeling simplifies the reordering process to a straightforward sequence reconstruction, reducing complexity while maintaining order restoration capability in any-to-any applications
Solution Approach 2:
The patent changes the parameter representation by using sequence numbers as a key attribute attached to each data block. This parameter transformation enables efficient reordering through simple comparison and sorting of sequence numbers rather than complex content-based sorting, significantly reducing the computational complexity of the reordering operation
Data Source
AI summary
A data processing system includes an input circuit, a plurality of processing paths and an output circuit. The input circuit receives blocks of data on a plurality of data streams and distributes the blocks of data to the plurality of processing paths. The plurality of processing paths receive and process the distributed blocks of data. The output circuit selectively queues and dequeues the processed blocks of data based on a determined maximum differential delay among each of the processing paths and transmits the processed blocks of data.