Asynchronous Stream Parsing for Out-of-Order Protocol Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data stream parsing technologies struggle with handling out-of-order, duplicate, and overlapping data units, leading to inefficiencies and errors in data processing.
Innovation Solution
A system that reorders out-of-order data units, discards duplicates, and generates new units to replace overlapping ones, maintaining an ordered queue for further processing, and organizes protocol frames by sorting them into data streams while eliminating duplicates and overlaps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data stream parsing is performed using traditional sequential methods, then data processing completeness can be maintained, but processing efficiency and speed deteriorate due to the need to handle out-of-order, duplicate, and overlapping data units
Solution Approach 1:
The patent segments the data stream processing into distinct operational phases: receiving data units, determining their status (out-of-order, duplicate, overlapping), and applying specific handling methods for each type. This segmentation allows parallel processing of different data unit types, improving efficiency while maintaining accuracy through specialized handling for each category.
Solution Approach 2:
The patent performs preliminary classification of incoming data units by determining their status (out-of-order, duplicate, overlapping) before main processing. This preliminary action enables the system to prepare appropriate handling strategies in advance, reducing processing time and improving throughput while ensuring correct treatment of each data unit type.
2Quantity of substance
If all incoming data units are processed and stored, then data completeness is improved, but system resource consumption and complexity increase due to handling duplicates and overlaps
Solution Approach 1:
The patent extracts and identifies duplicate and overlapping data units from the incoming stream, separating them from unique valid data. By taking out duplicates and overlaps for special handling (discarding or merging), the system maintains data completeness of unique information while reducing the burden of processing redundant data, thereby simplifying system complexity.
Solution Approach 2:
The patent discards duplicate data units and merges overlapping data units, keeping only the necessary unique information. This selective discarding and recovering approach maintains data completeness by preserving all unique data while eliminating redundancy, reducing system resource consumption and complexity associated with storing and processing unnecessary duplicates.
3Speed
If data units are processed in strict sequential order, then data consistency is maintained, but processing speed deteriorates due to waiting for out-of-order units
Solution Approach 1:
The patent implements dynamic processing where data units are handled based on their specific status (out-of-order, duplicate, overlapping) rather than following a rigid sequential order. The system adapts its processing approach for each data unit type, allowing out-of-order units to be identified and handled appropriately, thereby improving processing speed while maintaining data consistency through status-based dynamic routing.
Solution Approach 2:
The patent changes the processing parameter from strict sequential ordering to status-based processing. By determining the status parameter of each data unit (out-of-order, duplicate, overlapping) and adjusting handling methods accordingly, the system improves processing speed by not waiting for strict ordering while maintaining consistency through parameter-driven processing decisions.
Data Source
AI summary
Techniques for parsing stream data asynchronously are disclosed. A system translates protocol frames that are delivered asynchronously into an ordered flow by reordering protocol frames that arrive out of order, dropping duplicate protocol frames, and eliminating overlaps between protocol frames. When a protocol frame is delivered to the system, the system compares a frame offset of the protocol frame to an expected offset that is maintained by the system. If the frame offset is lower than the expected offset, the system drops at least a part of that protocol frame. If the frame offset matches the expected offset, the system releases at least part of that protocol frame for further processing. If the frame offset is greater than the expected offset, the system adds at least part of that protocol frame to an ordered queue of frames that are being held in buffer memory by the system.