Video Frame Data Unit Ordering for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing video streaming technologies face challenges in efficiently delivering video data units, such as NAL units, in an order different from the normative order specified by standards like HEVC, leading to increased latency and processing resource requirements due to the need for reordering at both the sender and receiver devices.
Innovation Solution
A method that determines whether data units are likely to be received out of order and encapsulates them with ordering information, allowing receivers to declare non-normative order and enabling flexible transmission and decoding without the need for reordering, thereby reducing latency and processing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data units are transmitted in normative decoding order, then decoder compatibility is ensured, but transmission latency increases due to reordering requirements
Solution Approach 1:
The transmitter performs preliminary assessment of receiver capabilities and network conditions before transmission, determining whether out-of-order delivery is acceptable. This advance preparation allows the system to skip the reordering step at the receiver end, reducing transmission latency while maintaining compatibility when needed.
Solution Approach 2:
The system dynamically adapts the transmission order of data units based on real-time assessment of receiver capabilities and network conditions. When receivers can handle out-of-order data and network conditions permit, the system transmits data units immediately without reordering, optimizing latency while preserving compatibility when required.
2Reliability
If reordering of NAL units is performed at sender device, then normative order is ensured, but processing resources and time are consumed
Solution Approach 1:
The system eliminates the need for sender-side reordering by having receivers that can self-handle out-of-order data units. Receivers with appropriate capabilities process data units in the order received, and only perform reordering if absolutely necessary, thereby improving processing efficiency while maintaining normative order compliance when required.
Solution Approach 2:
The system changes the operational parameters of the transmission system by allowing flexible data unit ordering based on receiver capabilities. Instead of enforcing normative order universally, the system adapts the ordering parameter dynamically, eliminating unnecessary reordering operations at the sender and improving processing efficiency.
3Reliability
If reordering is performed at receiver device, then normative order is achieved, but additional latency and processing resources are required
Solution Approach 1:
The invention extracts the reordering function from the receiver's critical path by having the transmitter assess receiver capabilities in advance. When receivers indicate they can handle out-of-order data, the transmitter sends data units without reordering, and receivers process them immediately, eliminating the reordering latency at the receiver end while maintaining normative order compliance when necessary.
Solution Approach 2:
The transmitter performs preliminary assessment of receiver capabilities before transmission, determining whether out-of-order delivery is acceptable. This advance preparation allows the system to skip the reordering step at the receiver end, reducing reception latency while maintaining compatibility when needed.
4Productivity
If parallel processing of partitioned data is implemented, then encoding efficiency is improved, but data units may be generated in non-normative order
Solution Approach 1:
The system dynamically adapts to the output order from parallel processing operations. Instead of enforcing a fixed normative order, the system flexibly handles data units in the order they are generated by parallel encoders, transmitting them immediately without reordering when receivers can handle out-of-order data, thereby preserving encoding efficiency while maintaining reliability.
Solution Approach 2:
The transmitter performs preliminary assessment of receiver capabilities to determine whether out-of-order delivery is acceptable before transmission. This advance preparation allows the system to maintain the parallel processing-generated order without reordering, preserving encoding efficiency while ensuring reliability through capability-matched transmission.
Data Source
AI summary
A method of encapsulating data units of at least one encoded video frame into a data stream, said data units representing frame portions of the video frame, wherein said data stream is associated with an ordering information indicating the compliance of the order of the data units with a nominal data unit decoding order. Embodiments of the invention provide flexible transmission with robust and flexible decoders.