Synchronizing Transcoded RTP Streams via RTCP NTP Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for synchronizing Real-time Transport Protocol (RTP) packet streams, particularly in transcoded and transrated audio and video data, face challenges due to the loss of original send time and inaccurate timestamp alignment, leading to desynchronization issues caused by intermediary devices like transcoders and transraters.
Innovation Solution
A method and apparatus that generate new RTP and RTCP packet streams by correlating RTP timestamp values with NTP timestamp values, ensuring synchronization by maintaining the original send time information through intermediary devices and using interpolation for accurate timestamp adjustments, thereby synchronizing audio and video streams effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RTP timestamp values are used for synchronization, then timing information is available for each packet, but original send time information is lost after transcoding and transrating
Solution Approach 1:
The patent introduces RTCP packets as an intermediary carrier that transports the original send time information (NTP timestamp) through the transcoding and transrating process. The RTCP packet acts as a mediator between the original RTP stream and the processed stream, preserving timing information that would otherwise be lost. The system extracts NTP timestamps from RTCP packets and uses them to correlate with new RTP timestamps after processing.
Solution Approach 2:
The patent creates a copy of the timing information by extracting NTP timestamps from RTCP packets and embedding them in the processed RTP packet stream. This copying mechanism ensures that the original send time information is replicated and preserved alongside the new RTP packets, allowing synchronization without relying on the original RTP timestamps which have been altered by processing.
2Adaptability or versatility
If transcoders and transraters are introduced to process audio and video streams, then format compatibility and data rate adaptation are improved, but synchronization accuracy deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors the correlation between NTP timestamps from RTCP packets and RTP packet arrival times. This feedback loop allows the system to detect and compensate for synchronization drift introduced by transcoders and transraters, maintaining accuracy despite the processing operations. The feedback enables dynamic adjustment of synchronization parameters.
3Ease of operation
If new RTP timestamp values are generated after transcoding and transrating, then packet processing flexibility is improved, but timestamp correlation with original send time is lost
Solution Approach 1:
The patent performs preliminary action by extracting and preserving the NTP timestamp information from RTCP packets before the transcoding and transrating processes alter the RTP stream. This preliminary capture of timing information ensures that even though new RTP timestamps are generated during processing, the original send time correlation is maintained through the pre-extracted NTP values that accompany the processed packets.
Data Source
AI summary
Methods and apparatus for synchronizing packet streams and in particular to the synchronization of transcoded and/or transrated Real-time Transport Protocol (RTP) packet streams, e.g., transcoded and/or transrated RTP packet streams carrying audio and/or video data. In one embodiment, a packet processing device receives packets of a first RTP packet stream and a first RTCP packet stream and generates a second RTP packet stream from the first RTP packet stream and generates a second RTCP packet stream from the first RTCP packet stream. The second RTP packet stream including packets with timestamps different than packets of the first RTP packet stream. The second RTCP packet stream including NTP timestamps received in or based on the NTP timestamps of the first RTCP packet stream and associated with RTP timestamps corresponding to content in the second RTP packet stream which was generated by transrating or trancoding content in the first RTP packet stream.