Media Streaming Endpoint Offload for RTP Packetization Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based media rendering and streaming systems face issues of CPU and GPU overutilization, leading to increased latency and memory bandwidth overuse due to video and audio processing, which can be addressed by offloading media frame segmentation and RTP packetization tasks to a media streaming endpoint (EP).
Innovation Solution
Implementing a media streaming endpoint (EP) that performs offloaded tasks such as media frame segmentation and RTP packetization, reducing the burden on CPUs and GPUs by handling these tasks directly, thereby optimizing memory bandwidth usage and lowering latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If media processing tasks (encoding, packetization, decoding, reassembly) are performed by host CPU and GPU, then media transmission and reconstruction can be completed, but CPU cycles and memory bandwidth are overutilized causing increased latency
Solution Approach 1:
The patent extracts media processing tasks (encoding, packetization, decoding, reassembly) from the host CPU and GPU and relocates them to a dedicated network interface device. This separation allows the host processing units to focus on other tasks while the network device handles media stream operations independently, thereby reducing CPU cycle consumption and memory bandwidth pressure that caused latency.
Solution Approach 2:
The network interface device acts as an intermediary between the host CPU/GPU and the media stream transmission. It receives media frames from the host, performs processing tasks, and transmits packets to the network, or vice versa for reception. This intermediary role eliminates the need for continuous CPU/GPU involvement in media processing, reducing overall system latency.
2Productivity
If media frame segmentation and RTP packetization are performed by host CPU, then media can be transmitted, but memory bandwidth is overutilized reducing transmission rate
Solution Approach 1:
The patent extracts memory-intensive operations (media frame segmentation, RTP packetization, de-packetization, reassembly) from the host memory system and performs them in the network interface device. This relocation significantly reduces the quantity of data that must be copied through the host memory bandwidth, thereby increasing the effective media transmission rate.
3Productivity
If CPU performs video and audio processing tasks, then media can be processed, but CPU cycles are consumed reducing availability for other tasks
Solution Approach 1:
The patent extracts media processing capabilities from the CPU and relocates them to the network interface device. This allows the CPU to maintain media processing capability through the network device while becoming available for other system tasks, thereby improving overall system adaptability and versatility.
4Productivity
If media encoding and decoding are performed using host memory, then media can be processed, but host memory bandwidth is overutilized
Solution Approach 1:
The patent extracts encoding and decoding operations from the host memory system and implements them in the network interface device. This relocation reduces the volume of data traffic through the host memory bandwidth, allowing high-throughput media processing without overutilizing the host memory subsystem.
Data Source
AI summary
Examples described herein relate to a network interface device that includes circuitry to: receive a request to perform offloaded media encoding and packetization prior to transmission of the media to a receiver, wherein the media comprises video and audio; perform encoding of the media; perform packetization of the encoded media into one or more packets based on Real-time Transport Protocol (RTP) protocol; and cause transmission of the one or more packets to the receiver.