Intermediary Video Analysis for Reduced Transmission Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing techniques, such as Network-Based Media Processing (NBMP), introduce significant delays in video stream transmission due to decoding and encoding operations, which are undesirable for real-time communication and can burden the system, especially in applications like VR and AR where computationally complex processing is required.
Innovation Solution
A processor system is configured to receive and decode video streams, analyze the video data, and generate processing assist data, which is then sent to the receiver system to enable it to process the video without the need for decoding and re-encoding, thereby reducing transmission delay by allowing parallel decoding and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video processing is performed in the cloud using NBMP techniques, then complex video processing tasks can be handled, but additional decoding and encoding operations introduce significant delay
Solution Approach 1:
The patent extracts only the essential analysis information from the video stream rather than performing complete decoding and re-encoding. The intermediary system analyzes the compressed video stream to extract key features and generates processing assist data, eliminating the need for full decode-process-encode cycles and reducing transmission delay while maintaining processing capability.
Solution Approach 2:
The patent performs preliminary analysis of the video stream in compressed form before transmission to the receiver. The intermediary system prepares processing assist data that contains pre-computed information about the video content, allowing the receiver to perform complex processing operations more efficiently without waiting for complete decoding of the entire stream.
2Reliability
If complete video decoding and re-encoding is performed, then processed video can be obtained, but system burden increases and performance suffers
Solution Approach 1:
The patent creates a simplified representation or copy of the video content in the form of processing assist data that contains essential information needed for processing. Instead of working with the complete decoded video stream, the system uses this compressed representation to guide processing operations, reducing computational burden while maintaining processing accuracy.
3Loss of time
If video stream is transmitted without preprocessing, then transmission delay is minimized, but receiver system must handle complex processing alone
Solution Approach 1:
The patent introduces an intermediary system that acts as a mediator between the video source and the receiver. This intermediary performs preliminary analysis and generates processing assist data that is transmitted along with the video stream, enabling the receiver to perform complex processing operations more easily without significantly increasing transmission delay.
4Loss of time
If real-time communication is prioritized, then delay must be kept below 150ms, but NBMP operations cannot meet this requirement
Solution Approach 1:
The patent extracts only the critical processing information from the video stream in compressed form, avoiding the time-consuming complete decode-process-encode cycle of traditional NBMP. This extraction approach enables real-time processing with delay below 150ms while maintaining adequate processing throughput for VR and AR applications.
Data Source
AI summary
An intermediary system and method may be provided for assisting a receiver system in processing video data which is streamed as a video stream to the receiver system via a network. The processing of the video data by the receiver system may be dependent on an analysis of the video data. The intermediary system may provide processing assist data to the receiver system which comprises an analysis result or a processing instruction derived from the analysis results. Accordingly, the receiver system may process the video data without a need for the receiver system itself to analyze the video data, thereby offloading computational complexity to the intermediary system. Compared to techniques in which most or all of the processing is performed by the intermediary system, an advantage of continuing to process the video data at the receiver system is that the receiver system may already decode the video stream while the video stream is decoded and/or analyzed by the intermediary system, thereby reducing the delay from transmission to display of the video stream.