Motion Vector Analysis for Video Endpoint Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multipoint video conference systems face inefficiencies in processing and switching between participant endpoints, particularly in determining which endpoints to display based on motion, as they often rely on computationally intensive methods or miss non-verbal cues due to sole reliance on audio signals.
Innovation Solution
The system calculates motion values from compressed video inputs using motion vectors, aggregates these values to determine the level of motion for each endpoint, and selectively displays endpoints with the highest motion, either solo or prominently, without the need for tracking specific video characteristics, thereby improving processing efficiency and capturing non-verbal communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If voice switching is used to select endpoints based on audio activity, then only actively speaking participants are shown, but non-verbal cues and silent participants are missed
Solution Approach 1:
The patent combines audio-based switching (voice activity detection) with motion-based switching (motion vector analysis) into a hybrid system. The MCU evaluates both audio and motion characteristics of video streams, allowing it to detect both verbal communication (audio) and non-verbal cues (motion), thereby resolving the contradiction by merging two complementary detection methods
Solution Approach 2:
The patent introduces motion vectors as an intermediary indicator to represent participant activity. Instead of directly analyzing complex video frames for non-verbal cues, the system uses motion vectors extracted during video encoding as a simplified intermediary that captures motion information, making the system both accurate and efficient
2Measurement precision
If motion analysis is performed on compressed video frames, then motion information can be extracted, but processing time and computational resources increase
Solution Approach 1:
The patent performs motion vector extraction during the video encoding process itself, which is a preliminary action completed before the MCU receives the compressed stream. Since motion vectors are already calculated and embedded in the compressed video data by the encoder, the MCU can directly use them without performing additional motion analysis, thus avoiding extra processing time while maintaining accuracy
Solution Approach 2:
The video encoding process serves dual purposes: it compresses the video data for transmission and simultaneously extracts motion vectors as a byproduct. The motion information is essentially self-generated during encoding, eliminating the need for separate motion analysis operations at the MCU
3Quantity of substance
If all video streams are transmitted to the MCU, then complete video data is available, but channel bandwidth and processing load increase
Solution Approach 1:
The patent implements dynamic video stream selection where the MCU dynamically determines which endpoints should transmit video based on real-time motion analysis. Instead of statically receiving all streams, the system adapts its behavior by selecting only the N endpoints with highest motion activity at any given time, optimizing bandwidth usage while maintaining communication quality
Data Source
AI summary
In one embodiment, an apparatus comprises a memory that stores executable instructions and a processor that executes the instructions in order to determine, for plural received compressed video inputs, at least one motion vector included in each of the plural compressed video inputs; calculate, based on the determined motion vectors, a motion value corresponding to each of the plural compressed video inputs, the motion values representing a level of motion over several frames for each of the plural compressed video inputs; and select, based on the calculated motion values, at least one of the plural compressed video inputs to be displayed.