360-Degree Video Object Tracking via Pre-Computed Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360-degree video capture and playback technologies face challenges in efficiently encoding, decoding, and rendering immersive videos with object tracking, leading to spatial discontinuities and high computational requirements, which affect rendering quality and user experience.
Innovation Solution
The system employs a network environment with a 360-degree video capture device, stitching device, encoding device, decoding device, and rendering device, utilizing equirectangular and cube projection formats to minimize spatial discontinuities and implement object tracking, allowing users to control field of view and viewing directions for immersive experiences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 360-degree video is captured and rendered with object tracking, then immersive experience and object tracking capability are improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent performs preliminary actions by pre-processing the 360-degree video to identify and tag objects of interest before rendering. This allows the rendering device to quickly locate and track objects without performing complex real-time analysis during playback, thus reducing computational complexity while maintaining object tracking capability
Solution Approach 2:
The patent introduces an intermediary layer (metadata or indexing structure) that stores pre-computed object information such as bounding boxes, tracking IDs, and spatial coordinates. This intermediary enables the rendering device to efficiently access and track objects without directly processing the entire 360-degree video stream, thereby reducing computational burden
2Manufacturing precision
If equirectangular and cube projection formats are used to minimize spatial discontinuities, then rendering quality is improved, but encoding and decoding complexity increases
Solution Approach 1:
The patent segments the 360-degree video into multiple projection formats (equirectangular and cube) and encodes them separately. This segmentation allows the system to select the most appropriate format for different viewing scenarios, minimizing spatial discontinuities in each format while managing encoding complexity through divide-and-conquer approach
Solution Approach 2:
The patent changes the projection parameter (from single format to multiple formats) to optimize rendering quality. By providing both equirectangular and cube projection formats, the system can adapt to different viewing conditions and minimize spatial discontinuities, with the encoding complexity managed through standardized encoding processes for each format
3Ease of operation
If 360-degree video is rendered with user control of field of view and viewing directions, then user experience and interactivity are improved, but rendering time and processing requirements increase
Solution Approach 1:
The patent performs preliminary rendering preparations by pre-processing the 360-degree video into multiple projection formats and pre-computing rendering parameters. This allows the rendering device to quickly adapt to user-controlled field of view and viewing directions without performing complex real-time rendering calculations, thus reducing rendering time while maintaining user interactivity
Solution Approach 2:
The patent implements dynamic rendering that adapts to user-controlled field of view and viewing directions in real-time. By using pre-processed multiple projection formats, the rendering system can dynamically adjust the displayed view without significant computational overhead, enabling user interactivity with acceptable rendering time
Data Source
AI summary
In a system for 360 degree video capture and playback, 360 degree video may be captured, stitched, encoded, decoded, rendered, and played-back. In one or more implementations, a decoding device receives a 360 degree video stream as input and decodes the 360 degree video stream, and a memory device stores the 360 degree video stream and viewing history data associated with the 360 degree video stream. A rendering device may render the decoded stream using view angles from the viewing history data. In one or more implementations, an object tracking device tracks one or more objects in the decoded 360 degree video stream and provides one or more tracking angles associated with the objects. The rendering device may render the decoded 360 degree video stream using the one or more tracking angles to keep at least one object in the 360 degree video stream for one or more rendered frames.