Composite Frame Packing for 360-Degree Video Bandwidth Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for capturing and streaming 360-degree views with fisheye lenses face bandwidth inefficiencies due to unused portions of frames, as the captured images only occupy a central portion of the sensor, leading to wasted bandwidth and the need for methods to efficiently communicate additional image content without compromising the viewing experience.
Innovation Solution
The method involves generating composite images by combining portions of images from multiple cameras, including those captured with fisheye lenses, with additional content such as logos, textures, or advertisements, and encoding these composite images using HD or UHD encoders, allowing for efficient use of bandwidth by incorporating unused frame areas with supplementary content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If fisheye lenses are used to capture wide viewing area images, then the viewing area coverage is improved, but bandwidth efficiency deteriorates due to unused frame portions
Solution Approach 1:
The patent combines the captured image from the fisheye lens with additional image content (such as advertisements, logos, or other visual information) into a single composite frame. This merging allows the previously unused portions of the frame to carry useful content, thereby improving bandwidth efficiency while maintaining the wide viewing area coverage benefit of fisheye lenses.
2Measurement precision
If full 360 degree view in full high definition is streamed, then the viewing quality is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent merges multiple content sources (captured images and additional image content) into a single composite frame that is then encoded and transmitted. This approach allows high-definition quality content to be delivered while utilizing the entire frame area efficiently, reducing the overall bandwidth required compared to transmitting separate streams for captured content and additional content.
3Ease of manufacture
If unused frame portions are left blank, then the encoding process is simple, but bandwidth is wasted
Solution Approach 1:
The patent combines captured image content with additional image content into a composite frame before encoding. This maintains relative encoding simplicity while eliminating bandwidth waste by ensuring that every portion of the transmitted frame contains useful information rather than blank or unused areas.
4Loss of energy
If additional image content is incorporated into frames, then bandwidth efficiency is improved, but device complexity increases
Solution Approach 1:
The patent segments the frame into different content areas, with some portions containing captured image content and other portions containing additional image content. This segmentation allows the system to manage complexity by treating different frame regions separately while still achieving improved bandwidth efficiency through the composite frame approach.
Data Source
AI summary
Methods and apparatus for packing images into a frame and/or including additional content and/or graphics are described. A composite image is generated including at least one image in addition to another image and/or additional image content, e.g., a logo, texture, sign, or advertisement. In some embodiments, first and second pairs of stereoscopic images are combined, e.g., with additional image content to generate a composite image which is then encoded, e.g., using a UHD (Ultra High Definition) encoder. In some other embodiments, the rather than two pairs of stereo images a pair of stereo images are combined with images captured by two mono cameras. In various embodiments, the set of cameras which are sources of captured images for the composite image are dynamically selected, e.g., with different sets of cameras in a camera rig being selected at different times.