Indoor Panoramic Video Generation Using Spherical Coordinate Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating indoor panoramic videos are costly and suffer from poor real-time performance due to the need for multiple cameras and complex image stitching algorithms, or result in two-dimensional images with limited user experience when using fish-eye cameras.
Innovation Solution
A method and apparatus that convert fish-eye video frames into spherical coordinate system-based images, determine frustum parameters based on the room shape, and render texture images onto corresponding faces to generate panoramic videos, eliminating the need for complex stitching and reducing camera costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple cameras or aerial camera are used to capture different viewing angles, then panoramic video quality is improved, but system cost increases
Solution Approach 1:
The panoramic video generation process is segmented into distinct processing stages: fish-eye video input, spherical coordinate conversion, frustum parameter determination, texture image extraction, and rendering. This segmentation allows a single camera to capture all necessary data that would traditionally require multiple cameras, reducing hardware costs while maintaining video quality.
Solution Approach 2:
The patent transforms the two-dimensional fish-eye video frames into a three-dimensional spherical coordinate system representation, then projects them onto multiple faces of a virtual room model. This dimensional transformation enables a single camera to capture panoramic information that traditionally required multiple cameras positioned at different locations.
2Manufacturing precision
If image stitching algorithm is used to combine multiple video images, then panoramic video is generated, but processing complexity increases
Solution Approach 1:
The patent replaces the mechanical image stitching process with a mathematical coordinate transformation system. Instead of detecting features, matching points, and warping images across multiple frames, the system converts all images to spherical coordinates and renders them onto a standardized virtual room model, eliminating the complexity of traditional stitching algorithms.
Solution Approach 2:
The patent changes the parameter space from two-dimensional image coordinates to three-dimensional spherical coordinates (azimuth, elevation, radius). This parameter transformation provides a unified framework for processing all views simultaneously, avoiding the iterative matching and adjustment required by traditional stitching methods.
3Quantity of substance
If fish-eye camera with very large viewing angle is used, then single camera cost is reduced, but viewing effect becomes two-dimensional
Solution Approach 1:
The patent adds the third dimension by mapping the two-dimensional fish-eye image onto a three-dimensional spherical coordinate system and then projecting it onto multiple faces of a virtual room. This transformation converts the inherently two-dimensional fish-eye capture into a stereoscopic multi-face representation, providing immersive 360-degree viewing effects.
Solution Approach 2:
The patent utilizes the spherical nature of fish-eye lens projection by converting images into spherical coordinates and mapping them onto the surfaces of a virtual spherical room. This approach preserves the curved field-of-view characteristics of the fish-eye lens while organizing the information into a structured three-dimensional format that provides authentic panoramic viewing experience.
4Manufacturing precision
If complex image stitching algorithm is used, then panoramic video is generated, but real-time performance deteriorates
Solution Approach 1:
The patent performs preliminary conversion of all input frames to spherical coordinates at the beginning of processing, establishing a unified coordinate framework before any rendering operations. This pre-processing step eliminates the need for repeated coordinate transformations and iterative adjustments during stitching, enabling real-time processing of panoramic video streams.
Solution Approach 2:
The patent replaces the computationally intensive mechanical stitching process with efficient mathematical projection operations. By using predefined frustum parameters and standardized rendering equations, the system achieves real-time performance through optimized linear algebra operations rather than iterative image processing.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Embodiments of the present application disclose a method and apparatus for generating an indoor panoramic video. For each of frames of the fish-eye video, coordinates of each of pixels of this frame in an image coordinate system are converted into coordinates in a spherical coordinate system to obtain a spherical coordinate system-based hemispherical fish-eye image. The frustum parameters of each of the N texture images of N viewing angles for the hemispherical fish-eye image are determined according to a shape of a preset living room. Based on the frustum parameters of each of the N texture images of N viewing angles, the N texture images of N viewing angles for the hemispherical fish-eye image are obtained. The N texture images of N viewing angles are rendered onto the N faces inside the preset living room, to generate the panoramic video image corresponding to the frame. As such, in the embodiment of the present application, a panoramic video image having a stereoscopic effect can be generated. The real-time performance for generating a panoramic video is improved, as no complicated image stitching algorithm is used. In addition, the cost for camera devices can be reduced, as there is no need for several cameras or an aerial camera.