360 Degree Image Stitching and 3D Object Reconstruction for Immersive Media
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies lack a method to efficiently combine inward and outward images for creating realistic media content that provides a 360° immersive experience with stereoscopic effects and allows for free viewpoint control, which is essential for personal broadcasting and virtual reality applications.
Innovation Solution
A device that acquires inward and outward images using multiple cameras, generates a 360° cylindrical background image, reconstructs 3D objects, and combines them with user-controlled views, enabling a realistic media content display system that provides a sense of immersion and allows for up, down, left, right, and zoom controls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple cameras are used to capture inward and outward images for creating realistic media content, then the sense of immersion and stereoscopic effect is improved, but the device complexity and processing difficulty increase
Solution Approach 1:
The system segments the image capture process into two distinct methods: inward-facing cameras capturing images from the center outward, and outward-facing cameras capturing images from the periphery inward. This segmentation allows each camera system to specialize in its specific function, improving the overall quality of the 360° content while maintaining manageable complexity through divided responsibilities
Solution Approach 2:
The patent introduces an intermediary processing system that receives images from both inward and outward camera arrays, performs distortion correction, stitches the images together, and generates the final equirectangular projection. This intermediary processing layer coordinates the complex interactions between multiple cameras and output formats, resolving the complexity through centralized mediation
2Area of stationary object
If 360° panoramic images are created by stitching multiple outward facing images, then the coverage area is improved, but the image distortion and stitching difficulty increase
Solution Approach 1:
The system performs preliminary distortion correction on individual camera images before the stitching process. By correcting barrel distortion and other optical distortions in advance, the subsequent stitching operation becomes more accurate and produces a higher quality equirectangular projection with minimal artifacts
Solution Approach 2:
The patent transforms images from their original cylindrical projection format to equirectangular projection format through parameter changes in the coordinate system. This mathematical transformation adjusts the mapping of spherical coordinates to a 2D plane, enabling seamless 360° coverage while maintaining geometric accuracy across the entire panoramic view
3Adaptability or versatility
If free viewpoint control is implemented for 360° images, then the user experience is improved, but the computational requirements and processing time increase
Solution Approach 1:
The system pre-processes and stores the equirectangular projection of the 360° content in a standardized format that enables rapid viewpoint changes. By having the complete spherical image already processed and available, the system can quickly extract any desired viewpoint without performing full re-stitching operations, significantly reducing processing time for user interactions
Data Source
AI summary
A device for providing a realistic media content, includes: inward facing image acquisition unit configured to acquire multiple inward facing images that are taken by an inward method using multiple cameras arranged around an object; an outward facing image acquisition unit configured to acquire multiple outward facing images that are taken by an outward method using the multiple cameras arranged around a point; a background image generation unit configured to generate a 360° cylindrical image by stitching the multiple outward facing images and then convert the 360° cylindrical image into a 360° spherical image to generate a background image; a 3D object reconstruction unit configured to extract a region of the object from the multiple inward facing images to generate a 3D object; and a combined image transmission unit configured to combine the background image with the 3D object and transmit the combined image to a display device.