Spherical Image Processing With GPU Cubemaps for Embedded Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sophisticated image processing techniques, such as multi-band blending, are not efficiently applicable to resource-constrained embedded devices like action cameras and mobile phones due to high memory and computational complexity, leading to unsatisfactory image quality and processing speed.
Innovation Solution
Implement hardware-accelerated image processing using GPUs by translating and rotating EAC images into rectilinear cubemap formats, leveraging existing GPU hardware for parallel processing and addressing, and performing operations like Gaussian blurring directly in 3D space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If sophisticated image processing techniques (e.g., multi-band blending) are applied, then image quality is improved, but memory and computational complexity increase
Solution Approach 1:
The patent introduces an intermediary data structure (rectilinear cubemap) that bridges the gap between spherical equirectangular images and GPU hardware capabilities. This intermediate format allows the system to leverage existing GPU hardware for parallel processing while maintaining the ability to handle sophisticated image processing techniques, thus resolving the contradiction between image quality and computational complexity
Solution Approach 2:
The patent changes the parameter representation from spherical coordinates (equirectangular projection) to a different parameter space (rectilinear cubemap with 6 faces). This transformation enables the use of hardware-accelerated parallel processing while maintaining the necessary image processing capabilities, effectively reducing the computational burden on embedded devices
2Manufacturing precision
If sophisticated image processing techniques are applied, then image quality is improved, but processing speed decreases
Solution Approach 1:
The patent transitions from 2D equirectangular projection to a 3D cubemap representation with 6 faces. This dimensional change enables parallel processing across multiple faces simultaneously using GPU hardware, significantly improving processing speed while maintaining sophisticated image processing capabilities through operations like multi-band blending
Solution Approach 2:
The patent divides the spherical image into 6 separate cubemap faces, each of which can be processed independently in parallel. This segmentation allows the system to apply sophisticated image processing techniques to each face simultaneously using hardware acceleration, thereby improving overall processing speed while maintaining image quality
3Productivity
If existing GPU hardware is used for processing, then processing capability is improved, but EAC image format compatibility is lost
Solution Approach 1:
The patent uses rectilinear cubemap as an intermediary format that translates between EAC spherical images and GPU-friendly data structures. This intermediary allows the system to maintain compatibility with existing EAC image formats while leveraging GPU hardware capabilities for accelerated processing, thus resolving the contradiction between processing capability and format compatibility
Solution Approach 2:
The patent creates a universal processing pipeline that can handle both EAC spherical images and GPU hardware requirements through the cubemap intermediate format. This universal approach enables the system to process various image formats and apply sophisticated techniques while maintaining compatibility with existing hardware and software ecosystems
Data Source
AI summary
Methods and apparatus for image processing of spherical content via hardware acceleration components. In one embodiment, an EAC image is subdivided into facets via existing software addressing and written into the memory buffers (normally used for rectilinear cubemaps) in a graphics processing unit (GPU). The EAC facets may be translated, rotated, and/or mirrored so as to align with the expected three-dimensional (3D) coordinate space. The GPU may use existing hardware accelerator logic, parallelization, and/or addressing logic to greatly improve 3D image processing effects (such as a multi-band blend using Gaussian blurs.)