Spherical Image Audio Segmentation for Viewing Comfort
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current digital cameras capable of capturing 360° omnidirectional panoramic images and videos struggle to effectively reproduce three-dimensional audio data in association with spherical images, leading to an awkward viewing experience when displaying specific areas of the image.
Innovation Solution
An information processing apparatus and system that accepts designation of a displaying direction on a spherical image, generates two-dimensional image data, and corresponding two-dimensional audio data from three-dimensional audio data, using an image processing server connected through a communication network to process and reproduce the audio data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If three-dimensional audio data is reproduced in association with spherical images, then the omnidirectional viewing experience is improved, but the viewing comfort when displaying specific areas deteriorates
Solution Approach 1:
The patent segments the spherical image into a display area and a non-display area, and correspondingly segments the three-dimensional audio data into two-dimensional audio data for the display area and preserves the original three-dimensional audio data for the non-display area. This segmentation allows the system to provide optimized audio for the visible region while maintaining omnidirectional capabilities for other areas, resolving the contradiction between omnidirectional viewing experience and viewing comfort.
Solution Approach 2:
The patent applies different audio quality characteristics to different regions: two-dimensional audio data is generated for the display area to enhance viewing comfort for the current view, while three-dimensional audio data is maintained for the non-display area to preserve omnidirectional spatial audio experience. This local differentiation of audio quality resolves the contradiction by providing appropriate audio characteristics for each specific region.
2Adaptability or versatility
If two-dimensional video is displayed from omnidirectional video, then compatibility with existing devices is improved, but the three-dimensional audio reproduction capability deteriorates
Solution Approach 1:
The patent segments the audio data processing by generating two-dimensional audio data from three-dimensional audio data specifically for the display area, while maintaining the original three-dimensional audio data for the non-display area. This allows the system to provide two-dimensional audio that is compatible with existing devices for the visible region while preserving the capability for three-dimensional audio reproduction for other regions.
Solution Approach 2:
The patent creates a multi-functional audio processing system that can simultaneously provide both two-dimensional audio data for compatibility with existing devices and three-dimensional audio data for enhanced spatial audio experience. The image processing server generates two-dimensional audio data from the three-dimensional audio data, enabling the system to serve multiple purposes: device compatibility and advanced audio reproduction capability.
3Loss of information
If spherical images are displayed as-is, then the omnidirectional information is preserved, but the viewing experience for specific areas becomes awkward
Solution Approach 1:
The patent segments the spherical image into a display area (for comfortable viewing) and a non-display area (for omnidirectional information preservation). Correspondingly, it segments the audio data into two-dimensional audio data for the display area and three-dimensional audio data for the non-display area. This segmentation allows the system to provide comfortable viewing experience for the current view while preserving omnidirectional information and audio for other areas.
Solution Approach 2:
The patent transforms the audio data from three-dimensional (spatial audio) to two-dimensional (stereophonic audio) for the display area, creating a dimensional change that optimizes the viewing experience for the current view. Meanwhile, the three-dimensional audio data is maintained for the non-display area, preserving the spatial audio experience for omnidirectional information.
Data Source
AI summary
An information processing apparatus, an information processing system, a data generation method, and a recording medium storing program code. The information processing apparatus and data generation method include displaying a spherical image obtained by an imaging apparatus, reproducing three-dimensional audio data obtained by the imaging apparatus in association with the spherical image, accepting designation of a displaying direction on the spherical image, generating two-dimensional image data that corresponds to the displaying direction, based on data of the spherical image, and generating two-dimensional audio data that corresponds to the displaying direction, from the three-dimensional audio data. The information processing system includes the information processing, and an image processing server connected to the information processing apparatus through a communication network. The image processing server generates two-dimensional audio data that corresponds to the displaying direction, based on the three-dimensional audio data.