Curved Audio Grid Mapping for Vertical Sound-Field Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display devices with large screens lack vertical audio sound field information, leading to a weak spatial sense of audio that does not match video images, resulting in reduced user immersion.
Innovation Solution
Construct a curved grid matching the display screen, acquire coordinate transformation relationships, identify motion track coordinates of sound source objects, and construct a stereo video based on spatial track coordinates to enhance audio spatial sense.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a large display screen is used, then the visual experience is improved, but the audio spatial sense becomes weak due to missing vertical sound field information
Solution Approach 1:
The patent transforms the traditional two-dimensional audio spatial representation (left-right, front-back) into a three-dimensional curved grid system that incorporates vertical dimension. By mapping audio elements onto a spherical curved grid with latitude and longitude coordinates, the system adds the missing vertical sound field information, enabling immersive spatial audio experience that matches the large display screen's visual capabilities.
Solution Approach 2:
The patent employs a curved grid based on spherical geometry to represent the audio spatial field. Instead of using a flat two-dimensional coordinate system, the audio elements are positioned on a curved surface defined by latitude and longitude angles, which naturally captures the three-dimensional spatial distribution including vertical information, thereby resolving the contradiction between large screen display and weak audio spatial sense.
2Device complexity
If traditional two-channel audio is used, then the audio processing is simple, but the spatial sense of audio is weak and does not match video images
Solution Approach 1:
The patent extends traditional two-channel audio by adding spatial dimension information through the curved grid coordinate system. Each audio element is assigned latitude and longitude coordinates on the spherical grid, transforming plain audio signals into spatially-encoded audio objects that convey directional and vertical information, thereby enhancing spatial sense without excessive complexity increase.
Solution Approach 2:
The patent changes the parameter representation of audio from simple stereo channels to a coordinate-based system with latitude and longitude angles on a curved grid. This parameter transformation enables the audio system to express spatial position information, allowing audio to match video images more accurately while maintaining manageable processing complexity through standardized coordinate transformations.
Data Source
AI summary
A method of processing video data includes constructing a curved grid matching a display screen of the display device and acquiring a coordinate transformation relationship between the display screen and the curved grid; acquiring an image stream and an audio stream in the video data and identifying motion track coordinates of sound source objects in the image stream corresponding to different audio elements in the audio stream according to the image stream and the audio stream; acquiring spatial track coordinates of the sound source object corresponding to each of the audio elements on the curved grid according to the motion track coordinates of the sound source object and the coordinate transformation relationship; and constructing a stereo video based on the image stream, each of the audio elements in the audio stream, and the spatial track coordinates of each of the audio elements corresponding to the sound source object.