Curved-Grid Video Processing for Vertical Audio Sound Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display devices suffer from a lack of vertical audio sound field, leading to a weak spatial sense of audio that does not match video images, resulting in reduced user immersion.
Innovation Solution
Construct a curved grid matching the display screen, acquire coordinate transformation relationships, identify motion track coordinates of sound source objects, and construct a stereo video based on spatial track coordinates to enhance audio-visual synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional two-channel audio is used in display devices, then the audio output position can be set at the bottom or both sides of the TV, but the audio sound field in the vertical direction is missing and the spatial sense of audio is weak
Solution Approach 1:
The patent transforms the traditional two-channel audio (horizontal plane) into a three-dimensional spatial audio system by introducing vertical dimension through curved grid construction. Audio elements are positioned on a curved grid that includes vertical coordinates, enabling audio to be output from multiple spatial positions including above and below the display screen, thus complementing the vertical audio sound field information that was previously missing.
2Device complexity
If traditional two-channel audio is used, then the audio system is simple, but the spatial sense of audio is weak and does not match video images
Solution Approach 1:
The patent segments the audio processing system into distinct functional modules: a curved grid construction module that creates the three-dimensional spatial framework, a motion track acquisition module that identifies sound source positions, a spatial track acquisition module that converts to curved grid coordinates, and a stereo video construction module that integrates spatial audio with video. This segmentation enables complex spatial audio processing while maintaining clear system architecture.
Solution Approach 2:
The patent introduces a curved grid as an intermediary coordinate system that bridges the gap between traditional two-channel audio and three-dimensional spatial audio. The curved grid serves as a transformation medium, converting motion track coordinates from video images into spatial track coordinates on the curved grid, thereby enabling accurate spatial audio positioning without requiring complete redesign of the audio system.
3Device complexity
If audio spatial sense does not match video images, then the audio system is simple, but user immersion is reduced
Solution Approach 1:
The patent implements feedback through the motion track acquisition process, where the system continuously identifies sound source object positions in video images and uses this information to determine audio element positions on the curved grid. This creates a closed-loop system where video image information feeds back into audio positioning, ensuring that audio spatial sense dynamically matches video images and enhances user immersion.
Data Source
Figure 1~2
Figure 3~4A
Figure 4B~5A
AI summary
A method of processing video data, a video data processing device, a display device, and a storage medium are disclosed. The method includes constructing a curved grid matching a display screen of the display device and acquiring a coordinate transformation relationship between the display screen and the curved grid; acquiring an image stream and an audio stream in the video data and identifying motion track coordinates of sound source objects in the image stream corresponding to different audio elements in the audio stream according to the image stream and the audio stream; acquiring spatial track coordinates of the sound source object corresponding to each of the audio elements on the curved grid according to the motion track coordinates of the sound source object and the coordinate transformation relationship; and constructing a stereo video based on the image stream, each of the audio elements in the audio stream, and the spatial track coordinates of each of the audio elements corresponding to the sound source object. By increasing spatial track coordinates of audio elements on the curved grid, an audio sound field information in a vertical direction is complemented to enhance immersion of users when watching stereo videos.