3D Audio Rendering Position Correction for View Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D audio rendering technologies can only accurately render audio when switching views if the reference view is used, failing to maintain accurate rendering when switching to different camera views.
Innovation Solution
A transmission device encodes and transmits video streams and audio streams with positional information, along with position correction information that accounts for the differences in camera positions and orientations between views, ensuring accurate 3D audio rendering across view switches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D audio rendering is performed based on positional information from a single reference view, then accurate audio rendering is achieved for that view, but accurate rendering fails when switching to other camera views
Solution Approach 1:
The patent segments the audio stream by creating separate audio streams for different views (first view audio stream and second view audio stream), each with its own positional information adapted to that specific view's coordinate system. This allows each view to have optimized 3D audio rendering while maintaining overall system compatibility.
Solution Approach 2:
The patent changes the parameters of positional information based on the view being displayed. Positional information is transformed from the reference view's coordinate system to match the coordinate system of the currently displayed view, ensuring accurate 3D audio rendering adapts to each camera's perspective.
2Measurement precision
If positional information is transformed to match each camera's coordinate system, then accurate 3D audio rendering is achieved across view switches, but system complexity increases
Solution Approach 1:
The patent performs preliminary transformation of positional information during the audio encoding stage. Positional information is pre-adjusted to match the coordinate system of each view before transmission, so that the reception device can directly use this pre-transformed information without requiring complex real-time transformation calculations.
Solution Approach 2:
The patent introduces a coordinate system transformation as an intermediary process between the reference view's positional information and the display view's requirements. This transformation layer adapts positional data from the reference coordinate system to the target view's coordinate system, simplifying the overall system architecture.
3Measurement precision
If separate audio streams are created for each view, then accurate 3D audio rendering is maintained during view switching, but data transmission volume increases
Solution Approach 1:
The patent creates simplified copies of audio streams for different views rather than transmitting completely separate audio content. The audio content itself remains largely the same, but positional information metadata is copied and transformed for each view, significantly reducing the additional data required compared to fully independent audio streams.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
It is possible to accurately perform 3D audio rendering even in a case where view switching is performed at a reception side. A first video stream having video data of a first view, a second video stream having video data of a second view, and an audio stream having audio data of an object sound source and positional information based on the first view of the obj ect sound source are generated. A container of a predetermined format including the first video stream, the second video stream, the audio stream, and position correction information for correcting the positional information of the object sound source to positional information based on the second view is transmitted.