Head Direction Prediction for Immersive Audio Bit Rate Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Achieving highest call quality with minimum bit rates in immersive multimedia communication systems, particularly in 5th generation mobile communication, where omnidirectional media transmission is challenging due to limited frequency resources and the need for synchronized audio and video adjustments based on the user's head direction.
Innovation Solution
A method and apparatus that involve a terminal device extracting head tracking information, predicting the user's head direction, and generating audio output signals by combining audio input signals from multiple microphones, while compressing and transmitting this information along with video frames to ensure synchronized and high-quality audio and video transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If omnidirectional media transmission is implemented in 5th generation mobile communication, then immersive multimedia communication quality is improved, but frequency resource consumption increases
Solution Approach 1:
The patent applies local quality by transmitting audio and video data selectively based on the user's head direction. Instead of uniformly transmitting omnidirectional media in all directions, the system identifies the actual viewing direction and transmits high-quality audio-video data only in that specific direction, while reducing or eliminating transmission in other directions. This resolves the contradiction by maintaining high call quality in the user's actual viewing direction while significantly reducing overall frequency resource consumption.
2Reliability
If audio and video transmission are synchronized based on head direction, then call quality is improved, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by obtaining head direction information in advance before audio-video synchronization is performed. The system retrieves the user's head direction from previously acquired tracking data, predicts the actual viewing direction, and then synchronizes the audio and video transmission accordingly. This approach resolves the contradiction by enabling high-quality synchronized transmission without requiring complex real-time processing, thus managing system complexity effectively.
Solution Approach 2:
The system uses feedback by continuously monitoring head direction information and using it to adjust audio-video synchronization dynamically. The head direction data obtained from tracking devices feeds back into the transmission system, allowing it to adapt the synchronization and transmission parameters based on the user's actual viewing direction, thereby maintaining high call quality.
3Measurement precision
If head tracking information is extracted and processed in real-time, then audio output accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by obtaining and processing head direction information in advance before it is needed for audio output. The system retrieves head direction data from previously acquired tracking information, performs prediction of the actual viewing direction, and prepares the audio synchronization parameters beforehand. This resolves the contradiction by enabling accurate head direction prediction without requiring time-consuming real-time processing during the actual audio output moment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a method and an apparatus for increasing call quality while reducing consumption of bit rates in immersive multimedia communication. A method of transmitting, by a terminal device, an audio signal to a counterpart terminal device, includes: extracting head tracking information of a user of the counterpart terminal device from a packet received from the counterpart terminal device; predicting a head direction of the user of the counterpart terminal device after a predetermined time period, based on the extracted head tracking information; generating audio output signals by combining audio input signals received via a plurality of microphones, based on the predicted head direction; and transmitting information about the predicted head direction and a sound frame in which the audio output signals are compressed, to the counterpart terminal device.