Spatial Audio Source Manipulation via Visual Graphical Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spatial audio recording and processing technologies face challenges in intuitively modifying the spatial audio image of composite audio signals, particularly in separating and manipulating multiple sound sources within a stereo or binaural audio signal, due to a lack of user-friendly processing tools and interfaces.
Innovation Solution
A method and apparatus that utilize angle-based representation of sound source positions within a display, allowing users to visually manipulate sound sources by changing their spatial positions and signal levels, and incorporating audio analysis to determine and maintain accurate inter-aural level and time differences for binaural audio signals, enabling intuitive modification of spatial audio images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If audio watermarking techniques are used to embed stereo mix structure in the sound signal for source separation, then sound source separation capability is improved, but device complexity and processing difficulty increase
Solution Approach 1:
The patent creates visual copies (graphical objects) of sound sources that represent their spatial positions and characteristics. These graphical objects serve as simplified interfaces that map directly to audio parameters, allowing users to manipulate sound sources through visual interaction rather than complex audio processing. The graphical objects are updated in real-time to reflect changes in sound source positions, providing an intuitive control mechanism.
Solution Approach 2:
The patent introduces graphical objects as an intermediary layer between the user and the audio signal. Instead of directly manipulating audio waveforms or complex spatial parameters, users interact with visual representations that mediate the control of sound source positions. This intermediary interface simplifies the user task while maintaining precise control over the audio output.
2Ease of operation
If graphical objects are displayed representing sound sources with manipulative elements, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The graphical objects serve multiple functions simultaneously: they display the spatial position of sound sources, provide visual feedback about audio characteristics, and act as control interfaces for manipulating sound sources. The same graphical element that represents a sound source's location also serves as the manipulative element for adjusting that sound source's position and parameters, eliminating the need for separate control interfaces.
Solution Approach 2:
The patent merges the display function and control function into a single graphical object. The visual representation of the sound source and the interface for manipulating it are combined into one element, allowing users to interact with the audio signal through the same visual feedback they receive. This merging reduces the number of separate interface elements needed while maintaining full functionality.
3Adaptability or versatility
If sound sources are separated and manipulated individually in composite audio signals, then audio processing capability is improved, but difficulty of detecting and measuring sound source parameters increases
Solution Approach 1:
The patent uses visual characteristics of graphical objects (such as position, size, or other visual attributes) to represent sound source parameters. By mapping audio parameters to visual properties that are easily perceived and differentiated, the system makes it simple for users to detect and measure sound source characteristics through visual inspection rather than complex audio analysis.
Data Source
Figure 1~3
Figure 2a~2b
Figure 4~5
AI summary
The present invention provides an arrangement, e.g. a method, an apparatus and a computer program. The arrangement comprises obtaining information indicative of spatial positions of one or more sound sources within an audio image of a composite audio signal, obtaining information indicative of the types of the one or more sound sources, determining, for each of the one or more sound sources, a position in a display of the apparatus indicative of the spatial position of the sound source and causing an apparatus to display an item representing the sound source in the determined position in the display, wherein the item representing the sound source is determined on basis of the type of the sound source. The arrangement further comprises receiving an indication of a user action associated with an item indicative of a first sound source, determining an indication of a user selected modification of the audio image on basis of the user action, and modifying the composite audio signal in accordance with the user selected modification.