Neural Audio Decoding with Latent Pitch and Pulse Shaping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio encoding and decoding technologies using artificial neural networks are too complex and resource-intensive, making real-time processing infeasible on devices with lower computational resources, such as smartphones, and result in suboptimal audio quality and increased computational load.
Innovation Solution
A hybrid approach combining parametric representation of pitch/sinusoidal/harmonic signal components with targeted trained artificial neural networks, allowing for efficient encoding and decoding of audio signals with reduced complexity and resource usage, using a receiver, pulse sequence generator, shaping circuit, and output circuit to generate an output signal from a latent space feature set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If large artificial neural networks are used to achieve desired audio encoding and decoding quality, then audio quality is improved, but device complexity and computational resource requirements increase
Solution Approach 1:
The patent segments the audio signal into multiple components (e.g., harmonic components, residual components) and processes each component separately through specialized neural network modules. This segmentation allows each module to be smaller and less complex while collectively achieving high audio quality through targeted processing of specific signal characteristics.
Solution Approach 2:
The patent applies different processing strategies to different parts of the audio signal. Specific neural network modules are designed to handle particular audio characteristics (e.g., pitch prediction for harmonic components, spectral shaping for residual components), allowing each module to be optimized for its specific function rather than requiring a single large universal network.
2Manufacturing precision
If large artificial neural networks are used to achieve desired audio encoding and decoding quality, then audio quality is improved, but computational load increases
Solution Approach 1:
By dividing the audio processing into separate modules that handle different signal components, the computational workload is distributed across multiple smaller networks rather than requiring one large network. Each module processes only its specific portion of the signal, reducing overall computational load while maintaining audio quality.
Solution Approach 2:
The patent extracts and separately processes key audio characteristics (such as pitch, harmony, and spectral features) using dedicated neural network modules. By extracting these specific features and processing them independently, the system avoids the computational overhead of feeding entire audio signals through large general-purpose networks.
3Manufacturing precision
If existing audio encoding approaches are used, then audio quality can be maintained, but real-time processing on portable devices becomes infeasible
Solution Approach 1:
The patent divides audio processing into modular components that can be executed efficiently on portable devices. Each module handles a specific aspect of audio processing (e.g., pitch detection, harmonic synthesis, residual encoding), enabling real-time processing through parallel or sequential execution of lightweight modules rather than requiring a single heavy processing pipeline.
Solution Approach 2:
The patent transforms audio signals into parameter-based representations (such as pitch contours, harmonic indices, spectral coefficients) that require less computational resources to process and transmit. By working with compact parameter sets rather than full-waveform data, the system achieves real-time processing capability on portable devices while maintaining audio quality through accurate parameter-based reconstruction.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An audio apparatus receives an encoded audio data signal comprises a latent space feature set representing properties of a signal component of an audio signal. A trained artificial neural network (205) generates a frequency, and typically a pitch, parameter reflecting a pitch property of the signal component of the audio signal from the latent space feature set. A pulse sequence generator (207) generates a pulse sequence having properties dependent on the pitch parameter. A shaping circuit (209) generates a first audio signal from the pulse sequence by applying a spectro-temporal shaping to the pulse sequence with the spectro-temporal shaping being dependent on the latent space feature set. An output circuit (211) arranged to generate an output signal from the first audio signal. An encoding apparatus may generate the latent space feature set using a trained artificial neural network receiving parameters describing sinusoids/harmonics/harmonic grids as inputs.