Neural Audio Codec with Hierarchical Vector Quantization

The patent divides a single large codebook into multiple smaller codebooks arranged in a codebook tree structure. Each node in the tree represents a smaller codebook, and the tree is traversed to select code vectors hierarchically. This segmentation reduces the memory footprint of individual codebooks while maintaining quantization accuracy through the combined use of multiple codebooks.

The patent introduces a hierarchical dimension to the codebook structure by organizing codebooks in a tree with multiple levels. Instead of using a single flat codebook, the system traverses through multiple levels of codebooks, adding a dimensional aspect to the codebook access pattern. This allows the system to manage large codebook sizes through hierarchical organization rather than requiring a single large memory structure.

The patent merges the audio compression and audio enhancement functions into a single integrated neural codec system. The same encoder neural network performs both compression (converting audio to feature vectors) and enhancement (applying effects), while the decoder neural network handles both decompression and effect reversal. This unified architecture eliminates the latency introduced by sequential separate processing while maintaining the reliability of both functions through joint training.

The neural codec system is designed with multi-functionality, where the encoder and decoder networks serve multiple purposes simultaneously. The encoder not only compresses audio but also prepares enhanced representations, and the decoder not only reconstructs audio but also applies enhancement effects. This universal design allows a single system to perform multiple functions that were traditionally handled by separate specialized systems.

The patent employs parameter changes by transforming audio data into a different representation space using neural networks. The encoder converts raw audio waveforms into compressed feature vectors that capture essential audio characteristics in a more efficient format. This parameter transformation allows for high-quality audio representation at lower bitrates by encoding audio in a optimized latent space rather than raw waveform space.

The patent replaces traditional mechanical audio compression methods (such as transform coding and quantization) with neural network-based compression. The neural networks learn optimal compression representations directly from data, substituting conventional signal processing mechanics with learned transformations. This substitution enables more efficient compression ratios while maintaining audio quality, as the neural networks can capture complex audio patterns that traditional methods miss.