Audio data compression method and device, electronic equipment and storage medium

Abstract

The invention discloses an audio data compression method and device, electronic equipment and a storage medium, relates to the technical field of voice processing, can be applied to financial science and technology and medical health business scenarios, and comprises the following steps: obtaining target audio data to be compressed; and inputting the target audio data into an audio coding and decoding quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, the audio coding and decoding quantization compression model being obtained through audio adversarial training. In the model, feature extraction and compression can be performed on target audio data by using a hierarchical neural network of an encoder to obtain a low-dimensional potential feature vector; a residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascaded codebook to obtain a discrete quantization code; and performing audio waveform reduction processing on the discrete quantization code by using a decoder to obtain a reconstructed audio signal corresponding to the target audio data. According to the invention, audio high-fidelity compression can be realized, and audio reconstruction tone quality is improved.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to an audio data compression method, device, electronic device and storage medium. Background Art

[0002] In the financial technology sector, scenarios such as remote identity authentication and customer service center voice recording place stringent demands on the security and real-time nature of audio data. Low-bitrate codec technology can reduce cloud storage costs and encrypted transmission bandwidth consumption while ensuring the integrity of voice features. In healthcare, the storage and transmission of remote diagnosis and treatment voice recordings and pathology audio archives rely on high-fidelity audio compression technology to preserve the spectral details of physiological audio signals such as heart and lung sounds, providing high-quality data support for AI-assisted diagnosis. With the widespread adoption of 5G and IoT technologies, achieving low-bitrate, high-fidelity, and low-latency audio codecs with limited resources has become a key research direction for multimedia technology in scenarios such as financial risk control and smart healthcare.

[0003] Although traditional audio coding and decoding technology can achieve a certain compression rate, it is prone to spectral distortion, timing discontinuity and other problems due to quantization noise in low bit rate scenarios. Especially in signals rich in high-frequency details and long-term dependencies such as speech and music, the reconstructed audio often has auditory defects such as fuzzy timbre and incoherent rhythm, which makes it difficult to meet the demanding sound quality requirements of scenarios such as virtual reality and high-definition voice communication. Summary of the Invention

[0004] In view of this, the present application provides an audio data compression method, device, electronic device and storage medium, which can achieve high-fidelity audio compression and improve the sound quality of audio reconstruction.

[0005] According to a first aspect of the present application, there is provided an audio data compression method, comprising:

[0006] Obtaining target audio data to be compressed;

[0007] Inputting the target audio data into a trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training, and the audio codec quantization compression model includes an encoder, a residual vector quantization module, and a decoder;

[0008] Inputting the target audio data into the trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data includes:

[0009] Using the hierarchical neural network of the encoder to extract and compress the target audio data to obtain a low-dimensional latent feature vector;

[0010] The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code;

[0011] The decoder is used to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data.

[0012] According to a second aspect of the present application, there is provided an audio data compression device, comprising:

[0013] An acquisition module, used for acquiring target audio data to be compressed;

[0014] a processing module, configured to input the target audio data into a trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training and comprises an encoder, a residual vector quantization module, and a decoder;

[0015] The processing module is specifically used to:

[0016] Using the hierarchical neural network of the encoder to extract and compress the target audio data to obtain a low-dimensional latent feature vector;

[0017] The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code;

[0018] The decoder is used to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data.

[0019] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned audio data compression method is implemented.

[0020] According to a fourth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned audio data compression method when executing the program.

[0021] By means of the above technical solution, the audio data compression method, device, electronic device and storage medium provided by this application can perform multi-level feature extraction and feature dimension compression on the target audio through the hierarchical neural network of the encoder to obtain a low-dimensional potential feature vector. Then, the multi-layer cascade codebook of the residual vector quantization module is used to discretely quantize it, and the feature details are retained through iterative processing. Finally, the decoder restores the waveform of the quantized code, and combines audio adversarial training to optimize the reconstruction effect. Thus, while achieving audio compression, it effectively retains the audio feature information, improves the quality of the reconstructed audio, and solves the problem of poor sound quality after compression in traditional audio coding and decoding technology.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram showing a flow chart of an audio data compression method provided in an embodiment of the present application is shown;

[0024] Figure 2 An audio codec quantization compression model architecture diagram provided in an embodiment of the present application is shown;

[0025] Figure 3 A schematic flow chart of an audio data compression method provided in another embodiment of the present application is shown;

[0026] Figure 4 A schematic structural diagram of an audio data compression device provided in an embodiment of the present application is shown;

[0027] Figure 5 A structural diagram of another audio data compression device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0029] Although traditional audio coding and decoding technology can achieve a certain compression rate, it is prone to spectral distortion, timing discontinuity and other problems due to quantization noise in low bit rate scenarios. Especially in signals rich in high-frequency details and long-term dependencies such as speech and music, the reconstructed audio often has auditory defects such as fuzzy timbre and incoherent rhythm, which makes it difficult to meet the demanding sound quality requirements of scenarios such as virtual reality and high-definition voice communication.

[0030] In order to solve the above technical problems, the embodiment of the present invention provides an audio data compression method, such as Figure 1 As shown, the method includes:

[0031] Step 110: Obtain target audio data to be compressed.

[0032] For the embodiments of the present disclosure, the original audio signal can be obtained from an audio acquisition device, storage medium or communication link. The original audio signal can be any type of time domain waveform data such as speech, music, etc., that is, the target audio data, which serves as the input object of the subsequent encoding and decoding compression process, providing a processing basis for achieving efficient compression and high-quality reconstruction of audio data.

[0033] Step 120: Input the target audio data into the trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training.

[0034] The audio codec quantization compression model is a model that has been pre-trained for audio codec quantization compression tasks, such as Figure 2 As shown in Figure 1, the audio codec quantization compression model can include an encoder, a residual vector quantization module, and a decoder. Furthermore, a discriminator can be introduced into the architecture of the audio codec quantization compression model to enable audio adversarial training of the model. This forces the encoder and decoder to optimize their feature extraction and restoration capabilities. This can improve the spectral detail and temporal dynamic coherence of the reconstructed audio from the perspective of human perception, enhance compression robustness at low bit rates, and achieve joint optimization of the entire process, effectively resolving the audio quality distortion problem caused by traditional compression techniques.

[0035] For the embodiments of the present disclosure, the target audio data can be directly input into the codec quantization compression model optimized by audio adversarial training. The model consists of an encoder, a residual vector quantization module and a decoder: the encoder extracts and compresses audio features to generate low-dimensional latent feature vectors, the residual vector quantization module discretely quantizes them through a multi-layer codebook to reduce the amount of data, and the decoder restores the quantized code to an audio waveform, and finally outputs a reconstructed signal that is highly close to the original audio, realizing end-to-end processing of compression and reconstruction.

[0036] Accordingly, the discriminator may include a feature extraction layer and a classification layer. The feature extraction layer includes a two-dimensional convolution layer with a convolution kernel of 5×5, four two-dimensional convolution layers with a convolution kernel of 3×9 and a stride of (1, 2), and two 3×3 two-dimensional convolution layers. The classification layer is a one-dimensional convolution layer. In audio adversarial training, the sample audio data can be first generated into a sample reconstructed audio signal through the original audio codec quantization compression model. Then, the sample audio data and the sample reconstructed audio signal are converted into a spectrogram using a short-time Fourier transform. The purpose is to convert the time domain signal into a frequency domain feature representation, so that the discriminator can analyze the differences from the level of acoustic characteristics perceived by the human ear (such as harmonic structure, high-frequency details, and temporal dynamics). At the same time, the convolutional neural network is adapted to extract multi-level frequency domain patterns, accurately identifying subtle distortions between the reconstructed audio and the original audio, and driving the optimization of the codec model. The two spectrograms are further input into the feature extraction layer of the discriminator to extract multi-layer features, and the authenticity probability distribution is discriminated by the classification layer. Based on this, the discriminator loss and the generator loss including reconstruction loss, discrimination loss, and quantization loss are calculated. The discriminator, encoder, and decoder parameters are updated through gradient backpropagation until the iteration conditions are met, thereby optimizing the audio codec quantization compression model.

[0037] Correspondingly, when using the discriminator to perform audio adversarial training on the audio codec quantization compression model, the implementation steps may specifically include: using the audio codec quantization compression model to determine the sample reconstructed audio signal corresponding to the sample audio data; using short-time Fourier transform to convert the sample audio data and the sample reconstructed audio signal into spectrograms respectively, and then input them into the feature extraction layer of the discriminator to extract multi-level feature representations; using the classification layer of the discriminator, binary classification is performed on the multi-level feature representation to generate the authenticity probability distribution of the sample reconstructed audio signal and the original audio signal corresponding to the sample audio data; based on the authenticity probability distribution, the discriminator loss and the generator loss are calculated, and the network parameters of the discriminator, encoder and decoder are respectively updated through gradient back propagation until the preset adversarial training iteration conditions are reached to obtain the trained audio codec quantization compression model.

[0038] Among them, the generator loss is the weighted calculation result of reconstruction loss, discrimination loss and quantization loss.

[0039] The reconstruction loss is defined as follows:

[0040]

[0041] Where, L recon is the reconstruction loss, x is the original audio signal, The audio signal is reconstructed from the samples, and S(·) represents the short-time Fourier transform (STFT).

[0042] The discriminative loss is defined as follows:

[0043]

[0044] Where, L g To determine the loss, is the sample-reconstructed audio signal, and D(·) is the authenticity probability of the sample-reconstructed audio signal output by the discriminator (the larger the value, the closer it is to the original audio signal).

[0045] The quantization loss is defined as follows:

[0046]

[0047] Where, L rvq To quantify the loss, r i-1 is the input residual of the i-th layer codebook (initial residual r0=z, z is the low-dimensional potential feature vector output by the encoder), and M is the preset number of codebook layers.

[0048] By weighted addition of the above reconstruction loss, discrimination loss, and quantization loss, the final generator loss can be obtained:

[0049] L all =λ recon L recon +λ g L g +λ rvq L rvq

[0050] Where, L all is the generator loss, L recon To reconstruct the loss, L g is the discrimination loss, L rvq is the quantization loss, λ recon ,λ g ,λ rvq The weights corresponding to reconstruction loss, discrimination loss, and quantization loss respectively. The specific weight values can be set according to the actual application scenario and are not specifically limited here.

[0051] The discriminator loss is defined as follows:

[0052]

[0053] Where, L d is the discriminator loss, x is the original audio signal, is the sample-reconstructed audio signal, and D(·) is the authenticity probability of the sample-reconstructed audio signal output by the discriminator (the larger the value, the closer it is to the original audio signal).

[0054] In summary, the audio data compression method provided by the present invention can perform multi-level feature extraction and feature dimension compression on the target audio through the encoder's hierarchical neural network, obtaining a low-dimensional latent feature vector. This is then discretely quantized using the multi-layer cascaded codebook of the residual vector quantization module, retaining feature details through iterative processing. Finally, the decoder performs waveform restoration on the quantized code, and combines audio adversarial training to optimize the reconstruction effect. This effectively preserves audio feature information while achieving audio compression, improving the quality of the reconstructed audio and resolving the problem of poor sound quality after compression with traditional audio codec technologies.

[0055] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the implementation of this embodiment, this embodiment also provides another audio data compression method, such as Figure 3 As shown, the method includes:

[0056] Step 310: Obtain target audio data to be compressed.

[0057] For the embodiment of the present disclosure, the specific implementation process can be found in the relevant description of step 110 of the embodiment, which will not be repeated here.

[0058] Step 320: Use the encoder's hierarchical neural network to extract and compress the target audio data to obtain a low-dimensional latent feature vector.

[0059] The hierarchical neural network includes sequentially connected 1D convolution layers, first convolution block groups, and long short-term memory network layers. The 1D convolution layer can use a 1D convolution layer with a convolution kernel size of 7 to extract local time domain features from the original audio waveform, converting the 1D time domain signal into a feature vector (preliminary feature vector) with multiple channels, capturing the basic frequency components of the audio (such as fundamental frequency, low-order overtones) and short-term energy changes. The convolution kernel size is selected as 7 (an odd number) to ensure input and output timing alignment, avoid phase shift, and provide a stable feature basis for subsequent processing; the first convolution block group is composed of multiple cascaded first convolution blocks, each of which contains: 2 1D convolution layers with a convolution kernel size of 3, followed by a LeakReLU activation layer and a weight normalization layer; a residual connection (located between the weight normalization layer and the downsampling layer); and a downsampling layer (such as a strided convolution or pooling layer). A dual convolutional layer structure (3x3 convolution) can extract mid-level features (such as instrument timbre characteristics and speech formants). The activation function enhances nonlinear expression, and weight normalization accelerates training convergence. The downsampling layer gradually reduces the time dimension of the feature vector (such as compressing the time series length by stride 2), reducing the amount of computation while capturing long-term contextual information. The residual connection fuses the original input features of each first convolutional block with the output of the current layer, preventing the vanishing gradient of the deep network and ensuring the complete transmission of low-frequency basic features and high-frequency details. The long-short-term memory network layer can use a two-layer LSTM network. After receiving the low-dimensional feature vector output by the first convolutional block group, it can capture the long-term and short-term temporal relationships in the audio signal (such as the coherence of musical rhythm and the contextual dependence of speech intonation) through a gating mechanism (forget gate, input gate, output gate), solving the problem that convolutional networks have difficulty modeling long-distance dependencies.

[0060] Accordingly, for the embodiment of the present disclosure, step 320 may include the following steps:

[0061] Step 320 - 1 : Input the target audio data into the 1-dimensional convolution layer of the encoder for initial feature extraction to obtain a preliminary feature vector.

[0062] In the disclosed embodiment, the target audio data can be input into the encoder's 1D convolution layer, which uses a 1D convolution with a kernel size of 7 to extract local time-domain features from the original audio waveform. Through the convolution operation, the 1D audio signal is converted into a multi-channel feature vector (i.e., a preliminary feature vector), capturing the fundamental frequency components (such as the fundamental frequency and low-order overtones) and short-term energy variations in the audio.

[0063] Step 320-2: Input the preliminary feature vector into the first convolution block group for multi-level feature extraction and feature dimension compression to obtain the target down-sampled feature vector.

[0064] For the embodiment of the present disclosure, the preliminary feature vector can be input into the first convolution block group, which is composed of multiple cascaded first convolution blocks. Each first convolution block contains two 1-dimensional convolution layers with a convolution kernel of 3, a LeakReLU activation layer, a weight normalization layer and a downsampling layer, and the input features are fused through residual connections between the weight normalization layer and the downsampling layer. Each first convolution block further extracts mid-level features (such as instrument timbre and speech resonance peaks) through double convolution layers, and the activation and normalization operations enhance the feature expression capability; the downsampling layer compresses the time dimension of the feature through strided convolution, reducing the amount of calculation while capturing long-term context information; the residual connection avoids the disappearance of the deep network gradient and retains the key information in the original feature. Through the layer-by-layer processing of multi-level convolution blocks, the target downsampled feature vector with significantly reduced output dimension and containing high-level semantics can lay the foundation for subsequent temporal feature modeling.

[0065] Correspondingly, for the embodiment of the present disclosure, the embodiment steps may include: for any one first convolution block among multiple first convolution blocks, using the 1-dimensional convolution layer, LeakReLU activation layer and weight normalization layer configured therein, the first input feature vector is successively convolved, activated and normalized to obtain the first feature vector of the current first convolution block, and using the downsampling layer of the current first convolution block to perform feature dimension compression processing on the fusion vector of the first feature vector and the first input feature vector to obtain a downsampled feature vector; the downsampled feature vector output by the last first convolution block among the multiple first convolution blocks is determined as the target downsampled feature vector extracted by the first convolution block group.

[0066] Among them, when the current first convolution block is the first first convolution block among multiple first convolution blocks, the first input feature vector is the preliminary feature vector; when the current first convolution block is any first convolution block other than the first first convolution block among multiple first convolution blocks, the first input feature vector is the downsampled feature vector output by the current first convolution block corresponding to the previous first convolution block.

[0067] For example, suppose the first convolutional block group consists of two cascaded first convolutional blocks (block 1 and block 2): The first first convolutional block (block 1): The input is the preliminary feature vector Feat0, which is first processed by two 1-dimensional convolutional layers with a convolution kernel of 3, a LeakReLU activation layer, and a weight normalization layer in block 1 to obtain the first feature vector Feat1_conv; Feat1_conv is then fused with the residual of Feat0, and the time dimension is compressed to T1 = T0 / 2 through a downsampling layer (such as a stride 2 convolution), and the downsampled feature vector Feat1 is output. The input of the subsequent first convolutional block (block 2) is Feat1 output by block 1. The convolution-activation-normalization process of block 1 is then repeated to obtain Feat2_conv, which is fused with the residual of Feat1 and compressed to T2 = T1 / 2 = T0 / 4 through a downsampling layer, and the downsampled feature vector Feat2 is output. Finally, Feat2 output by block 2 is the target down-sampled feature vector, which realizes progressive compression and feature abstraction from Feat0 to low-dimensional features through multi-layer cascade.

[0068] Step 320-3: Use the long short-term memory network layer to capture the temporal features in the target downsampled feature vector to obtain a low-dimensional latent feature vector.

[0069] For the embodiment of the present disclosure, when using the long short-term memory network layer (LSTM layer) to capture the temporal features in the target down-sampled feature vector, the target down-sampled feature vector output by the first convolution block group (the spatiotemporal dimensions have been compressed by multi-level convolution) can be input into a two-layer LSTM network. Through the synergistic effect of the forget gate, input gate and output gate, LSTM selectively retains or forgets the feature information at different times, effectively modeling the long-term and short-term temporal dependencies in the audio signal (such as the contextual semantic coherence of speech and the temporal dynamics of music rhythm). After the temporal feature modeling of the LSTM layer, a low-dimensional potential feature vector with further compressed output dimension is output. This vector combines frequency domain features and temporal dynamic information, and can provide a compact and semantically rich input representation for the subsequent residual vector quantization module, ultimately achieving efficient compression and perceptual fidelity of audio data.

[0070] Step 330: The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code.

[0071] In specific application scenarios, when the residual vector quantization module discretely quantizes the low-dimensional potential feature vector, a multi-layer cascade codebook structure can be used: first, the input vector z is matched with the nearest codeword q1 in the first layer codebook C1, and the residual r1 = z-q1 is calculated; then the residual r1 is input into the next layer codebook C2 to match the codeword q2, and the residual r2 = r1-q2 is updated; and so on, until the preset number of codebook layers M is reached. Each layer of codebook gradually decomposes the residual information of the input vector, and finally outputs the sum of all layer codewords z q=q1+q2+...+q M , as discrete quantization coding. This process uses multi-layer residual iterative quantization to capture feature details layer by layer, reducing the amount of data while retaining high-frequency information, which can improve the reconstruction fidelity of audio compression.

[0072] Accordingly, for the embodiment of the present disclosure, step 330 may include the following steps:

[0073] Step 330-1: Determine a multi-layer cascade codebook corresponding to a preset compression rate.

[0074] For the embodiments of the present disclosure, an adapted multi-layer cascaded codebook structure can be selected or generated according to the target compression rate requirement. The number of codebook layers (such as M layers) and the number of codewords in each layer directly affect the compression rate and quantization accuracy: the more layers and the fewer codewords in each layer, the higher the compression rate but the quantization error may increase; conversely, the compression rate decreases but the fidelity improves. For example, if the preset compression rate is 16:1, a 3-layer codebook can be used, with the number of codewords in each layer being 256, 128, and 64, respectively. Progressive feature compression is achieved through layer-by-layer residual quantization, achieving the target compression ratio while ensuring a certain sound quality.

[0075] Step 330 - 2 : Determine the matching codeword and residual of the low-dimensional latent feature vector in the first-level codebook of the multi-layer cascade codebook.

[0076] In the disclosed embodiment, the low-dimensional latent feature vector z output by the encoder is input into the first-level codebook (e.g., C1) of a multi-layer cascaded codebook. A vector distance metric (e.g., Euclidean distance) is used to find the codeword q1 closest to z (i.e., the matching codeword), and the residual r1 = z - q1 is calculated. This step is the initial decomposition of residual vector quantization (RVQ). The first-level codebook is responsible for capturing the main energy distribution of the feature vector, while the residual r1 retains detailed information not represented by the first-level codeword, providing input for the refined quantization of subsequent codebook levels.

[0077] Step 330-3: For any codebook layer except the first codebook layer in the multi-layer cascaded codebook, iteratively update the residual output by the previous codebook layer and determine the matching codeword until the preset number of codebook layers is reached, thereby obtaining an updated target residual and a multi-level matching codeword sequence. The multi-level matching codeword sequence includes the matching codeword determined when the residual of each codebook layer is updated.

[0078] For any codebook layer other than the first in a multi-layer cascaded codebook, the residual output from the previous layer is used as input. A vector distance metric (such as Euclidean distance) is used to find the closest matching codeword in the current codebook layer. A new residual is calculated (current layer residual = previous layer residual - matching codeword), and this residual is passed to the next codebook layer for further iteration. This process continues layer by layer until the preset number of codebook layers is reached, ultimately resulting in a target residual (typically approaching zero) and a sequence of matching codewords for each layer (such as q1, q2, ..., qM). This process progressively decomposes the residuals through multiple codebook layers, allowing each codebook layer to focus on capturing detailed information not represented in the previous layer. For example, the second codebook layer processes high-frequency components in the first-layer residual, while the third layer processes subtle fluctuations in the second-layer residual. This preserves the multi-level details of the feature vector during discrete quantization, ensuring that the compressed code reduces data volume while reconstructing high-fidelity audio features through the codeword sequence.

[0079] Step 330 - 4 : Generate discrete quantization codes based on the matching codewords of each layer in the multi-level matching codeword sequence.

[0080] In the disclosed embodiment, when generating discrete quantization codes based on matching codewords at each layer in a multi-level matching codeword sequence, the index values corresponding to the codewords obtained by matching each layer of the codebook (e.g., matching codewords q1 at the first layer, q2 at the second layer, and so on, up to qM at the Mth layer) can be concatenated in hierarchical order to form a continuous sequence containing the indexes of the codewords at each layer, i.e., a discrete quantization code zq. This code uniquely maps the codewords at each layer through the index values, and represents the multi-layer decomposition results of the original low-dimensional latent feature vector in a compact numerical form. For example, a three-layer codebook structure can achieve a compressed representation of the feature vector by concatenating three index values (each index value corresponds to the codeword number in a layer of the codebook), significantly reducing the amount of data while retaining the multi-level detailed information of the feature vector, facilitating storage or transmission. A subsequent decoder can retrieve the corresponding codewords from each layer of the codebook through the index sequence in zq and sum them to reconstruct the low-dimensional latent feature vector to restore the audio signal.

[0081] Step 340: Use a decoder to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data.

[0082] The decoder includes a second convolutional block group and a 1D convolutional layer connected in sequence. The second convolutional block group is symmetrical with the first convolutional block group of the encoder and consists of multiple cascaded second convolutional blocks. Each second convolutional block includes: two 1D convolutional layers with a convolution kernel of 3, followed by a LeakReLU activation layer and a weight normalization layer; a residual connection (located between the weight normalization layer and the upsampling layer); and an upsampling layer (such as a transposed convolution). The input of the second convolutional block group is a quantized representation of a low-dimensional latent feature vector. It first passes through the first second convolutional block of the second convolutional block group, and is processed by two 1D convolutional layers, a LeakReLU activation layer, and a weight normalization layer to extract the basic semantics of the quantized features (such as the waveform contour corresponding to the fundamental frequency). The input features are fused with the convolution output through a residual connection to avoid information loss during the upsampling process. A transposed convolutional upsampling layer (e.g., with a stride of 2) is used to expand the temporal dimension of the features (e.g., restore from T3 to T2 = T3 × 2), while increasing the number of channels or restoring the feature dimension. The second convolutional block then repeats the above operations, gradually restoring the temporal resolution of the features (e.g., from T2 to T1 and then to T0) through multi-level upsampling, and gradually restoring spectral details (e.g., harmonic structure and formant positions) layer by layer through residual connections and convolution operations. The final output is a high-dimensional feature vector whose time dimension matches the original audio input of the encoder, containing the complete frequency domain and timing information required to reconstruct the audio.

[0083] Accordingly, for the embodiment of the present disclosure, step 340 may include the following steps:

[0084] Step 340 - 1 : Input the discrete quantization code into the second convolution block group, gradually restore the feature dimension and fuse the context information to obtain a context fused feature vector.

[0085] For the embodiment of the present disclosure, the discrete quantization code is input into the second convolution block group of the decoder. This module is symmetrical with the first convolution block group of the encoder and is composed of multiple cascaded second convolution blocks. Each second convolution block extracts features through two 1-dimensional convolution layers with a convolution kernel of 3, and enhances the expression ability through LeakReLU activation and weight normalization. The current layer output and input features are fused through residual connection to retain details. Finally, the transposed convolution upsampling layer is used to expand the time dimension of the features (such as upsampling with a step size of 2 to double the time resolution). Through the layer-by-layer processing of multi-level convolution blocks, the discrete quantization code is gradually restored from low-dimensional features to high-dimensional feature dimensions close to the original audio. At the same time, the second convolution block of each layer captures contextual dependencies (such as reconstructing the temporal coherence of the audio) by expanding the receptive field, and finally outputs a context-fused feature vector containing rich frequency domain details and temporal semantics, laying the foundation for the subsequent generation of time domain waveforms.

[0086] Correspondingly, for the embodiment of the present disclosure, the embodiment steps may include: for any second convolution block among multiple second convolution blocks, using at least two 1-dimensional convolution layers, LeakReLU activation layers and weight normalization layers configured therein, the second input feature vector is successively convolved, activated and normalized to obtain the second feature vector of the current second convolution block, and using the transposed convolution upsampling layer of the current second convolution block to perform feature dimension expansion processing on the fusion vector of the second feature vector and the second input feature vector through a transposed convolution operation to obtain an upsampled feature vector; the upsampled feature vector output by the last second convolution block among the multiple second convolution blocks is determined as the context fusion feature vector extracted by the second convolution block group.

[0087] Among them, when the current second convolution block is the first second convolution block among multiple second convolution blocks, the second input feature vector is discrete quantization coding; when the current second convolution block is any second convolution block other than the first second convolution block among multiple second convolution blocks, the second input feature vector is the upsampled feature vector output by the current second convolution block corresponding to the previous second convolution block.

[0088] For example, suppose the second convolutional block group consists of two cascaded second convolutional blocks (Block A and Block B): The first second convolutional block (Block A) takes as input the discrete quantized code zq. It is first processed within the block by two 1D convolutional layers with kernel size 3, a LeakReLU activation layer, and a weight normalization layer to produce the second feature vector FeatA_conv. FeatA_conv is then fused with the residual of zq and upsampled to T2 = T3 × 2 via a transposed convolutional upsampling layer (e.g., with stride 2), producing the upsampled feature vector FeatA. The subsequent second convolutional block (Block B) takes as input FeatA and repeats the convolution-activation-normalization process of Block A to produce FeatB_conv. This is then fused with the residual of FeatA and upsampled to T1 = T2 × 2 = T3 × 4 via a transposed convolution, producing the output FeatB. Finally, FeatB, the output of Block B, is the context-fused feature vector. This multi-layer cascade achieves progressive recovery from low-dimensional code to high-dimensional features and the fusion of contextual information.

[0089] Step 340 - 2 : Input the context fusion feature vector into the 1-dimensional convolution layer of the decoder, map it to the time domain space through linear transformation, and generate a reconstructed audio signal corresponding to the target audio data.

[0090] For the disclosed embodiment, the context-fused feature vector (restored to a time dimension and frequency domain feature close to the original audio) output by the second convolution block group of the decoder can be input into a 1-dimensional convolution layer, which uses a linear transformation operation with a convolution kernel size of 7 to map the high-dimensional feature vector from the frequency domain to the time domain space. Through the dot product operation of the convolution kernel and the feature vector, the temporal dynamic information in the feature (such as waveform amplitude changes) is extracted, and the output range is limited by the activation function, and finally a reconstructed audio signal that matches the time domain morphology of the original audio is generated, achieving end-to-end restoration from compressed coding to audible waveforms, ensuring that the reconstructed audio still maintains time domain continuity and auditory fidelity at high compression rates.

[0091] It should be noted that the technical solutions in this application can be applied to financial technology and medical health business scenarios.

[0092] As a possible implementation method, in the FinTech scenario, this audio quantization and compression technology can be applied to voice identity authentication and transaction voice evidence in remote financial services. By using a lightweight encoder to compress customer voice at a low bit rate, the spectral details of the voiceprint characteristics (such as fundamental frequency and resonance peak distribution) are retained at a bit rate of 1.5-6kbps. This not only meets the storage needs of massive voice data in the cloud (compression ratio increased to over 16:1), but also captures subtle voiceprint features through the residual vector quantization module to ensure the authentication accuracy of the voiceprint recognition system. At the same time, the dynamic adjustment of the compression rate adapts to the bandwidth fluctuations of the financial private network. Combined with the high-fidelity reconstruction capability of adversarial training, the transaction voice records can still clearly retain semantic information and tone characteristics after restoration, meeting regulatory compliance requirements.

[0093] As another possible implementation method, in the field of medical health, this technology can be applied to remote diagnosis and treatment voice data management and pathological audio archive storage. For physiological audio signals such as heart sounds and lung sounds, the residual quantization mechanism of the multi-layer cascade codebook can effectively retain weak high-frequency components such as noise and abnormal heartbeats (quantization loss is controlled within 5%), avoiding the loss of pathological features caused by traditional compression methods; the symmetric convolution architecture of the decoder can reconstruct the waveform with high precision, allowing remote doctors to obtain time domain features consistent with the original signal through the stethoscope system. In addition, the modular design supports streaming transmission, and in emergency scenarios, the patient's vital signs audio can be compressed and transmitted in real time, and transmitted to the emergency center with low latency through the 5G network, providing high-quality audio data support for AI-assisted diagnosis, while reducing medical cloud storage costs.

[0094] In summary, the technical solution of this application can achieve efficient compression and high-fidelity reconstruction of audio data through the end-to-end architecture of lightweight encoders and decoders, the multi-layer cascade codebook design of residual vector quantization, and the weighted optimization of multiple loss functions (reconstruction loss, discrimination loss, and quantization loss) in adversarial training, significantly improving compression efficiency at extremely low bit rates. The frequency domain details and temporal dynamics of the audio are captured through hierarchical feature extraction of convolutional block groups and LSTM layers, and the integrity of feature transfer is ensured by residual connections and transposed convolutions. The authenticity judgment of the spectrum graph by the discriminator is combined to enhance the perceived quality. Ultimately, the model supports dynamic compression rate adjustment in real-time streaming processing scenarios, combining computational efficiency and sound quality fidelity, solving the problems of low bit rate and poor sound quality of traditional codecs, complex training of neural network models, and insufficient real-time performance.

[0095] Further, as Figure 1 and Figure 3 The specific implementation of the method shown in this embodiment provides an audio data compression device, such as Figure 4 As shown, the device includes: an acquisition module 41 and a processing module 42.

[0096] An acquisition module 41 is configured to acquire target audio data to be compressed;

[0097] A processing module 42 may be configured to input the target audio data into a trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training and includes an encoder, a residual vector quantization module, and a decoder;

[0098] The processing module 42 may be specifically configured to:

[0099] The encoder's hierarchical neural network is used to extract and compress the target audio data to obtain a low-dimensional latent feature vector.

[0100] The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain discrete quantization code;

[0101] The decoder is used to restore the audio waveform of the discrete quantization code to obtain the reconstructed audio signal corresponding to the target audio data.

[0102] In some embodiments of the present application, the hierarchical neural network includes a 1-dimensional convolutional layer, a first convolutional block group, and a long short-term memory network layer connected in sequence; when the hierarchical neural network of the encoder is used to extract and compress the target audio data to obtain a low-dimensional latent feature vector, the processing module 42 can be specifically used to: input the target audio data into the 1-dimensional convolutional layer of the encoder for initial feature extraction to obtain a preliminary feature vector; input the preliminary feature vector into the first convolutional block group for multi-level feature extraction and feature dimension compression processing to obtain a target down-sampled feature vector; use the long short-term memory network layer to capture the temporal features in the target down-sampled feature vector to obtain a low-dimensional latent feature vector.

[0103] In some embodiments of the present application, the first convolution block group includes multiple first convolution blocks, each first convolution block includes a 1-dimensional convolution layer, a LeakReLU activation layer, a weight normalization layer and a downsampling layer are cascaded after the 1-dimensional convolution layer, and each first convolution block is connected between the weight normalization layer and the downsampling layer through a residual connection; when the preliminary feature vector is input into the first convolution block group for multi-level feature extraction and feature dimension compression processing to obtain the target downsampling feature vector, the processing module 42 can be specifically used to: for any one of the multiple first convolution blocks, use the 1-dimensional convolution layer, LeakReLU activation layer and weight normalization layer configured therein to successively perform convolution, activation and normalization processing on the first input feature vector to obtain the target downsampling feature vector. The first feature vector of the previous first convolution block, and the downsampling layer of the current first convolution block, perform feature dimension compression processing on the fusion vector of the first feature vector and the first input feature vector to obtain a downsampled feature vector; the downsampled feature vector output by the last first convolution block in the multiple first convolution blocks is determined as the target downsampled feature vector extracted by the first convolution block group; wherein, when the current first convolution block is the first first convolution block in the multiple first convolution blocks, the first input feature vector is the preliminary feature vector; when the current first convolution block is any first convolution block other than the first first convolution block in the multiple first convolution blocks, the first input feature vector is the downsampled feature vector output by the current first convolution block corresponding to the previous first convolution block.

[0104] In some embodiments of the present application, when the residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code, the processing module 42 can be specifically used to: determine the multi-layer cascade codebook corresponding to a preset compression rate; determine the matching codeword and residual of the low-dimensional potential feature vector in the first-layer codebook of the multi-layer cascade codebook; for any layer of codebook other than the first-layer codebook in the multi-layer cascade codebook, iteratively update the residual output by the previous layer codebook in sequence and determine the matching codeword until a preset number of codebook layers is reached, to obtain an updated target residual and a multi-level matching codeword sequence, where the multi-level matching codeword sequence includes the matching codeword determined by each layer of the codebook when the residual is updated; and generate a discrete quantization code based on the matching codewords of each layer in the multi-level matching codeword sequence.

[0105] In some embodiments of the present application, the decoder includes a second convolution block group and a one-dimensional convolution layer connected in sequence; when the decoder is used to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data, the processing module 42 can be specifically used to: input the discrete quantization code into the second convolution block group, gradually restore the feature dimension and fuse the context information to obtain a context fusion feature vector; input the context fusion feature vector into the one-dimensional convolution layer of the decoder, map it to the time domain space through linear transformation, and generate a reconstructed audio signal corresponding to the target audio data.

[0106] In some embodiments of the present application, the second convolution block group includes multiple second convolution blocks, each second convolution block includes a 1-dimensional convolution layer, a LeakReLU activation layer, a weight normalization layer and a transposed convolution upsampling layer cascaded after the 1-dimensional convolution layer, and each second convolution block is connected between the weight normalization layer and the transposed convolution upsampling layer through a residual connection; when the discrete quantization code is input into the second convolution block group, the feature dimension is gradually restored and the context information is fused to obtain a context fusion feature vector, the processing module 42 can be specifically used to: for any one of the multiple second convolution blocks, use at least two 1-dimensional convolution layers, LeakReLU activation layers and weight normalization layers configured therein to perform convolution, activation and normalization processing on the second input feature vector in succession to obtain the context fusion feature vector. The second feature vector of the previous second convolution block, and the transposed convolution upsampling layer of the current second convolution block, perform feature dimension expansion processing on the fusion vector of the second feature vector and the second input feature vector through the transposed convolution operation to obtain an upsampled feature vector; the upsampled feature vector output by the last second convolution block in the multiple second convolution blocks is determined as the context fusion feature vector extracted by the second convolution block group; wherein, when the current second convolution block is the first second convolution block in the multiple second convolution blocks, the second input feature vector is discrete quantization coding; when the current second convolution block is any second convolution block other than the first second convolution block in the multiple second convolution blocks, the second input feature vector is the upsampled feature vector output by the current second convolution block corresponding to the previous second convolution block.

[0107] In some embodiments of the present application, Figure 5 As shown, the device further includes: a training module 43;

[0108] A training module 43 may be used to perform audio adversarial training on an audio codec quantization compression model using a discriminator;

[0109] Correspondingly, the training module 43 can be specifically used to determine the sample reconstructed audio signal corresponding to the sample audio data using the audio codec quantization compression model; after converting the sample audio data and the sample reconstructed audio signal into spectrograms respectively using short-time Fourier transform, the sample audio data and the sample reconstructed audio signal are input into the feature extraction layer of the discriminator to extract multi-level feature representations; the multi-level feature representations are subjected to binary classification discrimination using the classification layer of the discriminator to generate the authenticity probability distribution of the sample reconstructed audio signal and the original audio signal corresponding to the sample audio data; based on the authenticity probability distribution, the discriminator loss and the generator loss are calculated, and the network parameters of the discriminator, encoder and decoder are updated respectively through gradient back propagation until the preset adversarial training iteration conditions are reached, thereby obtaining the trained audio codec quantization compression model, wherein the generator loss is the weighted calculation result of the reconstruction loss, the discriminant loss and the quantization loss.

[0110] It should be noted that for other corresponding descriptions of the functional units involved in the audio data compression device provided in this embodiment, please refer to Figure 1 and Figure 3 The corresponding description in will not be repeated here.

[0111] Based on the above Figure 1 and Figure 3 The method shown in FIG. 1 is a method for performing the above-mentioned operation. Accordingly, this embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 and Figure 3 The audio data compression method shown.

[0112] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0113] Based on the above Figure 1 and Figure 3 The method shown, and Figure 4 and Figure 5 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device, which can be a personal computer, a tablet computer, a server, or other network equipment, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 3 The audio data compression method shown.

[0114] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like. The user interface may include a display, an input unit such as a keyboard, and the like. The optional user interface may also include a USB interface, a card reader interface, and the like. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), and the like.

[0115] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0116] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware.

[0118] The embodiment of the present invention can achieve efficient compression and high-fidelity reconstruction of audio data through the end-to-end architecture of lightweight encoders and decoders, the multi-layer cascade codebook design of residual vector quantization, and the weighted optimization of multiple loss functions (reconstruction loss, discrimination loss, and quantization loss) in adversarial training, significantly improving compression efficiency at extremely low bit rates. The frequency domain details and temporal dynamics of the audio are captured through hierarchical feature extraction of convolutional block groups and LSTM layers, and the integrity of feature transfer is ensured by residual connections and transposed convolutions. The authenticity judgment of the spectrum graph by the discriminator is combined to enhance the perceived quality. Ultimately, the model supports dynamic compression rate adjustment in real-time streaming processing scenarios, combining computational efficiency and sound quality fidelity, solving the problems of low bit rate and poor sound quality of traditional codecs, complex training of neural network models, and insufficient real-time performance.

[0119] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application. Those skilled in the art will understand that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0120] The serial numbers of the above application are for descriptive purposes only and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosure only discloses several specific implementation scenarios of the present application, but the present application is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present application.

Claims

1. An audio data compression method, characterized in that: include: Obtaining target audio data to be compressed; Inputting the target audio data into a trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training, and the audio codec quantization compression model includes an encoder, a residual vector quantization module, and a decoder; Inputting the target audio data into the trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data includes: Using the hierarchical neural network of the encoder to extract and compress the target audio data to obtain a low-dimensional latent feature vector; The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code; The decoder is used to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data.

2. The method according to claim 1, characterized in that The hierarchical neural network includes a 1-dimensional convolutional layer, a first convolutional block group, and a long short-term memory network layer connected in sequence; The target audio data is subjected to feature extraction and compression using the hierarchical neural network of the encoder to obtain a low-dimensional latent feature vector, including: Inputting the target audio data into the 1-dimensional convolutional layer of the encoder for initial feature extraction to obtain a preliminary feature vector; Inputting the preliminary feature vector into the first convolution block group for multi-level feature extraction and feature dimension compression processing to obtain a target down-sampled feature vector; The long short-term memory network layer is used to capture the temporal features in the target downsampled feature vector to obtain a low-dimensional latent feature vector.

3. The method according to claim 2, characterized in that The first convolution block group includes a plurality of first convolution blocks, each of the first convolution blocks includes a 1-dimensional convolution layer, a LeakReLU activation layer, a weight normalization layer, and a downsampling layer are cascaded after the 1-dimensional convolution layer, and each of the first convolution blocks is connected between the weight normalization layer and the downsampling layer through a residual connection; Inputting the preliminary feature vector into the first convolution block group for multi-level feature extraction and feature dimension compression processing to obtain a target down-sampled feature vector, including: For any one of the multiple first convolution blocks, using the 1-dimensional convolution layer, the LeakReLU activation layer, and the weight normalization layer configured therein, successively perform convolution, activation, and normalization processing on the first input feature vector to obtain a first feature vector of the current first convolution block, and using the downsampling layer of the current first convolution block to perform feature dimension compression processing on a fusion vector of the first feature vector and the first input feature vector to obtain a downsampled feature vector; Determining a down-sampled feature vector output by a last first convolution block among the multiple first convolution blocks as a target down-sampled feature vector extracted by the first convolution block group; Wherein, when the current first convolution block is the first first convolution block among the multiple first convolution blocks, the first input feature vector is the preliminary feature vector; when the current first convolution block is any first convolution block among the multiple first convolution blocks except the first first convolution block, the first input feature vector is the downsampled feature vector output by the current first convolution block corresponding to the previous first convolution block.

4. The method according to claim 1, wherein The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code, including: Determine a multi-layer cascade codebook corresponding to a preset compression rate; Determining a matching codeword and a residual of the low-dimensional potential feature vector in a first-layer codebook of the multi-layer cascaded codebook; For any codebook layer other than the first codebook layer in the multi-layer cascaded codebook, iteratively updating the residual output by the previous codebook layer and determining a matching codeword until a preset number of codebook layers is reached, thereby obtaining an updated target residual and a multi-level matching codeword sequence, wherein the multi-level matching codeword sequence includes the matching codeword determined when the residual of each codebook layer is updated; A discrete quantization code is generated based on the matching codewords of each layer in the multi-level matching codeword sequence.

5. The method according to claim 1, wherein The decoder includes a second convolution block group and a 1-dimensional convolution layer connected in sequence; Performing audio waveform restoration processing on the discrete quantization code using the decoder to obtain a reconstructed audio signal corresponding to the target audio data includes: Inputting the discrete quantization code into the second convolution block group, gradually restoring feature dimensions and fusing context information to obtain a context-fused feature vector; The context fusion feature vector is input into the 1-dimensional convolution layer of the decoder, mapped to the time domain space through linear transformation, and a reconstructed audio signal corresponding to the target audio data is generated.

6. The method according to claim 5, characterized in that The second convolution block group includes a plurality of second convolution blocks, each of the second convolution blocks includes a 1-dimensional convolution layer, a LeakReLU activation layer, a weight normalization layer, and a transposed convolution upsampling layer connected in cascade to the 1-dimensional convolution layer, and each of the second convolution blocks is connected between the weight normalization layer and the transposed convolution upsampling layer through a residual connection; Inputting the discrete quantization code into the second convolution block group, gradually restoring feature dimensions and fusing context information to obtain a context-fused feature vector, including: For any one of the multiple second convolution blocks, using the at least two one-dimensional convolutional layers, the LeakReLU activation layer, and the weight normalization layer configured therein, successively perform convolution, activation, and normalization processing on the second input feature vector to obtain a second feature vector of the current second convolution block, and using the transposed convolution upsampling layer of the current second convolution block to perform feature dimension expansion processing on a fusion vector of the second feature vector and the second input feature vector through a transposed convolution operation to obtain an upsampled feature vector; Determining an up-sampled feature vector output by a last second convolution block among the multiple second convolution blocks as a context fused feature vector extracted by the second convolution block group; Wherein, when the current second convolution block is the first second convolution block among the multiple second convolution blocks, the second input feature vector is the discrete quantization code; when the current second convolution block is any second convolution block among the multiple second convolution blocks except the first second convolution block, the second input feature vector is the upsampled feature vector output by the current second convolution block corresponding to the previous second convolution block.

7. The method according to claim 1, characterized in that The method further comprises: Performing audio adversarial training on the audio codec quantization compression model using a discriminator; Performing audio adversarial training on the audio codec quantization compression model using a discriminator, including: Determine the sample reconstructed audio signal corresponding to the sample audio data by using the audio codec quantization compression model; After converting the sample audio data and the sample reconstructed audio signal into spectrograms respectively using short-time Fourier transform, the spectrograms are input into the feature extraction layer of the discriminator to extract multi-level feature representations; Using the classification layer of the discriminator, a binary classification is performed on the multi-level feature representation to generate a probability distribution of authenticity between the sample reconstructed audio signal and the original audio signal corresponding to the sample audio data; Based on the authenticity probability distribution, the discriminator loss and the generator loss are calculated, and the network parameters of the discriminator, the encoder and the decoder are updated respectively through gradient back propagation until the preset adversarial training iteration conditions are reached, thereby obtaining a trained audio codec quantization compression model, wherein the generator loss is a weighted calculation result of the reconstruction loss, the discriminant loss and the quantization loss.

8. An audio data compression device, characterized in that: include: An acquisition module, used for acquiring target audio data to be compressed; a processing module, configured to input the target audio data into a trained audio codec quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, wherein the audio codec quantization compression model is obtained through audio adversarial training and comprises an encoder, a residual vector quantization module, and a decoder; The processing module is specifically used to: Using the hierarchical neural network of the encoder to extract and compress the target audio data to obtain a low-dimensional latent feature vector; The residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascade codebook to obtain a discrete quantization code; The decoder is used to perform audio waveform restoration processing on the discrete quantization code to obtain a reconstructed audio signal corresponding to the target audio data.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images