Neural network-based audio signal encoding and decoding method and apparatus, device and medium

Abstract

The present application belongs to the technical field of artificial intelligence, and is applicable to the medical field and the financial field. In order to solve a technical problem that it is difficult for audio codecs to effectively balance compression efficiency and sound quality restoration capability at low bit rates, disclosed are a neural network-based audio signal encoding and decoding method and apparatus, a device, and a medium. The method comprises: inputting a target encoded vector of an audio signal to be processed into a residual vector quantization module for quantization processing to obtain a magnitude latent vector and a phase latent vector of said audio signal, and respectively inputting the magnitude latent vector and the phase latent vector into a magnitude decoder and a phase decoder for parallel decoding to obtain an initial magnitude spectrum and an initial phase spectrum; and performing inverse short-time Fourier transform processing on the initial magnitude spectrum and the initial phase spectrum of said audio signal, so as to generate a reconstructed audio signal. The present application improves sound quality and compression efficiency of audio codecs at low bit rates.

Description

Neural network-based audio signal encoding and decoding methods, devices, equipment, and media

[0001] This application claims priority to Chinese Patent Application No. 202510112907.5, filed on January 23, 2025, entitled "Audio Signal Encoding and Decoding Method, Apparatus, Device and Medium Based on Neural Network", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application belongs to the field of artificial intelligence technology and is applicable to the medical and financial fields. In particular, it relates to an audio signal encoding and decoding method, device, equipment and medium based on neural networks. Background Technology

[0003] Currently, audio compression technology has been widely applied in many areas of daily life, particularly in voice communication, online music streaming, and audio storage and transmission. For example, codecs such as MP3, AAC, and Opus use signal processing algorithms to compress audio signals in the time or frequency domain, thereby reducing data volume. In the financial sector, for instance, in financial customer service centers, a large number of voice calls need to be stored and processed; audio compression technology can effectively reduce storage costs and transmission bandwidth, improving service efficiency. In the medical field, for example, in telemedicine consultations, voice communication between doctors and patients needs to be transmitted over a network; audio compression technology can ensure clear voice transmission, enabling smooth communication between doctors and patients even in remote areas with poor network conditions.

[0004] The inventors realized that although these traditional audio codecs have achieved certain results in practical applications, they still have some shortcomings in terms of sound quality assurance, compression efficiency improvement, phase information processing, and real-time performance, especially under low bit rate conditions, it is difficult to effectively balance compression efficiency and sound quality reproduction. Summary of the Invention

[0005] This application presents a neural network-based audio signal encoding and decoding method, apparatus, device, and medium to address the technical problem that existing audio codecs struggle to effectively balance compression efficiency and sound quality reproduction under low bit rate conditions.

[0006] In a first aspect, this application provides an audio signal encoding and decoding method based on a neural network, comprising:

[0007] The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0008] The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0009] The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed.

[0010] The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0011] The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0012] The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

[0013] Secondly, this application provides an audio signal encoding and decoding apparatus based on a neural network, the apparatus being used to implement the audio signal encoding and decoding method based on a neural network as described in the first aspect above, comprising:

[0014] The acquisition module is used to acquire the initial audio signal to be processed and perform short-time Fourier transform processing on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0015] The encoding module is used to input the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and the phase encoder respectively for parallel feature encoding, so as to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0016] The fusion module is used to perform feature fusion on the amplitude coding vector and phase coding vector corresponding to the audio signal to be processed to obtain the target coding vector of the audio signal to be processed.

[0017] The quantization module is used to input the target encoded vector of the audio signal to be processed into the residual vector quantization module for quantization processing, so as to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0018] The decoding module is used to input the amplitude latent vector and phase latent vector of the audio signal to be processed into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0019] The reconstruction module is used to perform inverse short-time Fourier transform processing on the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed, so as to generate the reconstructed audio signal.

[0020] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:

[0021] The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0022] The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0023] The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed.

[0024] The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0025] The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0026] The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

[0027] Fourthly, a non-volatile computer-readable storage medium is provided, the non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following steps:

[0028] The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0029] The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0030] The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed.

[0031] The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0032] The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0033] The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

[0034] In the above-mentioned scheme implemented by the neural network-based audio signal encoding and decoding method, apparatus, device and medium, the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are input in parallel to the amplitude encoder and phase encoder for feature encoding; then the target encoded vector of the audio signal to be processed is input to the residual vector quantization module for quantization processing; finally, the amplitude latent vector and phase latent vector of the audio signal to be processed are input to the amplitude decoder and phase decoder respectively for parallel decoding, thereby achieving efficient compression and high-fidelity decoding of audio signals, effectively improving the sound quality and compression efficiency of audio codecs at low bit rates. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 is a schematic diagram of an application environment for an audio signal encoding and decoding method based on a neural network according to an embodiment of this application.

[0037] Figure 2 is a flowchart illustrating an audio signal encoding / decoding method based on a neural network according to an embodiment of this application.

[0038] Figure 3 is a schematic flowchart of a specific implementation of step S10 in Figure 2.

[0039] Figure 4 is a schematic flowchart of a specific implementation of step S20 in Figure 2.

[0040] Figure 5 is a schematic flowchart of a specific implementation of step S40 in Figure 2.

[0041] Figure 6 is a schematic flowchart of a specific implementation of step S50 in Figure 2.

[0042] Figure 7 is a schematic diagram of a neural network-based audio signal encoding and decoding device according to an embodiment of this application;

[0043] Figure 8 is a structural schematic diagram of a computer device according to an embodiment of this application.

[0044] Figure 9 is another structural schematic diagram of a computer device according to one embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] The neural network-based audio signal encoding and decoding method provided in this application embodiment can be applied in the application environment shown in Figure 1. Figure 1 is a schematic diagram of an application environment for the neural network-based audio signal encoding and decoding method in one embodiment of this application; wherein, the client communicates with the server through a network. The server can obtain the initial audio signal to be processed through the client, and perform short-time Fourier transform processing on the obtained initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed; the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed; the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are feature fused to obtain the target encoding vector of the audio signal to be processed; the target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed; the amplitude latent vector and phase latent vector of the audio signal to be processed are quantized... The vectors are input to the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are then subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal. In this application, the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are input in parallel to the amplitude encoder and phase encoder for feature encoding. Then, the target encoded vector of the audio signal to be processed is input to the residual vector quantization module for quantization processing. Finally, the latent amplitude vector and latent phase vector of the audio signal to be processed are input to the amplitude decoder and phase decoder respectively for parallel decoding, achieving efficient compression and high-fidelity decoding of the audio signal, effectively improving the sound quality and compression efficiency of the audio codec at low bit rates. The following detailed description of specific embodiments further illustrates this application.

[0047] Please refer to Figure 2, which is a flowchart illustrating an audio signal encoding and decoding method based on a neural network provided in this application embodiment. The audio signal encoding and decoding method based on a neural network includes the following steps:

[0048] S10: Acquire the initial audio signal to be processed, and perform a short-time Fourier transform on the acquired initial audio signal to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. For example, when performing voice recording analysis of financial transactions, it is first necessary to acquire the initial audio signal to be processed during the transaction process. These audio signals may contain key information such as the dialogue between the two parties and transaction instructions. In telemedicine, it is necessary to acquire the initial audio signal to be processed containing information such as the patient's condition description. In order to further analyze these audio data in depth, it is necessary to perform a short-time Fourier transform on the acquired initial audio signal to be processed, thereby obtaining the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. Specifically, as shown in Figure 3, Figure 3 is a flowchart of a specific implementation of step S10 in Figure 2, which specifically includes the following steps S11-S14:

[0049] S11: The acquired initial audio signal to be processed is divided into multiple window audio signals, and a window function is selected to process the window audio signals. Specifically, in this application, the audio signal is usually a long, continuous sequence of data in time, and directly performing a Fourier transform on the whole signal is difficult to effectively present the frequency characteristics of the signal changing over time. Therefore, the acquired initial audio signal to be processed must first be divided into multiple relatively independent window audio signals. Secondly, the corresponding window function needs to be selected according to the specific application scenario.

[0050] S12: Set the window length and window step size for each window's audio signal. Specifically, in this application, the window length refers to the time range covered by each window's audio signal, and the window length needs to be set reasonably according to actual time requirements. Secondly, the window step size specifies the interval between two adjacent window audio signals, and an appropriate window step size needs to be determined based on the information completeness and computational cost of each window's audio signal.

[0051] S13: Windowing processing is performed on the audio signals of each window based on the selected window function, window length, and window step size. Specifically, in this application, after determining the window function, window length, and window step size in steps S11 and S12, windowing processing can be performed on the audio signals of each window based on these parameters. By multiplying by an appropriate window function, the transition of the window audio signals at the edges in the time domain can be made smoother, thereby making the spectral information obtained after subsequent short-time Fourier transform more accurate, reliable, and closer to the actual frequency characteristics of the original audio signal.

[0052] S14: The short-time Fourier transform is used to calculate the spectral information corresponding to the audio signal of each window after windowing processing, and the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are calculated based on the calculated spectral information corresponding to the audio signal to be processed. In this application, the short-time Fourier transform is used to calculate the audio signal of each window after windowing processing, converting the time-domain window audio signal to the frequency domain to obtain the amplitude spectrum and phase spectrum corresponding to each window audio signal. Specifically, the calculation of the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed includes:

[0053] The formula for calculating the initial amplitude spectrum corresponding to the audio signal to be processed is:

[0054] Where |Y(k)| represents the initial amplitude spectrum of the audio signal to be processed, Y(k) represents the spectral information of the audio signal to be processed, Re(Y(k)) represents the real part of Y(k), and Im(Y(k)) represents the imaginary part of Y(k);

[0055] The formula for calculating the initial phase spectrum corresponding to the audio signal to be processed is:

[0056] in, This represents the initial phase spectrum of the audio signal to be processed.

[0057] S20: The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are input into the amplitude encoder and phase encoder respectively for parallel feature encoding to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed. Specifically, as shown in Figure 4, which is a flowchart of a specific implementation of step S20 in Figure 2, it specifically includes the following steps S21-S24:

[0058] S21: The initial amplitude spectrum and phase spectrum corresponding to the audio signal to be processed are input in parallel to the convolutional layers of the amplitude encoder and phase encoder for feature extraction, thereby obtaining the amplitude feature vector and phase feature vector corresponding to the audio signal to be processed. In this application, the convolutional kernel of the convolutional layer captures the feature information in the amplitude spectrum and phase spectrum data within a local region. For example, for the amplitude spectrum, the intensity variation features exhibited locally in different frequency ranges are extracted. For the phase spectrum, the local trend of phase variation can be extracted. Through multiple different convolutional kernels and multiple convolution operations, more representative and abstract amplitude feature vectors and phase feature vectors can be extracted from the original amplitude spectrum and phase spectrum.

[0059] S22: The amplitude feature vector and phase feature vector corresponding to the audio signal to be processed are input in parallel to the normalization layer of the amplitude encoder and phase encoder for normalization processing, so as to obtain the normalized amplitude feature vector and phase feature vector of the audio signal to be processed. In this application, through normalization processing, the amplitude feature vector and phase feature vector can be unified to a relatively standard value range, which helps to improve the stability of subsequent processing and accelerate the calculation convergence speed.

[0060] S23: The normalized amplitude feature vector and phase feature vector of the audio signal to be processed are input in parallel to the feedforward layer of the amplitude encoder and phase encoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude feature vector and phase feature vector of the audio signal to be processed. Specifically, in this application, nonlinear transformation, for example, using an activation function, can create nonlinear relationships between the elements in the feature vector, thereby better fitting the actual complex changes of the audio signal and mining deeper feature information.

[0061] S24: The amplitude feature vector and phase feature vector of the nonlinearly transformed audio signal are input in parallel to the downsampling layers of the amplitude encoder and phase encoder for downsampling processing to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed. Specifically, in this application, by downsampling the amplitude feature vector and phase feature vector after nonlinear transformation, the dimensionality of the feature vector is effectively reduced, while retaining the most critical feature information as much as possible.

[0062] S30: The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed. Specifically, in this application, the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed can be spliced ​​and fused to obtain the target encoding vector of the audio signal to be processed. For example, in the analysis of customer service hotline recordings of financial institutions, the target encoding vector can comprehensively reflect various features in the customer's voice, such as tone strength and intonation changes, helping financial institutions to more accurately analyze customer emotions, intentions, and needs, thereby optimizing service strategies.

[0063] S40: The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. For example, in financial transaction voice monitoring, financial transaction audio often contains a large amount of sensitive information, such as transaction amount and transaction instructions. Through quantization processing, key features can be effectively extracted to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. These latent vectors can be used for subsequent risk assessment, such as analyzing sound features to determine whether there are abnormal emotions between the trading parties and whether they may cause transaction risks, providing risk warnings for financial institutions and ensuring the safety and stability of transactions. Specifically, as shown in Figure 5, Figure 5 is a flowchart of a specific implementation of step S40 in Figure 2, which specifically includes the following steps S41-S43:

[0064] S41: Perform cluster analysis on the target feature vector of the audio signal to be processed to generate a codebook corresponding to the audio signal to be processed. Specifically, this application generates a codebook corresponding to the audio signal to be processed by performing cluster analysis on the target feature vector of the audio signal to be processed. The generated codebook records relevant information of each cluster, such as the center vector of each cluster.

[0065] S42: Calculate the quantization codewords in the codebook corresponding to the generated audio signal to be processed, and calculate the residual feature vector of the audio signal to be processed based on the quantization codewords of each codebook. Specifically, in this application, for each target feature vector, the cluster to which each target feature vector belongs is found, and then the target feature vector is represented by the quantization codeword corresponding to that cluster. Then, the difference between the target feature vector and the corresponding quantization codeword is calculated, and the calculated difference vector is the residual feature vector of the audio signal to be processed.

[0066] S43: Based on the calculated residual feature vector of the audio signal to be processed, extract the amplitude latent vector and phase latent vector of the audio signal to be processed. Specifically, by extracting the amplitude latent vector and phase latent vector, this application helps to further mine higher-level feature information in the audio signal, effectively reducing data redundancy while preserving as much key amplitude and phase information in the audio signal as possible.

[0067] S50: The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. Specifically, as shown in Figure 6, which is a flowchart of a specific implementation of step S50 in Figure 2, it specifically includes the following steps S51-S54:

[0068] S51: The amplitude latent vector and feature latent vector of the audio signal to be processed are input in parallel to the upsampling layers of the amplitude decoder and phase decoder for upsampling processing, to obtain the upsampled amplitude encoded vector and phase encoded vector of the audio signal to be processed. Specifically, this application obtains the upsampled amplitude encoded vector and phase encoded vector of the audio signal to be processed by upsampling the amplitude latent vector and feature latent vector of the audio signal to be processed. For example, the audio information contained in the vector at low resolution or low dimension is not detailed enough. Upsampling enables the acquisition of more detailed audio feature information, making the subsequent reconstructed audio signal more accurate.

[0069] S52: The upsampled amplitude encoding vector and phase encoding vector of the audio signal to be processed are input to the normalization layer of the amplitude decoder and phase decoder for normalization processing to obtain the normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed. Specifically, in this application, the normalization processing can make the data distribution of the amplitude encoding vector and phase encoding vector more stable.

[0070] S53: The normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed are input to the feedforward layer of the amplitude decoder and phase decoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude encoding vector and phase encoding vector of the audio signal to be processed. Specifically, this application uses nonlinear transformation processing and an activation function to change the relationship between the elements in the amplitude encoding vector and phase encoding vector, so that the nonlinearly transformed amplitude encoding vector and phase encoding vector can better fit the complex characteristics of the audio signal itself, which is convenient for further reconstruction of high-quality audio-related spectrum information.

[0071] S54: The amplitude-encoded vector and phase-encoded vector of the nonlinearly transformed audio signal are input into the convolutional layers of the amplitude decoder and phase decoder for feature reconstruction, obtaining the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. Finally, the nonlinearly transformed amplitude-encoded vector and phase-encoded vector are input into the convolutional layers of the amplitude decoder and phase decoder. Utilizing the characteristics of the convolutional layer, feature reconstruction is performed on the input vectors, and the audio features contained in the previously processed amplitude-encoded vector and phase-encoded vector are reintegrated to finally obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0072] S60: Perform inverse short-time Fourier transform (ISFT) processing on the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed to generate a reconstructed audio signal. In this application, IFT processing is used to calculate the frequency energy information and frequency phase information provided by the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. By applying the calculation rules of IFT, the form of the audio signal in the time domain is effectively restored to generate a reconstructed audio signal. For example, in the financial field, audio recordings of financial transactions are crucial; any omission or distortion of information can lead to serious consequences. In this scenario, applying the calculation rules of IFT can effectively restore the form of the audio signal in the time domain to generate a reconstructed audio signal. The reconstructed audio signal can be used for subsequent auditing, dispute resolution, etc., ensuring the transparency and traceability of the financial transaction process.

[0073] In one embodiment of this application, the neural network-based audio signal encoding and decoding method further includes: calculating the signal-to-noise ratio (SNR) of the reconstructed audio signal to be processed, and updating the parameter information of the encoder and decoder of the neural network according to the calculated SNR. Specifically, in this application, by calculating the SNR of the reconstructed audio signal, the quality of the reconstruction can be intuitively understood. If the SNR is high, it means that the reconstructed audio signal is relatively pure and close to the original audio, indicating that the parameter settings of the current encoder and decoder are relatively reasonable; conversely, if the SNR is low, it indicates that the reconstructed audio signal is of poor quality and has a lot of noise, requiring parameter adjustment and optimization.

[0074] As can be seen, in the above scheme, by inputting the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed into the amplitude encoder and phase encoder respectively for parallel feature encoding; then inputting the target encoded vector of the audio signal to be processed into the residual vector quantization module for quantization processing; and finally inputting the amplitude latent vector and phase latent vector of the audio signal to be processed into the amplitude decoder and phase decoder respectively for parallel decoding, efficient compression and high-fidelity decoding of audio signals are achieved, effectively improving the sound quality and compression efficiency of audio codecs at low bit rates.

[0075] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0076] In one embodiment, a neural network-based audio signal encoding and decoding device is provided, which corresponds one-to-one with the neural network-based audio signal encoding and decoding methods described in the above embodiments. As shown in Figure 7, which is a structural schematic diagram of a neural network-based audio signal encoding and decoding device according to an embodiment of this application, the device includes an acquisition module 71, an encoding module 72, a fusion module 73, a quantization module 74, a decoding module 75, and a reconstruction module 76. Detailed descriptions of each functional module are as follows:

[0077] The acquisition module 71 is used to acquire the initial audio signal to be processed and to perform short-time Fourier transform processing on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0078] Encoding module 72 is used to input the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and the phase encoder respectively for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed;

[0079] The fusion module 73 is used to perform feature fusion on the amplitude coding vector and phase coding vector corresponding to the audio signal to be processed to obtain the target coding vector of the audio signal to be processed.

[0080] The quantization module 74 is used to input the target encoding vector of the audio signal to be processed into the residual vector quantization module for quantization processing, so as to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0081] The decoding module 75 is used to input the amplitude latent vector and phase latent vector of the audio signal to be processed into the amplitude decoder and the phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and the initial phase spectrum of the audio signal to be processed.

[0082] The reconstruction module 76 is used to perform inverse short-time Fourier transform processing on the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed, and generate the reconstructed audio signal.

[0083] In one embodiment, the acquisition module 71 is specifically used for:

[0084] The acquired initial audio signal to be processed is divided into multiple window audio signals, and a window function is selected to process the window audio signals.

[0085] Set the window length and window step size for the audio signals of each window;

[0086] Based on the selected window function for processing the audio signals of the window, the window length of the audio signals of each window, and the window step size of the audio signals of each window, the audio signals of each window are windowed.

[0087] The short-time Fourier transform is used to calculate the spectral information of the audio signal in each window after windowing. Based on the calculated spectral information of the audio signal to be processed, the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are calculated.

[0088] In one embodiment, the encoding module 72 is specifically used for:

[0089] The initial amplitude spectrum and phase spectrum corresponding to the audio signal to be processed are input in parallel to the convolutional layers of the amplitude encoder and the phase encoder for feature extraction, so as to obtain the amplitude feature vector and phase feature vector corresponding to the audio signal to be processed.

[0090] The amplitude feature vector and phase feature vector corresponding to the audio signal to be processed are input in parallel to the normalization layer of the amplitude encoder and the phase encoder for normalization processing, so as to obtain the normalized amplitude feature vector and phase feature vector of the audio signal to be processed.

[0091] The normalized amplitude feature vector and phase feature vector of the audio signal to be processed are input in parallel to the forward layer of the amplitude encoder and the phase encoder for nonlinear transformation processing to obtain the nonlinear transformed amplitude feature vector and phase feature vector of the audio signal to be processed.

[0092] The amplitude feature vector and phase feature vector of the audio signal to be processed after nonlinear transformation are input in parallel to the downsampling layer of the amplitude encoder and phase encoder for downsampling processing to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed.

[0093] In one embodiment, the quantization module 74 is specifically used for:

[0094] Cluster analysis is performed on the target feature vector of the audio signal to be processed to generate the codebook corresponding to the audio signal to be processed.

[0095] Calculate the quantization codewords in the codebook corresponding to the generated audio signal to be processed, and calculate the residual feature vector of the audio signal to be processed based on the quantization codewords of each codebook.

[0096] Based on the calculated residual feature vector of the audio signal to be processed, the amplitude latent vector and phase latent vector of the audio signal to be processed are extracted.

[0097] In one embodiment, the decoding module 75 is specifically used for:

[0098] The amplitude latent vector and feature latent vector of the audio signal to be processed are input in parallel to the upsampling layer of the amplitude decoder and the phase decoder for upsampling processing, so as to obtain the upsampled amplitude encoding vector and phase encoding vector of the audio signal to be processed.

[0099] The amplitude encoding vector and phase encoding vector of the upsampled audio signal to be processed are input to the normalization layer of the amplitude decoder and phase decoder for normalization processing to obtain the normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed.

[0100] The normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed are input into the forward layer of the amplitude decoder and the phase decoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude encoding vector and phase encoding vector of the audio signal to be processed.

[0101] The amplitude encoding vector and phase encoding vector of the audio signal to be processed after nonlinear transformation are input into the convolutional layers of the amplitude decoder and phase decoder for feature reconstruction to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0102] In one embodiment, the acquisition module 71 is specifically used for:

[0103] The formula for calculating the initial amplitude spectrum corresponding to the audio signal to be processed is:

[0104] Where |Y(k)| represents the initial amplitude spectrum of the audio signal to be processed, Y(k) represents the spectral information of the audio signal to be processed, Re(Y(k)) represents the real part of Y(k), and Im(Y(k)) represents the imaginary part of Y(k);

[0105] The formula for calculating the initial phase spectrum corresponding to the audio signal to be processed is:

[0106] in, This represents the initial phase spectrum of the audio signal to be processed.

[0107] In one embodiment, the neural network-based audio signal encoding / decoding device is further configured to:

[0108] The signal-to-noise ratio (SNR) of the reconstructed audio signal to be processed is calculated, and the parameters of the encoder and decoder of the neural network are updated based on the calculated SNR.

[0109] This application provides an audio signal encoding and decoding device based on a neural network. The device performs parallel feature encoding by inputting the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed into an amplitude encoder and a phase encoder, respectively. Then, the target encoded vector of the audio signal to be processed is input into a residual vector quantization module for quantization processing. Finally, the amplitude latent vector and phase latent vector of the audio signal to be processed are input into an amplitude decoder and a phase decoder, respectively, for parallel decoding. This achieves efficient compression and high-fidelity decoding of the audio signal, effectively improving the sound quality and compression efficiency of the audio codec at low bit rates.

[0110] Specific limitations regarding the neural network-based audio signal encoding / decoding device can be found in the limitations of the neural network-based audio signal encoding / decoding method described above, and will not be repeated here. Each module in the aforementioned neural network-based audio signal encoding / decoding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0111] In one embodiment, a computer device is provided, which may be a server. Its internal structure is shown in Figure 8, a schematic diagram of a computer device according to an embodiment of this application. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a neural network-based audio signal encoding / decoding method on the server side.

[0112] In one embodiment, a computer device is provided, which can be a client. Its internal structure is illustrated in Figure 9, which is another structural diagram of the computer device according to one embodiment of this application. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of a neural network-based audio signal encoding / decoding method.

[0113] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0114] The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0115] The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0116] The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed.

[0117] The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0118] The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0119] The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

[0120] In one embodiment, a non-volatile computer-readable storage medium is provided on which a computer program is stored. The computer-readable storage medium may be non-volatile or volatile. When the computer program is executed by a processor, it performs the following steps:

[0121] The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed.

[0122] The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed.

[0123] The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed.

[0124] The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed.

[0125] The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

[0126] The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

[0127] It should be noted that the functions or steps that can be implemented by the non-volatile computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0129] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0130] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An audio signal encoding and decoding method based on neural networks, wherein, include: The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed. The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed. The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

2. The audio signal encoding and decoding method based on neural networks according to claim 1, wherein, The step of performing a short-time Fourier transform on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed includes: The acquired initial audio signal to be processed is divided into multiple window audio signals, and a window function is selected to process the window audio signals. Set the window length and window step size for the audio signals of each window; Based on the selected window function for processing the audio signals of the window, the window length of the audio signals of each window, and the window step size of the audio signals of each window, the audio signals of each window are windowed. The short-time Fourier transform is used to calculate the spectral information of the audio signal in each window after windowing. Based on the calculated spectral information of the audio signal to be processed, the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are calculated.

3. The audio signal encoding and decoding method based on neural networks according to claim 1, wherein, The step of inputting the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and phase encoder respectively for parallel feature encoding to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed includes: The initial amplitude spectrum and phase spectrum corresponding to the audio signal to be processed are input in parallel to the convolutional layers of the amplitude encoder and the phase encoder for feature extraction, so as to obtain the amplitude feature vector and phase feature vector corresponding to the audio signal to be processed. The amplitude feature vector and phase feature vector corresponding to the audio signal to be processed are input in parallel to the normalization layer of the amplitude encoder and the phase encoder for normalization processing, so as to obtain the normalized amplitude feature vector and phase feature vector of the audio signal to be processed. The normalized amplitude feature vector and phase feature vector of the audio signal to be processed are input in parallel to the forward layer of the amplitude encoder and the phase encoder for nonlinear transformation processing to obtain the nonlinear transformed amplitude feature vector and phase feature vector of the audio signal to be processed. The amplitude feature vector and phase feature vector of the audio signal to be processed after nonlinear transformation are input in parallel to the downsampling layer of the amplitude encoder and phase encoder for downsampling processing to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed.

4. The audio signal encoding and decoding method based on neural networks according to claim 1, wherein, The step of inputting the target feature vector of the audio signal to be processed into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed includes: Cluster analysis is performed on the target feature vector of the audio signal to be processed to generate the codebook corresponding to the audio signal to be processed. Calculate the quantization codewords in the codebook corresponding to the generated audio signal to be processed, and calculate the residual feature vector of the audio signal to be processed based on the quantization codewords of each codebook. Based on the calculated residual feature vector of the audio signal to be processed, the amplitude latent vector and phase latent vector of the audio signal to be processed are extracted.

5. The audio signal encoding and decoding method based on neural networks according to claim 1, wherein, The step of inputting the amplitude latent vector and feature latent vector of the audio signal to be processed in parallel into the amplitude decoder and phase decoder for decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed includes: The amplitude latent vector and feature latent vector of the audio signal to be processed are input in parallel to the upsampling layer of the amplitude decoder and the phase decoder for upsampling processing, so as to obtain the amplitude encoding vector and phase encoding vector of the audio signal to be processed after upsampling. The amplitude encoding vector and phase encoding vector of the upsampled audio signal to be processed are input to the normalization layer of the amplitude decoder and phase decoder for normalization processing to obtain the normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed. The normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed are input into the forward layer of the amplitude decoder and the phase decoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude encoding vector and phase encoding vector of the audio signal to be processed. The amplitude encoding vector and phase encoding vector of the audio signal to be processed after nonlinear transformation are input into the convolutional layers of the amplitude decoder and phase decoder for feature reconstruction to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

6. The audio signal encoding and decoding method based on neural networks according to claim 2, wherein, The step of calculating the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed based on the calculated spectral information of the audio signal to be processed includes: The formula for calculating the initial amplitude spectrum corresponding to the audio signal to be processed is: Where |Y(k)| represents the initial amplitude spectrum of the audio signal to be processed, Y(k) represents the spectral information of the audio signal to be processed, Re(Y(k)) represents the real part of Y(k), and Im(Y(k)) represents the imaginary part of Y(k); The formula for calculating the initial phase spectrum corresponding to the audio signal to be processed is: in, This represents the initial phase spectrum of the audio signal to be processed.

7. The audio signal encoding and decoding method based on neural networks according to claim 1, wherein, The method further includes: The signal-to-noise ratio (SNR) of the reconstructed audio signal to be processed is calculated, and the parameters of the encoder and decoder of the neural network are updated based on the calculated SNR.

8. An audio signal encoding / decoding apparatus based on a neural network, the apparatus being used to implement the audio signal encoding / decoding method based on a neural network as described in any one of claims 1-7, wherein, include: The acquisition module is used to acquire the initial audio signal to be processed and perform short-time Fourier transform processing on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. The encoding module is used to input the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and the phase encoder respectively for parallel feature encoding, so as to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed. The fusion module is used to perform feature fusion on the amplitude coding vector and phase coding vector corresponding to the audio signal to be processed to obtain the target coding vector of the audio signal to be processed. The quantization module is used to input the target encoded vector of the audio signal to be processed into the residual vector quantization module for quantization processing, so as to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. The decoding module is used to input the amplitude latent vector and phase latent vector of the audio signal to be processed into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. The reconstruction module is used to perform inverse short-time Fourier transform processing on the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed, so as to generate the reconstructed audio signal.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it performs the following steps: The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed. The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed. The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

10. The computer device according to claim 9, wherein, The step of performing a short-time Fourier transform on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed includes: The acquired initial audio signal to be processed is divided into multiple window audio signals, and a window function is selected to process the window audio signals. Set the window length and window step size for the audio signals of each window; Based on the selected window function for processing the audio signals of the window, the window length of the audio signals of each window, and the window step size of the audio signals of each window, the audio signals of each window are windowed. The short-time Fourier transform is used to calculate the spectral information of the audio signal in each window after windowing. Based on the calculated spectral information of the audio signal to be processed, the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are calculated.

11. The computer device according to claim 9, wherein, The step of inputting the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and phase encoder respectively for parallel feature encoding to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed includes: The initial amplitude spectrum and phase spectrum corresponding to the audio signal to be processed are input in parallel to the convolutional layers of the amplitude encoder and the phase encoder for feature extraction, so as to obtain the amplitude feature vector and phase feature vector corresponding to the audio signal to be processed. The amplitude feature vector and phase feature vector corresponding to the audio signal to be processed are input in parallel to the normalization layer of the amplitude encoder and the phase encoder for normalization processing, so as to obtain the normalized amplitude feature vector and phase feature vector of the audio signal to be processed. The normalized amplitude feature vector and phase feature vector of the audio signal to be processed are input in parallel to the forward layer of the amplitude encoder and the phase encoder for nonlinear transformation processing to obtain the nonlinear transformed amplitude feature vector and phase feature vector of the audio signal to be processed. The amplitude feature vector and phase feature vector of the audio signal to be processed after nonlinear transformation are input in parallel to the downsampling layer of the amplitude encoder and phase encoder for downsampling processing to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed.

12. The computer device according to claim 9, wherein, The step of inputting the target feature vector of the audio signal to be processed into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed includes: Cluster analysis is performed on the target feature vector of the audio signal to be processed to generate the codebook corresponding to the audio signal to be processed. Calculate the quantization codewords in the codebook corresponding to the generated audio signal to be processed, and calculate the residual feature vector of the audio signal to be processed based on the quantization codewords of each codebook. Based on the calculated residual feature vector of the audio signal to be processed, the amplitude latent vector and phase latent vector of the audio signal to be processed are extracted.

13. The computer device according to claim 9, wherein, The step of inputting the amplitude latent vector and feature latent vector of the audio signal to be processed in parallel into the amplitude decoder and phase decoder for decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed includes: The amplitude latent vector and feature latent vector of the audio signal to be processed are input in parallel to the upsampling layer of the amplitude decoder and the phase decoder for upsampling processing, so as to obtain the amplitude encoding vector and phase encoding vector of the audio signal to be processed after upsampling. The amplitude encoding vector and phase encoding vector of the upsampled audio signal to be processed are input to the normalization layer of the amplitude decoder and phase decoder for normalization processing to obtain the normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed. The normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed are input into the forward layer of the amplitude decoder and the phase decoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude encoding vector and phase encoding vector of the audio signal to be processed. The amplitude encoding vector and phase encoding vector of the audio signal to be processed after nonlinear transformation are input into the convolutional layers of the amplitude decoder and phase decoder for feature reconstruction to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

14. The computer device according to claim 10, wherein, The step of calculating the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed based on the calculated spectral information of the audio signal to be processed includes: The formula for calculating the initial amplitude spectrum corresponding to the audio signal to be processed is: Where |Y(k)| represents the initial amplitude spectrum of the audio signal to be processed, Y(k) represents the spectral information of the audio signal to be processed, Re(Y(k)) represents the real part of Y(k), and Im(Y(k)) represents the imaginary part of Y(k); The formula for calculating the initial phase spectrum corresponding to the audio signal to be processed is: in, This represents the initial phase spectrum of the audio signal to be processed.

15. A non-volatile computer-readable storage medium storing a computer program, wherein, When the computer program is executed by the processor, it performs the following steps: The initial audio signal to be processed is acquired, and a short-time Fourier transform is performed on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed are respectively input into the amplitude encoder and the phase encoder for parallel feature encoding to obtain the amplitude encoding vector and the phase encoding vector corresponding to the audio signal to be processed. The amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed are fused to obtain the target encoding vector of the audio signal to be processed. The target encoding vector of the audio signal to be processed is input into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed. The amplitude latent vector and phase latent vector of the audio signal to be processed are input into the amplitude decoder and phase decoder respectively for parallel decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed. The initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are subjected to inverse short-time Fourier transform processing to generate the reconstructed audio signal.

16. The non-volatile computer-readable storage medium according to claim 15, wherein, The step of performing a short-time Fourier transform on the acquired initial audio signal to be processed to obtain the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed includes: The acquired initial audio signal to be processed is divided into multiple window audio signals, and a window function is selected to process the window audio signals. Set the window length and window step size for the audio signals of each window; Based on the selected window function for processing the audio signals of the window, the window length of the audio signals of each window, and the window step size of the audio signals of each window, the audio signals of each window are windowed. The short-time Fourier transform is used to calculate the spectral information of the audio signal in each window after windowing. Based on the calculated spectral information of the audio signal to be processed, the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed are calculated.

17. The non-volatile computer-readable storage medium according to claim 15, wherein, The step of inputting the initial amplitude spectrum and initial phase spectrum corresponding to the audio signal to be processed into the amplitude encoder and phase encoder respectively for parallel feature encoding to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed includes: The initial amplitude spectrum and phase spectrum corresponding to the audio signal to be processed are input in parallel to the convolutional layers of the amplitude encoder and the phase encoder for feature extraction, so as to obtain the amplitude feature vector and phase feature vector corresponding to the audio signal to be processed. The amplitude feature vector and phase feature vector corresponding to the audio signal to be processed are input in parallel to the normalization layer of the amplitude encoder and the phase encoder for normalization processing, so as to obtain the normalized amplitude feature vector and phase feature vector of the audio signal to be processed. The normalized amplitude feature vector and phase feature vector of the audio signal to be processed are input in parallel to the forward layer of the amplitude encoder and the phase encoder for nonlinear transformation processing to obtain the nonlinear transformed amplitude feature vector and phase feature vector of the audio signal to be processed. The amplitude feature vector and phase feature vector of the audio signal to be processed after nonlinear transformation are input in parallel to the downsampling layer of the amplitude encoder and phase encoder for downsampling processing to obtain the amplitude encoding vector and phase encoding vector corresponding to the audio signal to be processed.

18. The non-volatile computer-readable storage medium according to claim 15, wherein, The step of inputting the target feature vector of the audio signal to be processed into the residual vector quantization module for quantization processing to obtain the amplitude latent vector and phase latent vector of the audio signal to be processed includes: Cluster analysis is performed on the target feature vector of the audio signal to be processed to generate the codebook corresponding to the audio signal to be processed. Calculate the quantization codewords in the codebook corresponding to the generated audio signal to be processed, and calculate the residual feature vector of the audio signal to be processed based on the quantization codewords of each codebook. Based on the calculated residual feature vector of the audio signal to be processed, the amplitude latent vector and phase latent vector of the audio signal to be processed are extracted.

19. The non-volatile computer-readable storage medium according to claim 15, wherein, The step of inputting the amplitude latent vector and feature latent vector of the audio signal to be processed in parallel into the amplitude decoder and phase decoder for decoding to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed includes: The amplitude latent vector and feature latent vector of the audio signal to be processed are input in parallel to the upsampling layer of the amplitude decoder and the phase decoder for upsampling processing, so as to obtain the amplitude encoding vector and phase encoding vector of the audio signal to be processed after upsampling. The amplitude encoding vector and phase encoding vector of the upsampled audio signal to be processed are input to the normalization layer of the amplitude decoder and phase decoder for normalization processing to obtain the normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed. The normalized amplitude encoding vector and phase encoding vector of the audio signal to be processed are input into the forward layer of the amplitude decoder and the phase decoder for nonlinear transformation processing to obtain the nonlinearly transformed amplitude encoding vector and phase encoding vector of the audio signal to be processed. The amplitude encoding vector and phase encoding vector of the audio signal to be processed after nonlinear transformation are input into the convolutional layers of the amplitude decoder and phase decoder for feature reconstruction to obtain the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed.

20. The non-volatile computer-readable storage medium according to claim 16, wherein, The step of calculating the initial amplitude spectrum and initial phase spectrum of the audio signal to be processed based on the calculated spectral information of the audio signal to be processed includes: The formula for calculating the initial amplitude spectrum corresponding to the audio signal to be processed is: Where |Y(k)| represents the initial amplitude spectrum of the audio signal to be processed, Y(k) represents the spectral information of the audio signal to be processed, Re(Y(k)) represents the real part of Y(k), and Im(Y(k)) represents the imaginary part of Y(k); The formula for calculating the initial phase spectrum corresponding to the audio signal to be processed is: in, This represents the initial phase spectrum of the audio signal to be processed.

Images