Hierarchical voice coding and decoding method based on MDCT coefficient residual vector quantization

Abstract

The invention discloses a layered voice coding and decoding method based on MDCT coefficient residual vector quantization, which comprises the following steps: preprocessing voice data for training, and training to generate a multi-level quantization codebook of MDCT; the method comprises the following steps of: enhancing a high-frequency component of voice to be coded through a pre-emphasis filter, windowing a pre-emphasis voice signal, generating an MDCT coefficient based on an FFT (Fast Fourier Transform) algorithm, determining a bit allocation mode according to a code rate to generate a coding control bit, compressing a codebook index and the coding control bit through Huffman coding, and transmitting a code stream to a channel; the method comprises the steps of reading a code stream in a channel, obtaining a coding control bit and an MDCT codebook index through Huffman decoding, determining a voice sampling rate and a bit distribution mode through the coding control bit, obtaining an MDCT vector based on MDCT codebook index retrieval, and restoring an original voice signal through de-emphasis filtering. Through the design of MDCT layered coding, the coder can dynamically adjust coding bit distribution according to the code rate condition, so that the method adapts to the actual communication environment and deals with the sudden network condition.

Description

Technical Field

[0001] The invention relates to the fields of signal processing and speech compression coding, and belongs to a speech compression coding and decoding method based on signal processing, and in particular to a layered speech coding and decoding method based on MDCT coefficient residual vector quantization. Background Art

[0002] The purpose of speech coding is to compress speech signals, thereby reducing the required transmission bandwidth and storage space, and maintaining the clarity and naturalness of speech as much as possible. It is one of the core technologies in the field of communications. In recent years, with the continuous development of communication technology, the continuous increase in communication bandwidth, and the continuous improvement of people's requirements for voice communication quality, speech coding technology is gradually developing in the direction of wideband, variable rate, and high integration; at the same time, it has robustness in low-bitrate environments with poor network quality or sudden changes in communication conditions to cope with various actual communication scenarios.

[0003] Existing speech coding and decoding technologies are mainly divided into two categories: speech coding and decoding technologies based on signal processing, and coding and decoding methods based on deep learning. However, they have the following defects: Many traditional codecs can only encode normally under a fixed bit rate or frequency band range, and cannot dynamically adjust the bit rate according to the actual communication situation. For example, the encoding bit rate of the G.729 encoder is fixed at 8Kbps, while the frequency band supported by G.722.1 is only a narrowband (0.05-7kHz frequency band), which is less versatile in actual application scenarios. Secondly, the audio signal theory involved in the speech coding method based on signal processing is extensive. For example, the overall system architecture of the AAC algorithm is complex and highly coupled, making it difficult to iteratively improve or port the code on its basis; at the same time, the encoding speech quality of traditional signal processing-based algorithms is often inferior to that of deep learning algorithms, and the encoding bit rate of traditional algorithms is generally higher. Finally, the codec models based on deep learning, such as SoundStream or Transformer-based speech codecs, have complex network structures and require a large amount of inference computing costs, with high computational complexity; at the same time, the algorithm interpretability and theoretical nature of codecs based on deep learning are always lacking. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention discloses a layered speech coding and decoding method based on MDCT coefficient residual vector quantization, which specifically includes the following steps:

[0005] Offline codebook training generation process (training process):

[0006] S1: Preprocess the audio data for training. The processed speech is subjected to MDCT transform frame by frame to extract MDCT coefficients. The extracted MDCT coefficients are grouped and segmented in the frequency band range of 0 to 48 kHz. The segmented sub-datasets are used as network training data.

[0007] S2: For each grouped MDCT data set, unsupervised learning is performed through the RVQ-VAE network to achieve feature extraction and compression dimension reduction of MDCT, and generate a multi-level quantization codebook of MDCT, which is applied to the encoding and decoding process; the offline codebook training generation process is independent of the encoding and decoding process, and it only needs to be performed once. For the sake of simplicity, in the present invention, the offline codebook training generation process will be referred to as the training process.

[0008] Coding process:

[0009] S3: The input speech passes through the pre-emphasis filter, and the signal is high-pass filtered to enhance the high-frequency components. After windowing the pre-emphasized speech, the MDCT coefficients are generated through a fast algorithm.

[0010] S4: After the calculated MDCT coefficients are grouped according to their frequency ranges; in the code book search function, the code book index is calculated based on the corresponding code book obtained through training according to the minimum mean square error criterion; according to the currently allocated coding bit rate, the bit allocation mode is determined; after the coding control bits and the code book index are compressed through Huffman coding, the code stream is transmitted to the channel.

[0011] Decoding process: S5: Perform Huffman decoding on the received code stream to obtain the coding control bits and the MDCT code book index of each frame; retrieve the MDCT vector in the MDCT code book through the code book index, and restore the original MDCT coefficients by frame; perform MDCT inverse transform on the original MDCT coefficients, and restore the original speech signal through de-emphasis filtering.

[0012] Furthermore, RVQ-VAE is used as a vector quantizer to perform feature learning on the input MDCT coefficients to achieve data compression and dimensionality reduction. The original S×N dimensional MDCT data set is mapped to a D×N dimensional vector codebook through unsupervised learning. Since each quantization will produce quantization residuals, RVQ-VAE continues to quantize these residuals and generate a new quantization codebook.

[0013] During the encoding process, when generating MDCT coefficients, the number of floating-point operations of the MDCT generation module is reduced by using the FFT-based MDCT fast algorithm, thereby reducing the computational complexity of the system.

[0014] The definition of MDCT is decomposed into odd and even numbers to obtain the discrete Fourier transform DFT form. The DFT expression is simplified by mathematical transformation, and the one-dimensional DFT is converted into a two-dimensional DFT standard form through the prime factor algorithm of the non-radix 2-FFT. The target MDCT coefficients are obtained by solving it through a fast algorithm.

[0015] Furthermore, during the encoding process, each group of MDCT coefficients is searched through the corresponding first-level vector codebook, and the quantized residual is searched again through the second-level vector codebook, and so on. The codebook search is based on the minimum mean square error as the criterion to minimize the error between the quantized MDCT and the original MDCT vector;

[0016] During the decoding process, each group of MDCT divided by frequency band obtained by Huffman decoding is overlapped and added to reconstruct the original MDCT according to the hierarchical quantization mode during encoding, and the encoded speech signal is restored through MDCT inverse transform and post-processing.

[0017] Due to the adoption of the above technical scheme, the present invention provides a layered speech coding and decoding method based on MDCT coefficient residual vector quantization. In this method, the coding parameters are mainly MDCT coefficients. Compared with the traditional algorithm, there is no need to perform a large number of complex calculations of coding parameters. At the same time, the MDCT coefficients can be calculated by a fast algorithm based on FFT. Therefore, the computational complexity of the system is significantly reduced compared with other speech codecs. In addition, the present invention enables the encoder to dynamically adjust the coding bit allocation according to the visual bit rate through the design of MDCT layered coding, so as to adapt to the actual communication environment and cope with sudden network conditions. At the same time, based on the MDCT grouping of the frequency band, the system supports sampling rates such as narrowband (8KHz), broadband (16KHz), ultra-wideband (32KHz), and full-band (48KHz), which is more flexible and practical than other fixed bit rates and sampling rate codecs. Finally, in the training process, the vector code book will be pre-trained by RVQ-VAE, which can be repeatedly called in encoding and decoding, rather than using a complex network model in the speech encoder for reasoning. In addition, the coding and decoding framework structure of the present invention is relatively simple and direct, and the overall computational complexity of the system is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1It is a functional structure diagram of the speech encoding and decoding method in the present invention;

[0020] Figure 2 Schematic diagram of the principle of MDCT codebook training by RVQ-VAE in the present invention;

[0021] Figure 3 is the frequency response curve of the pre-emphasis filter of the encoding part in the present invention;

[0022] Figure 4 It is a schematic diagram of the principle of hierarchical coding of MDCT coefficients in the coding process of the present invention;

[0023] Figure 5 A schematic diagram of a hierarchical allocation method of bits based on frequency band range in the present invention;

[0024] Figure 6 It is a schematic diagram of the principle of MDCT coefficient reconstruction in the decoding process of the present invention;

[0025] Figure 7 is the frequency response curve of the high-pass filter of the decoding part in the present invention;

[0026] Figure 8 is the signal spectrogram of the original test speech in the present invention;

[0027] Fig. 9 It is the signal spectrogram of the encoded and decoded speech in the present invention. DETAILED DESCRIPTION

[0028] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0029] like Figure 1 The layered speech coding and decoding method based on MDCT coefficient residual vector quantization shown in the figure mainly includes three processes: training, coding and decoding. Among them, the training process is independent of the core framework of the codec, and only needs to be performed once in advance, and does not need to be repeated in the coding and decoding process. The MDCT code book obtained through the training process will be repeatedly applied to the coding and decoding process. The present invention specifically includes the following steps:

[0030] Training process: S1: Preprocess the audio data for training. The processed speech is transformed frame by frame to extract the MDCT coefficients. The extracted MDCT coefficients are grouped and cut in the frequency band range of 0 to 48KHz, and the segmented sub-datasets are used as network training data. The specific method is as follows:

[0031] S11: The audio processing tool Sph2pipe is used to convert the audio format of the WSJ0 dataset with a sampling rate of 16KHz. After converting the SPH format to wav format, the audio is merged with the VCTK dataset with a sampling rate of 48KHz through the open source processing tool FFmpeg.

[0032] The processed speech will be transformed frame by frame through MDCT to extract a set of MDCT coefficients (also called MDCT vectors) in the speech frame. The calculation expression of MDCT transformation is as follows:

[0033]

[0034] Assume that the dimension of the MDCT data set is S×N, where S is the number of MDCT vectors contained in the data set, which is numerically equal to the number of speech frames contained in the speech; N is the dimension of the MDCT vector, and its calculation expression is:

[0035] N=T·fs

[0036] Wherein, T is the speech frame length, and fs is the speech sampling rate. Therefore, for a training set speech data with a length of 10 hours and a sampling rate of 48KHz, after extracting the MDCT coefficients through MDCT transformation (in the present invention, the frame length is set to 20ms), an MDCT data set with a dimension of 1.8M×960 will be generated.

[0037] S12: The extracted MDCT data set is divided into groups in the frequency band range of 0 to 8 kHz and saved as a new MDCT sub-data set as training data for subsequent RVQ-VAE. The MDCT coefficients are divided as follows: Figure 2 shown.

[0038] According to the principles of psychoacoustics, the main frequencies of human speech are concentrated in the range of 1 to 3 kHz, among which the fundamental frequency is concentrated in the low-frequency band of 80 to 255 Hz, the resonance peak frequency is concentrated in the range of 0.2 to 1 kHz, and the human ear is more sensitive to speech in the medium and low frequency bands of 0.2 to 3.4 kHz, that is, the human ear's perception of audio is nonlinear.

[0039] Based on this principle, the present invention refers to the frequency band grouping method of the speech Bark scale, and non-uniformly divides the input speech signal according to the frequency range, and divides 0-24KHz (the full-band speech sampling rate is 48KHz, and according to the Nyquist sampling theorem, the folding frequency 24KHz is the highest frequency component of the signal) into 16 sub-bands. The specific frequency band division scheme is shown in the following table:

[0040]

[0041] Since MDCT coefficients are calculated based on the frequency domain, grouping and segmenting the frequency bands is equivalent to grouping and segmenting the MDCT coefficients, so that RVQ-VAE can better capture the local frequency domain characteristics of MDCT, while improving the quantization accuracy and reducing the training time and video memory usage. Figure 2 As shown, after grouping, the MDCT data of dimension S×N is divided into k groups of MDCT sub-data, and the following expression is satisfied:

[0042]

[0043] S2: Figure 2 As shown, RVQ-VAE is used to perform residual vector quantization on each group of MDCT data sets; at the same time, unsupervised learning is used to realize feature extraction and dimension reduction of MDCT, and multi-level quantization codebooks of each group of MDCT are obtained, and the trained codebooks of each level are applied to the encoding process. The specific method is as follows:

[0044] S21: The RVQ-VAE structure contains multiple consecutive vector quantization (VQ) modules. It requantizes the residual after each level of quantization and repeats this process to minimize the quantization error and improve the quantization accuracy. This structure is called residual vector quantization RVQ, and its calculation formula is as follows:

[0045]

[0046] Wherein, for the i-th level vector quantizer VecQuant, x is the MDCT vector to be quantized, y is the quantized MDCT vector, and residual is the residual of x and y.

[0047] Specifically for each level of MDCT data vector quantization, such as Figure 2 As shown, it will be realized through the VQ-VAE structure. VQ-VAE is a vector quantized variational autoencoder. Variational Auto-Encoder (VAE) is a network model based on probability generation, and its training adopts unsupervised learning. Data dimensionality reduction and compression are achieved by learning implicit features and distribution laws from high-dimensional data. The core of VQ-VAE is to introduce a discrete Codebook component into the network, which is used to extract the features of the input sequence and encode it into a discrete vector form. The present invention uses this feature of VQ-VAE to obtain the MDCT codebook by training the codebook in VQ-VAE.

[0048] S22: The core of this step is the training of the codebook, which is unsupervised learning through gradient descent. Ideally, the encoder will output a codebook vector that is close to the learned codebook vector. This step has a two-way problem: learning a codebook vector that aligns with the encoder output, and the encoder output that aligns with the codebook vector.

[0049] like Figure 2 As shown, the input MDCT data is assumed to be a two-dimensional vector x with a dimension of S×N3. In VQ-VAE, it first passes through the encoder encoder to obtain a continuous coding vector z:

[0050] z=encoder(x)

[0051] Among them, z is a d×N3 two-dimensional vector. In addition, VQ-VAE will maintain an Embedding layer (also known as an embedding layer or encoding table), whose expression is expressed as:

[0052] E=[e1,e2,...,e K ]

[0053] Among them, all e i are all d×N3 vectors. Then VQ-VAE uses the nearest neighbor search algorithm to map the vector z to one of these K two-dimensional vectors:

[0054] z→e k ,

[0055] The encoding table vector corresponding to z is recorded as z q , that is, the final MDCT target encoding result, corresponding to the generated MDCT codebook, the dimension is also d×N3. Finally, the MDCT reconstruction is realized through the decoder:

[0056]

[0057] In VQ-VAE training, z is used for reconstruction q , z q The construction process includes argmin. The latter is not differentiable with respect to the encoder, that is, there is no gradient and the encoder cannot be updated through the back propagation algorithm. Therefore, the stop gradient algorithm is used in VQ-VAE to ensure that the target variable z is used during forward propagation. q Perform the operation and use the variable z to ensure the back propagation update of the gradient during back propagation. The loss function of the training is calculated as:

[0058]

[0059] Where L eis the loss function, and sg is the stop gradient function. The value in sg remains unchanged during the forward propagation process; only when backpropagating, the value of sg is set to 0 and no gradient calculation is performed on it. Its calculation expression is:

[0060]

[0061] During the training process of S22, it is expected that the generated z q can fit z as much as possible. Therefore, this part is added to the total loss function. Therefore, the calculation expression of the loss function for MDCT codebook generation by VQ-VAE is as follows:

[0062]

[0063] During the training process of the present invention, the weights β = 0.75 and γ = 0.25 are set as the final results.

[0064] As Figure 2 shown, through the combination of VQ-VAE and RVQ, several new MDCT data are obtained by training the MDCT sub-dataset, and the data dimension is d×N (satisfying d << S). The present invention calls it the MDCT codebook, and these codebooks will be widely used in the encoding and decoding parts of the present invention.

[0065] Encoding process: S3: The input speech passes through a pre-emphasis filter to perform high-pass filtering on the signal to enhance the high-frequency components. After windowing the pre-emphasized speech, MDCT coefficients are generated through a fast algorithm. The specific method is as follows:

[0066] S31: The input speech will pass through a pre-emphasis filter to perform high-pass filtering on the signal to enhance the high-frequency components. This filter is a first-order FIR high-pass filter, which is used to enhance the high-frequency components with generally low energy in the signal, compensate for the loss of high-frequency components caused by coding quantization errors, and at the same time improve the quantization and search accuracy of the MDCT in the high-frequency band.

[0067] The amplitude-frequency response curve and phase-frequency response curve of the FIR high-pass filter are as Figure 3 shown, and its transfer function H(z) is as follows:

[0068] H(z) = 1 - 0.875z -1

[0069] S32: Perform windowing on the pre-emphasized speech, and generate MDCT coefficients through the MDCT fast algorithm based on FFT. Its detailed operations and calculation steps are as follows:

[0070] The pre-emphasized speech will first be processed by the window function. The purpose of signal windowing is to ensure the short-term stability of the speech and reduce the influence of signal spectrum leakage and fence effect. In this step, the window function uses the Hamming window, and its expression is:

[0071]

[0072] Calculate the MDCT coefficients frame by frame for the windowed speech. MDCT stands for improved discrete cosine transform, which is a derivative algorithm of discrete cosine transform. Compared with the traditional discrete cosine transform, it has fewer window boundary artifacts. The direct calculation of MDCT coefficients is relatively complex and time-consuming, and needs to be quickly solved by a fast algorithm based on FFT. The specific transformation and calculation steps are as follows:

[0073] First, the definition of MDCT transform is as follows:

[0074]

[0075] Transforming the above formula can be decomposed into the sum of odd and even parts, where the even part X(2k) is as follows:

[0076]

[0077] The cosine part of the above formula is expanded through trigonometric functions and angle formulas, and the above formula is rewritten as shown below:

[0078]

[0079] In the above formula, let

[0080] Similarly, the odd part X(N / 2-2k-1) is equal to X(N / 2+2k), and can be rewritten as follows through transformation:

[0081]

[0082] The odd part X(N / 2+2k) and the even part X(2k) of the MDCT transform are respectively regarded as the real part and the imaginary part of the complex number, and the following formula can be obtained by simplification:

[0083]

[0084] in,

[0085] According to the definition of the inverse discrete Fourier transform, the following formula can be obtained:

[0086]

[0087] The odd part of the MDCT transform X(N / 2-2k-1) corresponds to the real part of Z(k), while the even part of the MDCT transform X(2K) corresponds to the imaginary part of Z(k). Z(k) is the discrete Fourier transform of z(n), which can be quickly solved by the fast algorithm of FFT.

[0088] In the present invention, the fast algorithm solution after MDCT decomposition to DCT is based on the coprime factor algorithm of non-radix 2-FFT. The specific implementation method is as follows:

[0089] The principle of the coprime factor algorithm can be summarized as follows: transform the one-dimensional DFT of N=N1N2 points, reorganize the dimensions to represent the two-dimensional DFT transform of N1×N2, and N1 and N2 must be coprime. Calculate based on the transformation and decomposition of the DFT of size N1×N2: You can continue to recursively call the coprime factor algorithm program, or other fast Fourier transform algorithms (such as the radix 2-FFT algorithm and the Cooley-Tukey fast Fourier algorithm).

[0090] In the present invention, 48KHz input speech is taken as an example; since the speech frame length is 20ms, the input sequence length N0=960 for each frame MDCT transformation is transformed into a one-dimensional DFT with N=240 points through the formula. The one-dimensional DFT with 240 points is converted into a two-dimensional DFT with N1=15 and N2=16 through the coprime factor algorithm. Therefore, the original MDCT calculation can be realized by calculating 15 16-point DFTs and 16 15-point DFTs.

[0091] For a 15-point DFT, it can be converted to a 16-point DFT by padding with zeros. The 16-point DFT can be directly solved quickly using the radix 2-FFT fast Fourier transform algorithm.

[0092] S4: After the calculated MDCT coefficients are grouped according to their frequency ranges; in the codebook search function, the codebook search function is used to calculate the codebook index based on the corresponding codebook obtained through training according to the minimum mean square error criterion; according to the currently allocated coding rate, the bit allocation mode is determined; after the coding control bit and the codebook index are compressed through Huffman coding, the code stream is transmitted to the channel. The specific method is as follows:

[0093] S41: According to the designed codebook search function and based on the minimum mean square error criterion, the MDCT codebook index of each layer of each group of MDCT coefficients is calculated. The detailed operation steps are as follows:

[0094] like Figure 4As shown, a group of MDCT coefficients (referred to as MDCT vectors in the present invention) calculated based on the current speech frame will first be divided into 16 groups by frequency band in the same way as the training process. After that, each group of MDCT coefficients will pass through the designed code book search function. The role of the code book search function is to calculate the similarity of two groups of vectors one by one in the code book of the corresponding frequency band obtained in the training process for the current MDCT vector, and find the group of MDCT vectors closest to the MDCT vector of the current frame as the quantization result.

[0095] The similarity between the codebook vector and the MDCT vector to be quantized is measured by the mean square error L2 Loss, which is calculated as follows:

[0096]

[0097] Among them, x i With y i are the i-th value of the MDCT vector to be quantized and the codebook vector respectively, and n is the dimension of the MDCT vector.

[0098] Similar to the RVQ hierarchical vector quantization in the training process, the MDCT codebook search in the encoding process is also performed through a hierarchical structure. The input MDCT vector is searched through the codebook, and the closest set of prediction vectors is found in the codebook. The two sets of vectors are subtracted to calculate the residual, and the residual vector (referred to as MDCT quantization residual in the present invention) is passed into the second-level codebook search function with a similar structure, and the residual is quantized at the second level to generate a new codebook index; then the second-level residual vector is passed into the third-level codebook search function with a similar structure for third-level quantization, and the operation is repeated by analogy.

[0099] In fact, after each MDCT search, the element value of the quantized residual vector will be significantly reduced, and the quantization accuracy will tend to saturation; the marginal benefit of performing codebook search and quantization again will also decrease. Therefore, the present invention retains at most four levels of MDCT quantization codebooks during the training process, and performs four corresponding codebook searches during the encoding stage.

[0100] Through the codebook search function, an integer index value will be obtained, which corresponds to the subscript position of a set of vectors (or codewords) closest to the input MDCT vector retrieved from the codebook in the current codebook. This integer is called the codebook index.

[0101] exist Figure 4In the code, the core layer represents the original MDCT vector, and the index is calculated by searching the first-level codebook; the enhancement layer represents the quantized residual of each level of MDCT, and the index is calculated by searching the second-level, third-level, etc. secondary codebooks. During the encoding process, the core layer index will be transmitted to ensure that the most basic features of the speech can be transmitted, while the enhancement layer index will be determined based on the bit rate conditions and whether to transmit it will be determined through subsequent judgment.

[0102] S42: According to the currently allocated rated bit rate, determine the allocation mode of the coding bits through the bit allocation table, and the specific operation steps are as follows:

[0103] In the present invention, the compression coding of MDCT coefficients is implemented based on MDCT index; each group of MDCT vectors can be mapped to a unique index integer value through the codebook, and the value of each codebook index also corresponds to the MDCT vector in the codebook. Therefore, for each layer of codebook search, the more bits are allocated, the larger the value range of the codebook index, and the higher the quantization accuracy of the MDCT coefficients. The bit allocation pattern in the encoding process is as follows: Figure 5 shown.

[0104] Consistent with the frequency band division method during training, the full frequency band is non-uniformly divided into 16 sub-bands, and the maximum number of quantization layers and bit allocation of each layer of different frequency bands are different. In general, the number of quantization layers and the number of allocated bits in the low and medium frequency bands are more, so as to obtain more accurate low and medium frequency quantization accuracy and improve the voice quality of the low and medium frequency bands after decoding.

[0105] First, determine the frequency band grouping according to the speech sampling rate, and calculate the total number of available bits per frame according to the bit rate quota; traverse from top to bottom and from left to right in the order of frequency band range and quantization layer number. Figure 5 until there are no remaining bits available for further allocation.

[0106] Specifically, take a wideband speech with a sampling rate of 16KHz as an example, and assume that the initial bit rate quota is 12.8Kbps. According to the Nyquist sampling theorem, its highest frequency component is 8KHz. Figure 5 It can be seen that the frequency band of 0-8KHz will be divided into 9 sub-bands, and these sub-bands contain 2 or more enhancement layers. Since the speech frame length is 20ms, in order to transmit all the enhancement layer bits, the minimum bit rate required is:

[0107] Bit_Rate total =(1000ms / 20ms)*(96+70+66+20)bps=12.6kbps

[0108] Since the encoding consumption bit rate 12.6Kbps < 12.8Kbps threshold, all enhancement layer bits, including the core layer bits that are transmitted unconditionally, can be transmitted. The bit allocation at this time is as follows Figure 5 As shown in Mode 1.

[0109] Assume that due to the deterioration or sudden change of the communication network quality, the bit rate quota drops to 8Kbps (<12.6Kbps). At this time, the total available bits per frame are 8000 / 50=160 bits, which is not enough to transmit all the enhancement layer bits. Therefore, it is necessary to traverse the bit allocation table from top to bottom and from left to right. The remaining available bits will continue to subtract the currently consumed bits until the number of available bits is 0 or no further allocation can be made. The bit allocation at this time is as follows Figure 5 As shown in Mode 2.

[0110] In this way, the bit allocation mode is dynamically adjusted according to the actual communication conditions and rated bit rate conditions, and the enhancement layer bits are selectively transmitted to cope with special communication conditions and low bit rate scenarios, thereby enhancing the robustness and flexibility of the system.

[0111] S43: Losslessly compress the MDCT codebook index bits and coding control bits frame by frame through Huffman coding, and transmit the compressed code stream to the channel. The specific operation steps are as follows:

[0112] The bit allocation mode has been determined. Since the audio signal is grouped into 16 sub-bands at most, the total number of enhancement layer bits is at most 16+12+4=32 bits. Therefore, 4 bits can be used to represent the maximum frequency band range, and 5 bits can be used to represent the total number of enhancement layers used. These additional control information are collectively referred to as coding control bits, totaling 9 bits.

[0113] The coding control bits and the MDCT codebook index of each frame will be recompressed at the bit domain level through Huffman coding. Huffman coding is a commonly used entropy coding compression algorithm that can achieve lossless compression of data. The binary code stream compressed by Huffman coding will be transmitted to the communication channel.

[0114] Decoding process: S5: Perform Huffman decoding on the received code stream to obtain the coding control bit and the MDCT code book index of each frame; retrieve the MDCT vector in the MDCT code book through the code book index, and restore the MDCT coefficients by frame; perform MDCT inverse transform on the MDCT coefficients, and restore the original speech signal through de-emphasis filtering. The specific method is as follows:

[0115] S51: The received code stream is first Huffman decoded, and the original MDCT code book index and coding control bits are restored frame by frame. Huffman decoding is the inverse process of its encoding. The decoded code stream structure is: 9 bits of coding control bits and several MDCT code book indexes arranged in frame order. According to the coding control bits, the audio sampling rate and the bit allocation mode of the encoding process can be determined, and then the correct code book index sequence can be obtained for the subsequent MDCT coefficient recovery process.

[0116] S52: Extract the MDCT coefficients at the corresponding index position in the codebook. According to the grouping and layering mode of the MDCT coefficients, reconstruct the original MDCT coefficients frame by frame, and obtain the speech signal through the MDCT inverse transform. The specific MDCT reconstruction process is as follows: Figure 6 shown.

[0117] During the decoding process, since the code book index is known, the corresponding code book can be directly looked up according to the index to retrieve the corresponding MDCT vector.

[0118] At the decoding end, the quantized MDCT vector is superimposed with the possible residuals (corresponding to the enhancement layer codebook index) to restore the MDCT vector of the corresponding frequency band. Finally, these MDCT vectors are combined according to the grouping of the frequency band to reconstruct the MDCT coefficients.

[0119] IMDCT (Inverse MDCT, IMDCT) is the inverse process of MDCT transformation. According to the duality of discrete Fourier transform, MDCT inverse transform can also be calculated by FFT fast algorithm. Therefore, the present invention will not repeat the formula transformation derivation and process. The calculation formula of IMDCT is as follows:

[0120]

[0121] The restored MDCT coefficients are subjected to IMDCT frame by frame, the frequency domain MDCT is converted back into a time domain speech signal, and the restored speech frames are overlapped and added to achieve the reconstruction of the speech signal.

[0122] S53: De-emphasize and high-pass filter the voice signal to restore the original voice signal.

[0123] De-emphasis is the inverse process of pre-emphasis in the encoding process, which is used to enhance the low-frequency components of the signal to offset the impact of pre-emphasis and restore the frequency characteristics of the original signal. The transfer function of the de-emphasis filter G(z) is the inverse of the pre-emphasis filter H(z), and its expression is as follows:

[0124]

[0125] The de-emphasized speech signal will pass through a third-order IIR high-pass filter to suppress the 50Hz power frequency noise and the infrasound frequency band below 20Hz. The stopband cutoff frequency is designed to be 50Hz, the passband cutoff frequency is 140Hz, the stopband attenuation is 20dB, and the filter transfer function F(z) is expressed as follows:

[0126]

[0127] The amplitude-frequency and phase-frequency response curves of the high-pass filter are as follows: Figure 7 As shown in the figure, the frequency components below 50 Hz are effectively suppressed after high-pass filtering.

[0128] In order to test and verify the effectiveness of the present invention, the codec of the present invention is integrated into the Opus codec and replaces the CELT codec, and a test experiment is carried out in the hybrid mode of Opus. The signal spectrogram of the original uncoded speech and the spectrogram of the coded speech (one set is selected from the test data) are shown as follows: Figure 8 and Fig. 9 As shown in the spectrogram, it can be seen that the encoder can effectively restore the full-band frequency components of the original speech.

[0129] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A layered speech coding and decoding method based on MDCT coefficient residual vector quantization, characterized in that include: Preprocess the speech data used for training, extract MDCT coefficients from the preprocessed speech data frame by frame through MDCT transformation, and obtain MDCT data sets by non-uniform grouping and segmenting the MDCT coefficients according to the voice band range, and perform residual vector quantization on each group of MDCT data sets using residual vector quantization variational autoencoder to train and generate a multi-level quantization codebook of MDCT; The high-frequency components of the speech to be encoded are enhanced by a pre-emphasis filter, and the pre-emphasized speech signal is windowed to generate MDCT coefficients based on the FFT algorithm. The obtained MDCT coefficients are grouped according to the frequency band range. In the codebook search function, the codebook index is obtained based on the minimum mean square error criterion according to the multi-level quantization codebook, and the coding control bit is generated according to the bit allocation mode determined according to the code rate. The codebook index and the coding control bit are compressed by Huffman coding and then the code stream is transmitted to the channel; The code stream in the channel is read and the coding control bits and MDCT code book index are obtained through Huffman decoding. The speech sampling rate and bit allocation mode are determined by the coding control bits. The MDCT vector is retrieved based on the MDCT code book index, and the MDCT inverse transform is performed on it. Then, the original speech signal is restored through de-emphasis filtering.

2. A layered speech coding and decoding method based on MDCT coefficient residual vector quantization according to claim 1, characterized in that: When generating the multi-level vector codebook: RVQ-VAE is used as a vector quantizer, and unsupervised learning is used for training, and a multi-level quantization codebook of MDCT is generated by extracting high-dimensional features and performing compression and dimensionality reduction processing on the input MDCT data.

3. A layered speech coding and decoding method based on MDCT coefficient residual vector quantization according to claim 1, characterized in that: When generating MDCT coefficients: perform parity decomposition and mathematical transformation on the definition of MDCT transform, convert it into discrete Fourier transform form, reduce the number of floating-point operations of MDCT generation module based on FFT algorithm, combine the coprime factor algorithm of non-radix 2-FFT with radix 2-FFT algorithm, convert one-dimensional DFT into two-dimensional DFT standard form, and obtain MDCT coefficients through FFT algorithm.

4. A layered speech coding and decoding method based on MDCT coefficient residual vector quantization according to claim 1, characterized in that: For each group of MDCT coefficients, a codebook search is performed in the codebook search function through the corresponding first-level MDCT codebook, and the quantized residual is searched again through the second-level MDCT codebook, and so on; During the codebook search process, the minimum mean square error is used as the criterion to find a group of MDCT vectors closest to the original MDCT vector in the codebook, and record their index positions in the codebook, which are the codebook indexes.

5. A layered speech coding and decoding method based on MDCT coefficient residual vector quantization according to claim 1, characterized in that: When determining the bit allocation mode: determine the frequency band grouping according to the speech sampling rate, calculate the number of available bits for encoding the current speech frame according to the bit rate set during encoding, traverse the bit allocation table in the order of frequency band range and number of quantization layers, and allocate bits in sequence until there are no remaining bits.

6. The layered speech coding and decoding method based on MDCT coefficient residual vector quantization according to claim 1, characterized in that: When restoring the original speech signal: the corresponding MDCT coefficients are retrieved based on the codebook index, the original MDCT is reconstructed according to the grouping and layering mode of the encoding process, and the original speech signal is restored through MDCT inverse transform and de-emphasis filtering.

Images