Voice signal compression method and device, equipment and medium

Abstract

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice signal compression method, device, equipment and medium, which comprises the following steps: executing Fourier transform on an initial voice signal to extract an amplitude spectrum and a phase spectrum, respectively processing the amplitude spectrum and the phase spectrum to generate features, and splicing the features to obtain a compressed voice signal; the method comprises the following steps: carrying out quantization by adopting a residual vectorization mode to generate a compressed feature vector, executing entropy coding to obtain a compressed code stream, recovering the compressed feature vector at a decoding side, reconstructing a spliced feature through residual reverse quantization, enhancing feature expression through up-sampling operation, and restoring a voice signal through inverse Fourier transform. According to the method, a dual-path processing structure of amplitude features and phase features is constructed, feature compression is realized in combination with residual vectorization and entropy coding, inverse quantization and up-sampling enhancement are executed in a decoding stage, a spectrum energy structure and phase continuity are effectively reserved, and reconstruction precision and fidelity of voice signals are improved under the condition of low bit rate.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and particularly to a method, device, equipment and storage medium for compressing speech signals. Background Art

[0002] In the development process of audio compression technology, traditional methods mainly adopt two processing methods in the time domain and the frequency domain. Time-domain methods such as pulse code modulation directly act on the waveform level of audio signals during sampling and quantization. Although it is simple to implement and has low latency, its ability to express complex structural features and local changes is limited, and it is difficult to effectively adapt to speech segments with frequent dynamic changes. In contrast, frequency-domain methods map the signal to the frequency domain for compression processing through discrete cosine transform or Fourier transform, etc., which generally improves the compression ratio and signal structure modeling ability. However, in low-bitrate application scenarios, such frequency-domain methods still face a series of practical problems.

[0003] First of all, existing frequency-domain methods mostly perform time-frequency analysis based on the short-time Fourier transform (STFT) with a fixed length. When dealing with non-stationary audio signals, especially speech and sound effect segments containing a large number of transient changes or multi-level rhythm structures, the time-frequency resolution trade-off problem brought by the fixed window length is particularly prominent. A fixed wide window is beneficial to improving frequency accuracy, but it is difficult to capture high-dynamic time changes, while a narrow window has a fast time response but sacrifices the ability to express the frequency-domain energy distribution. This limitation results in that existing solutions are difficult to balance the preservation of time and frequency information under low-bitrate conditions.

[0004] Secondly, during the frequency-domain compression process, phase information is often simplified or even directly discarded. This approach has less impact on the sound quality at higher bitrates, but it will significantly reduce the fidelity of the reconstructed audio at low bitrates, especially the reconstruction effect of signals with strong instantaneousness is not good. For example, in a voice interaction system or beep generation, subtle phase errors may cause fluctuations in the time-domain waveform, thereby affecting the intelligibility and naturalness of speech.

[0005] In the field of fintech business, with the wide application of speech recognition and speech interaction systems in scenarios such as intelligent customer service, identity verification, and speech compliance analysis, there is a requirement for speech compression solutions to still have high fidelity under low-bitrate conditions. The time-frequency resolution problem and insufficient reconstruction fidelity of traditional methods directly restrict the speech interaction efficiency and user experience, especially in multi-round conversations and peak processing scenarios.

[0006] In the field of medical and health services, the usage scenarios of voice data are becoming increasingly rich, such as voice medical record entry, doctor-patient conversation recording, and remote consultation interaction, etc. There are relatively high requirements for voice reconstruction quality, and it is often accompanied by low-bandwidth environments or local storage limitations. Against this background, in existing compression methods, the phase details and key frequency band characteristics of audio cannot be effectively retained, which may lead to problems such as auditory distortion, semantic ambiguity, or content breakage during the decoding and playback of voice data, affecting the accuracy of subsequent automated analysis and medical decision-making. Summary of the Invention

[0007] The main object of the present invention is to provide a voice signal compression method, device, equipment, and storage medium, aiming to solve the technical problem that in the prior art, the key features of amplitude and phase information cannot be retained simultaneously under low bitrate conditions, resulting in insufficient fidelity of the reconstructed audio.

[0008] To achieve the above object, the present invention provides a voice signal compression method, including:

[0009] Perform a Fourier transform on the original voice signal to extract the amplitude spectrum and phase spectrum from the original voice signal;

[0010] Process the amplitude spectrum through a first processing path to generate processed amplitude features;

[0011] Process the phase spectrum through a second processing path to generate processed phase features;

[0012] Concatenate the processed amplitude features and the processed phase features to generate a concatenated feature;

[0013] Perform quantization processing on the concatenated feature through a residual vector quantization layer to generate a compressed feature vector;

[0014] Perform entropy coding on the compressed feature vector to generate a compressed code stream;

[0015] Recover the compressed feature vector from the compressed code stream through entropy decoding;

[0016] Perform inverse quantization processing on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed concatenated feature;

[0017] Perform upsampling processing on the reconstructed concatenated feature to generate an upsampled feature;

[0018] Perform an inverse Fourier transform on the upsampled feature to generate a reconstructed voice signal.

[0019] Furthermore, to achieve the above object, the present invention provides a voice signal compression device, including:

[0020] A spectrum extraction module, configured to perform Fourier transform on an initial voice signal and extract an amplitude spectrum and a phase spectrum from the initial voice signal;

[0021] An amplitude feature processing module, configured to process the amplitude spectrum through a first processing path to generate a processed amplitude feature;

[0022] A phase feature processing module, configured to process the phase spectrum through a second processing path to generate a processed phase feature;

[0023] A feature fusion module, configured to splice the processed amplitude feature and the processed phase feature to generate a spliced feature;

[0024] A vector quantization module, configured to perform quantization processing on the spliced feature through a residual vector quantization layer to generate a compressed feature vector;

[0025] An entropy encoding module, configured to perform entropy encoding on the compressed feature vector to generate a compressed code stream;

[0026] An entropy decoding module, configured to recover the compressed feature vector from the compressed code stream through entropy decoding;

[0027] A vector inverse quantization module, configured to perform inverse quantization processing on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed spliced feature;

[0028] An upsampling enhancement module, configured to perform upsampling processing on the reconstructed spliced feature to generate an upsampled feature;

[0029] A spectrum restoration module, configured to perform inverse Fourier transform on the upsampled feature to generate a reconstructed voice signal.

[0030] Furthermore, to achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a voice signal compression program stored in the memory and executable on the processor. When the voice signal compression program is executed by the processor, the steps of the voice signal compression method as described above are implemented.

[0031] Furthermore, to achieve the above object, the present invention further provides a computer-readable storage medium, on which a voice signal compression program is stored. When the voice signal compression program is executed by a processor, the steps of the voice signal compression method as described above are implemented.

[0032] Beneficial effects: The present invention relates to the field of speech processing technology and can be applied to business scenarios such as fintech and healthcare. A speech signal compression method, device, equipment, and medium are disclosed, including: performing Fourier transform on the initial speech signal to extract the amplitude spectrum and phase spectrum, generating features by separately processing the amplitude spectrum and phase spectrum and splicing them, using residual vector quantization to generate a compressed feature vector, performing entropy coding to obtain a compressed code stream, restoring the compressed feature vector on the decoding side and reconstructing the spliced features through residual inverse quantization, further enhancing the feature expression through upsampling operation, and finally restoring the speech signal through inverse Fourier transform. By constructing a dual-path processing structure for amplitude features and phase features, combining residual vector quantization and entropy coding to achieve feature compression, and synchronously performing inverse quantization and upsampling enhancement in the decoding stage, the spectral energy structure and phase continuity are effectively retained, and the reconstruction accuracy and fidelity of the speech signal are improved under low bit rate conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below in conjunction with the drawings and embodiments. In the drawings:

[0034] Figure 1 is a schematic diagram of an application environment of the speech signal compression method in an embodiment of the present invention;

[0035] Figure 2 is a schematic flowchart of an embodiment of the speech signal compression method of the present invention;

[0036] Figure 3 is a schematic diagram of the functional modules of a preferred embodiment of the speech signal compression device of the present invention;

[0037] Figure 4 is a schematic diagram of the structure of a computer device in an embodiment of the present invention;

[0038] Figure 5 is another schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] The speech signal compression method provided by the embodiments of the present invention can be applied in, for example Figure 1In the application environment, the client communicates with the server through the network. The server can perform Fourier transform on the initial voice signal through the client to extract the amplitude spectrum and phase spectrum, generate features by separately processing the amplitude spectrum and phase spectrum and splicing them, perform quantization using the residual vector quantization method to generate a compressed feature vector, perform entropy coding to obtain a compressed bitstream, restore the compressed feature vector on the decoding side and reconstruct the spliced features through residual inverse quantization, further enhance the feature expression through upsampling operation, and finally restore the voice signal through inverse Fourier transform. By constructing a dual-path processing structure for amplitude features and phase features, and combining residual vector quantization and entropy coding to achieve feature compression, and synchronously performing inverse quantization and upsampling enhancement in the decoding stage, the present invention effectively retains the spectral energy structure and phase continuity, and improves the reconstruction accuracy and fidelity of the voice signal under low bitrate conditions. Among them, the client can be, but is not limited to, various personal computers, laptop computers, smartphones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present invention will be described in detail below through specific embodiments.

[0041] Please refer to Figure 2 , Figure 2 FIG. is a schematic flowchart of an embodiment of the voice signal compression method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0042] As Figure 2 shown, the voice signal compression method proposed by the present invention includes the following steps:

[0043] S10, perform Fourier transform on the initial voice signal, and extract the amplitude spectrum and phase spectrum from the initial voice signal;

[0044] In this embodiment, to perform Fourier transform on the initial voice signal to extract the amplitude spectrum and phase spectrum, the original voice signal needs to be segmented in the time domain first. The voice signal is a non-stationary time series signal, and its energy and frequency characteristics show dynamic changes in different time periods. Therefore, it needs to be divided into a series of frames. Usually, each frame has a fixed length, and the length can be adjusted according to the sampling rate and computing load in the actual application environment. The framed voice signal constitutes local segments within multiple time windows, enabling subsequent frequency domain conversion operations to have time domain positioning capabilities.

[0045] Each frame of the signal undergoes a windowing operation to improve the accuracy of frequency-domain analysis. The purpose of windowing is to reduce the spectral leakage effect caused by signal truncation between frames. Window functions usually adopt functions with smooth edges, such as Hamming window, Hanning window or Blackman window. The specific window function selection is adjusted according to the trade-off requirements for the main lobe width and sidelobe energy suppression in the target scenario. After applying the window function, the envelope of each frame of the signal can be modulated into a signal form with more stable boundaries, thereby enhancing the spectral estimation accuracy in the Fourier transform process.

[0046] Performing the Fourier transform is a process of mapping the time-domain signal to the frequency domain, mainly using the fast Fourier transform algorithm to reduce the computational complexity. The windowed frame signal is regarded as a complex sequence, and the complex spectrum of each frame can be obtained through FFT. The complex value of each frequency point in the spectrum result contains a real part and an imaginary part. Its modulus value reflects the amplitude component of the frequency point, and the argument reflects the phase information. For further processing, it is necessary to extract the amplitude information and phase information from the complex spectrum respectively. The amplitude information is obtained by calculating the complex modulus value of each frequency point, which expresses the energy distribution of different frequency components and is the main factor determining the sound quality and structure of the speech signal. The phase information is calculated by the arctangent function of the complex number and is used to represent the time alignment relationship between frequency components, which has an important impact on the clarity and naturalness in the speech signal.

[0047] The extracted phase information may have jumps or multi-period overlaps in its original state. Especially in scenarios where high frame-to-frame phase continuity is required, to avoid breaks or waveform distortion during signal reconstruction, it is necessary to perform an unwrapping operation on the phase data. Phase unwrapping is a method to eliminate 2π jumps. By comparing the phase difference between the current frame and the previous frame and adjusting it to a continuous change trend, subsequent reconstruction operations can be based on a smooth phase sequence, improving the integrity and continuity of signal restoration.

[0048] Finally, in order to adapt to the unified requirements for tensor format and numerical precision in the subsequent model structure, the phase data needs to be quantized. The quantization process includes mapping the floating-point phase data into a set numerical precision interval, usually a fixed-point integer or a floating-point format with limited bit width, and maintaining the approximate representation ability of the original value within a certain range. This step not only compresses the data scale, but also reduces the storage requirements for subsequent model operations and lowers the overall computational load.

[0049] In one specific implementation, a voice signal with an original sampling rate of 16,000 Hz is divided into frames with a length of 512 sampling points, and the frame shift is 256 sampling points. A Hamming window is applied to each frame for windowing. The windowed signal is processed using a one-dimensional fast Fourier transform to generate a complex spectrum matrix composed of the frame dimension and the frequency point dimension. The modulus value of each frequency point is calculated to generate an amplitude spectrum matrix, and at the same time, the initial phase spectrum is extracted by calculating the arctangent angle of each complex frequency point. Then, a continuity analysis is performed on the phase change between frames, and a 2π compensation is performed on the positions with mutations to expand them into a continuous phase trajectory. The expanded phase values are linearly scaled and quantized using a 16-bit fixed-point floating-point format, and finally, the amplitude spectrum matrix and the quantized phase matrix are output.

[0050] The frame length and the window function type can also be parameterized, and different window functions (such as Blackman window) are used to adapt to the spectral stability requirements in different noise environments. In the scenario where the remote transmission bandwidth is limited, a shorter frame length can be adopted to improve the time-domain resolution, and at the same time, the quantization bit width can be increased accordingly to offset the decrease in the frequency-domain resolution caused by the shorter window length.

[0051] An adaptive window length strategy can also be adopted based on the energy distribution of the spectrum. A shorter window length is used in the voice activation segment to enhance the time-domain capture ability, and a longer window length is used in the silent segment to improve the frequency resolution, thereby dynamically adjusting the allocation method between computing resources and analysis capabilities and enhancing the adaptability of the compression system to changes in voice signals.

[0052] Example illustration: In the scenario of medical and health data processing, when a user describes a remote medical record or conducts a consultation record through voice input, the system needs to compress and store long-duration audio to ensure the restoration of semantic integrity and audio fluency during subsequent decoding. By obtaining complete amplitude spectrum and phase spectrum information through the above Fourier transform method, the accurate restoration of term pronunciation and stress rhythm in the reconstructed voice can be ensured, which is beneficial for doctors to judge the clinical features hidden in the language content.

[0053] In the voice quality inspection and voice interaction services in the financial field, such as customer hotline voice records or pre-processing of voice recognition, it is necessary to compress the voice data of user conversations and upload it to the central server for review or recognition in real time. In the case of bandwidth limitations or insufficient computing power at the edge side, this frequency-domain feature extraction method can effectively compress the amount of data required for voice signals, while retaining the complete capture of high-frequency details such as the speaking speed, intonation, and emotional changes of the speaker, enhancing the performance of the system in semantic interpretation and compliance analysis.

[0054] In this embodiment, by performing frame segmentation, windowing, fast Fourier transform, magnitude calculation, phase unwrapping, and quantization processing on the speech signal, the original non-stationary time-domain signal can be converted into structured frequency-domain amplitude spectrum and phase spectrum data, and while the compression ratio is limited, the frequency-domain energy expression and time continuity are taken into account, thereby providing a high-quality input feature representation basis for subsequent feature modeling and compression coding.

[0055] S20, process the amplitude spectrum through the first processing path to generate a processed amplitude feature;

[0056] In this embodiment, processing the amplitude spectrum through the first processing path to generate a processed amplitude feature is a key link to solve the problems of insufficient energy distribution modeling and inflexible feature expression granularity in traditional frequency-domain compression methods. The starting point of the processing is the amplitude spectrum, which reflects the energy intensity change of the speech signal at different frequencies and has the characteristics of strong structure and strong continuity. However, it also has problems such as fuzzy local peaks and unclear boundary transitions. To more effectively extract the semantic information and constituent features therein, it is necessary to introduce a deep learning structure to construct the processing path.

[0057] The first processing path includes multiple cascaded feature transformation modules. Its first link is a convolutional neural network structure, whose main function is to extract the feature changes of the amplitude spectrum between local frequency bands and across time frames. The convolutional network can include multiple groups of convolutional kernels, setting different receptive field scales, so as to simultaneously capture short-time details (such as plosives) and long-time modulation structures (such as vowel energy envelopes). The convolutional operation is essentially a filtering process with parameters, which can re-weight the spectrum without losing the spatial structure.

[0058] The intermediate amplitude feature after convolution output usually shows a non-standard distribution, and there may be problems such as mean drift and scale instability. Therefore, the next step is to perform normalization processing through layer normalization operation. Layer normalization is performed along the channel dimension, which can keep the structure in the time and frequency directions intact, and at the same time scale and centralize the activation responses between different channels, which helps to improve the consistency of the network in processing multi-frame speech data.

[0059] Subsequently, the normalized amplitude feature is fed into the forward layer, which is generally a feedforward neural network, including a set of fully connected layers and non-linear activation functions (such as ReLU or GELU), used to introduce non-linear expression ability and improve the network's modeling ability for non-linear speech structures (such as formant distribution). The activation function compresses the low-response region and highlights the significant features, thereby enhancing the recognition of specific frequency bands in the amplitude spectrum.

[0060] Finally, to match the input specifications of the subsequent compression module and further reduce the feature dimension and information redundancy, a downsampling operation is applied. Downsampling can be accomplished through strided convolution or pooling, and its essential function is to reduce the spatial dimension of the feature map while retaining the main energy information. This step generates the final processed magnitude features, which have a composite feature representation form with semantic concentration, high-dimensional compression, and spectral structure integrity.

[0061] One implementation is to use a three-layer one-dimensional convolutional module to perform feature extraction on the magnitude spectrum. The convolutional kernel sizes are three, five, and seven respectively, and the number of channels increases layer by layer to enhance the expression ability. Layer normalization is implemented using mean and variance normalization on the extracted feature tensor, and a forward layer module is constructed with the GELU activation function to perform non-linear mapping on the context features of each frequency point. A set of convolutional operations with a stride of two is added to the last layer to perform downsampling on the frequency dimension, reducing the feature dimension to half of the original, and generating the processed magnitude features.

[0062] Lightweight convolutions can also be used to replace ordinary convolutions. For example, depthwise separable convolutional networks can be adopted to improve the operation efficiency of the model in an embedded environment and reduce the computational complexity in voice edge computing applications. For high-dynamic voice signals, an attention mechanism can also be introduced and embedded in the convolutional block to achieve weighted highlighting of important frequency bands in the magnitude spectrum and enhance the selectivity of feature expression.

[0063] Example illustration: In the medical and health scenario, the voice medical records dictated by doctors usually contain a large number of medical terms and intonation mutation points, and there will be a mixed area of local high-energy peaks and weak background energy in the magnitude spectrum. Through this processing path, the features of vowel envelopes and plosive boundaries can be accurately extracted, improving the recognition accuracy and storage compression efficiency of the subsequent model for medical record voices.

[0064] In the financial field, such as in the multi-round voice interaction records of call centers, there are a large number of pauses and speech rate fluctuations in the pronunciation of customers. The processing path can extract these detailed frequency changes from the magnitude spectrum, which helps to construct high-fidelity reconstructed voices subsequently, ensuring the accurate capture of keywords and tones by the quality inspection system, and improving the structural restoration ability and compliance analysis basis of financial voice data.

[0065] Through the first processing path in this embodiment, not only the multi-level and multi-time-scale frequency features of the magnitude spectrum are extracted, but also the stability and expression intensity of the feature distribution are improved using the normalization and activation mechanisms. The finally output processed magnitude features are more suitable for the efficient representation requirements of the subsequent compression path, enhancing the overall model's ability to model the voice quality structure under limited compression ratios.

[0066] S30, process the phase spectrum through a second processing path to generate processed phase features;

[0067] In this embodiment, processing the phase spectrum through the second processing path and generating the processed phase features is a key technical measure to improve the reconstruction fidelity of speech signals, mainly solving the problems of weak phase information modeling ability and significant reconstruction distortion in traditional audio compression methods. Compared with the clear energy structure in the amplitude spectrum, the phase spectrum has a complex structure and no obvious rules, but it plays an irreplaceable role in retaining speech details and maintaining the integrity of the speech waveform, especially in restoring high-frequency edges and maintaining speech smoothness. Therefore, this path needs to have high robustness and context modeling ability to extract valuable structural features from the input phase information with strong uncertainty.

[0068] The processing flow of the phase spectrum has structural symmetry with that of the amplitude spectrum, but it also has an independent modeling logic. In the first step, a multi-scale convolutional network is also used to perform structural modeling on the original phase spectrum, simultaneously perceiving local phase changes (such as short-time speech breakpoints) and long-term phase trends (such as formant transitions) through convolutional kernels of different scales, and realizing the compression and extraction of the multi-layer structure of phase data. The convolutional operation performs channel expansion and region aggregation on the original phase spectrum, generating a set of intermediate phase features expressing phase continuity, jump boundaries, and relative differences.

[0069] Subsequently, to enhance the stability of the neural network during training and cope with the dynamic phase changes of different speech segments, layer normalization is performed on the intermediate phase features. Since the phase itself does not have a unified absolute value distribution and its statistical properties highly depend on the characteristics of speech segments, the layer normalization operation can dynamically balance the numerical scales in each channel, improving the robustness of the model when processing noisy or discontinuous phase spectrum inputs.

[0070] Next, the normalized phase features are fed into the forward neural network layer, and a higher-order feature space is constructed by introducing non-linear mapping, enabling the model to more effectively distinguish different types of phase behaviors, such as the boundary differences between continuous sliding phase trajectories and sudden jump-type phases. The non-linear function usually uses a smooth type, such as GELU, which has stronger gradient transfer ability and fine-grained activation boundaries.

[0071] Finally, a downsampling mechanism is used to further compress the data dimension while retaining the structural trends across frequency points and time frames. This downsampling can be achieved through strided convolution or pooling mechanisms, and its main role is to reduce redundant calculations and feature redundancy, strengthen the modeling ability of the global phase structure, and output the final processed phase features for splicing or joint modeling with the features in the amplitude path in subsequent stages.

[0072] One implementation is to use a two-channel three-layer one-dimensional convolutional network to perform a modeling operation on the phase spectrum. The first layer uses a small receptive field to capture local phase micro-changes, the second layer uses a large receptive field to construct the relative phase structure between frequency bands, and the third layer fuses multi-channel information to form stable intermediate phase features. This feature tensor is fed into a standard layer normalization module for mean and standard deviation normalization in the channel dimension, and then into a fully connected forward layer with an activation function to complete non-linear expansion. Finally, a cross-channel max pooling operation is performed to compress the feature dimension to half of the original input, obtaining the processed phase features adapted to the structure of the subsequent encoding module. It is also possible to adopt a joint modeling method of self-attention mechanism and convolutional layer to perform adaptive weighted learning between regions of the phase spectrum, calculate the similarity between inter-frame phase jumps and frequency point structures through a dot-product attention module, and improve the modeling performance for unstructured jump phase inputs. At the same time, a residual structure can be introduced to ensure that semantic information is not lost during the deep feature transformation process and improve training stability. It is also possible to add learnable position encoding information within a specific time window based on the periodicity and syntactic features existing in the speech scene, guiding the model to establish cross-segment phase mapping capabilities in the frequency domain structure, thereby improving the structural understanding of speech sentence boundaries and pronunciation rhythms.

[0073] Example illustration: In the medical and health application scenario, such as in speech medical record recognition or speech consultation, complex speech behaviors such as slight pauses, sentence breaks, and speech rate changes often occur in the doctor's oral narration. These changes are often only reflected in the phase trajectory rather than the amplitude structure. By constructing stable phase features through the second processing path, the speech coding system's ability to capture subtle intonation changes in medical record speech can be improved, thereby enhancing the complete restoration of medical terms and language semantics.

[0074] In the financial scenario, the speech interaction content between customers and intelligent customer service contains a large number of short-term pauses, repetitions, and rapid error correction behaviors, which will cause frequent phase breaks. By processing the phase spectrum, the compression system can maintain the coherence and naturalness of the speech response while retaining the data compression rate, which helps the financial speech recognition system to build high-precision speech quality inspection and playback functions, supporting multiple application goals such as risk control, compliance, and service evaluation.

[0075] In this embodiment, by constructing a second processing path to independently model the phase spectrum, it makes up for the deficiency of ignoring phase details in the traditional frequency domain coding system and enhances the modeling ability for complex phase changes. During the process of reconstructing the speech signal, the phase features generated by this path provide reference information with stable structure and clear details for inverse frequency domain restoration, effectively improving the smoothness, restoration degree, and naturalness during the speech recovery process.

[0076] S40, splice the processed amplitude features and the processed phase features to generate a spliced feature;

[0077] In this embodiment, splicing the processed amplitude feature and the processed phase feature to generate a spliced feature is an important operation for unifying and fusing the speech representations extracted by the two feature paths. The purpose is to establish a joint feature space that can simultaneously contain speech energy information and temporal structure information, thereby providing a structurally complete and informationally collaborative input basis for the subsequent compression and reconstruction processes.

[0078] The amplitude feature usually represents the energy distribution of the speech signal in each frequency dimension and is a main component of the speech structure, while the phase feature reflects information such as the microstructure, boundaries, and resonance migration of the signal in the time dimension. Since the two have different source paths and have different statistical characteristics after independent modeling, alignment and structural normalization are required before splicing to ensure the matchability of the features in the spatial dimension.

[0079] The splicing process usually unfolds along the channel dimension or the feature dimension. If the amplitude feature and the phase feature have the same time frame and frequency point structure, they can be directly spliced along the channel dimension to form a new multi-channel tensor. If there are differences in the number of time frames or frequency resolution between the two, upsampling or feature reorganization of the short-side data needs to be performed through linear interpolation or projection mapping methods to make their sizes consistent before splicing. The splicing method can be a simple concatenation or can be formed through weighted fusion to form an attention-weighted splicing structure, where the weights can be learned from multiple indicators such as amplitude energy, phase jump frequency, and historical reconstruction error.

[0080] This spliced feature simultaneously carries the amplitude hierarchical structure and the phase migration structure of the speech signal, and has stronger context awareness ability and compression adaptability. In the subsequent residual vector quantization stage, this feature can more effectively match with the trained codebook structure, improving the quantization residual modeling accuracy.

[0081] One implementation method is to represent the processed amplitude feature and phase feature as tensors with sizes of T×F×C1 and T×F×C2 respectively, where T is the number of frames, F is the number of frequency points, and C1 and C2 are the number of channels respectively. On the premise of ensuring that the number of frames and the number of frequency points are the same, the two tensors are directly spliced along the channel dimension to generate a spliced feature tensor with a dimension of T×F×(C1 + C2). The splicing operation is completed through Tensor Concatenate, and the sequential structure of the spliced tensor is kept consistent.

[0082] It is also possible to first use a convolutional channel mapping layer to uniformly map the amplitude feature and the phase feature to the same number of channels, and then use a weighted fusion module, where the fusion weights are dynamically adjusted through an attention mechanism to enhance the amplitude dominance in the region where the speech signal energy is concentrated and the phase dominance in the region sensitive to phase changes, realizing adaptive fusion splicing.

[0083] It is also possible to perform a feature similarity estimation before splicing. Through the cosine similarity or mutual information metric, local rotation adjustment is performed on the phase features to maximize the structural alignment degree between the phase features and the amplitude features, thereby reducing the feature conflict in the subsequent module modeling.

[0084] In this embodiment, by constructing the spliced features, the voice structure information extracted from the two heterogeneous paths is effectively fused to form a unified multi-dimensional spectrum representation, which has strong frequency-domain energy expression and time-continuous structure expression capabilities. This operation enhances the adaptability of the subsequent compressed representation to the global structure of the voice signal, helps the voice reconstruction process after compression to achieve a more natural, smooth and structurally continuous voice restoration effect, and shows higher restoration fidelity under low bitrate conditions.

[0085] S50, quantize the spliced features through a residual vector quantization layer to generate a compressed feature vector;

[0086] In this embodiment, the purpose of quantizing the spliced features through the residual vector quantization layer to generate a compressed feature vector is to compress the high-dimensional joint representation features into a sparse and encodable discrete representation form with the least information loss, significantly reducing data redundancy, so as to meet the low bitrate requirements of subsequent coding and transmission, while retaining the main structural information required for voice reconstruction.

[0087] The spliced features are a multi-dimensional representation tensor obtained by fusing the processed amplitude features and phase features in the channel dimension or feature dimension, and contain various voice structure information such as frequency energy distribution and phase migration trajectory. Directly performing quantization processing on this high-dimensional feature is likely to introduce serious structural distortion. Therefore, a residual vector quantization mechanism is introduced as a transition to improve the flexibility and adaptability of quantization.

[0088] This process first initializes a trainable codebook matrix containing multiple codebook vectors. Each codebook vector represents a semantic clustering center or a compressed subspace basis element. The codebook matrix is continuously updated through the training process to enable it to cover the distribution areas of diverse voice features. Subsequently, the distance between each local segment in the spliced features and all codebook vectors is calculated (such as Euclidean distance or cosine distance) to determine the nearest codebook vector, and the initial residual vector between the two is calculated. This residual reflects the difference part of the original feature that cannot be fully expressed by the codebook and is an important compensation source for compression accuracy.

[0089] To reduce the quantization error and improve the reconstruction accuracy, iterative optimization operations are performed on the initial residual vector. This optimization process can adjust the quantization direction based on backpropagation of gradients, or introduce a transformation network to encode the residuals. Finally, two outputs are generated based on the optimized residual vector: one is the quantization index vector, which represents the codebook number corresponding to each feature segment and is used for subsequent encoding and compression; the other is the residual parameter, which is used to assist in recovering and compensating for the quantization error during the reconstruction process. The two together constitute the compressed feature vector, which is the encoding output of the entire audio compression path.

[0090] One implementation method is to divide the concatenated features into fixed-length feature blocks. Each feature block is matched with all codebook vectors in the codebook matrix to determine the index of the minimum distance as the quantization index. The residual is the remaining part after subtracting the corresponding codebook vector from the original feature block. This residual is input into a lightweight neural network for non-linear mapping to output the residual parameter.

[0091] A residual iteration mechanism can also be adopted. Through multiple rounds of quantization structures, a residual layer-by-layer optimization is constructed. The residual output in each round of quantization is used for the next round of quantization until the error converges or reaches the maximum number of layers. The index and residual parameter output by each layer are concatenated to form the final compressed feature vector. This method is suitable for scenarios with high-precision and low-bitrate speech requirements.

[0092] A learnable weight gating mechanism can also be introduced to perform region-selective quantization processing on the concatenated features in different frequency bands, so that important semantic regions (such as intonation peaks and high-frequency components) can obtain higher-precision residual modeling, while less resources are used for encoding in low-weight regions, thereby achieving a better trade-off between speech subjective fidelity and compression efficiency.

[0093] In this embodiment, the concatenated features are quantized through the residual vector quantization layer, realizing the conversion from a high-dimensional structured feature vector to a low-dimensional sparse representation, significantly compressing the storage and transmission space required by the speech data while maintaining the structural integrity. The quantized output index vector and residual parameter retain the main structure and detailed information in the speech signal while ensuring the lightweight of the model, providing high-quality input for subsequent encoder optimization and decoder reconstruction. This mechanism exhibits stronger speech restoration ability under low-bitrate conditions, avoiding the speech blurring and distortion problems brought by traditional mean quantization schemes.

[0094] S60, perform entropy coding on the compressed feature vector to generate a compressed bitstream;

[0095] In this embodiment, performing entropy coding on the compressed feature vector to generate a compressed bitstream belongs to the steps of implementing the final bitrate control and effective coding encapsulation. The goal of this process is to achieve compressed storage with the minimum information entropy based on the statistical distribution characteristics of the compressed feature vector without introducing additional information loss, so that the compression result meets the efficiency and consistency requirements in data transmission, storage, streaming media scheduling, and other aspects.

[0096] The compressed feature vector is composed of the discrete index output by the residual vector quantization layer and the continuous residual parameters. It contains both discrete symbol distributions and floating-point features of continuous variables. To ensure coding adaptability, a distribution-aware coding strategy needs to be adopted, where the two types of components are processed separately and then integrated. This process can be decomposed into three core steps:

[0097] First, perform adaptive arithmetic coding on the quantization index vector in the compressed feature vector. The quantization index vector is based on the clustering result of the codebook, with a high symbol reuse rate and a clear probability distribution. Adaptive arithmetic coding dynamically estimates the probability of each index symbol according to the historical context, and then assigns shorter codewords to high-frequency symbols and longer codewords to low-frequency symbols, thereby achieving the optimal entropy coding compression efficiency. This coding method has a higher ability to approximate the ideal entropy lower bound compared to Huffman coding and can adapt to the dynamic distribution changes of online speech streams.

[0098] Second, perform floating-point quantization and context modeling coding on the residual parameters in the compressed feature vector. Since the residual parameters are usually continuous values, direct coding is not resolvable. Therefore, it is first necessary to convert them into discrete level representations through a non-uniform floating-point quantization strategy. This quantization strategy can adopt exponential magnitude mapping to retain high fidelity under small amplitude changes. The quantized residual symbol sequence is then processed by a context modeling encoder to establish the probability dependence relationship between it and space, frequency, and historical states, thereby further compressing the data volume. The context model can be implemented using a probability table, a Markov state diagram, or a lightweight deep network modeling module.

[0099] Finally, encapsulate and splice the index bitstream and the residual bitstream to form a compressed bitstream with a consistent structure and decodability. The splicing process needs to follow a preset format, such as using a structure header information with frame length, index interval, and residual segment boundaries to ensure that the decoding end can correctly separate the two bitstream segments. Redundancy check flags or cyclic redundancy check (CRC) information can also be embedded during the splicing process to enhance data integrity.

[0100] In one implementation, the quantization index vector is regarded as a one-dimensional symbol sequence. An adaptive probability model is constructed, and arithmetic coding based on a CDF table is used for symbol-by-symbol coding, with the output being a compressed index bitstream. The residual parameters are first subjected to log-scale segmented quantization, and each segment is encoded with an identifier. Then, the residual values within each segment are encoded using a context-aware probability model, and each context window is constructed based on the frequency distribution of the previous n frames.

[0101] An end-to-end compression model based on deep learning can also be adopted. The neural network is used to learn the joint probability distribution of the entire compressed feature vector, and then the entire segment of the feature vector is compressed using a Neural Arithmetic Encoder. This method is applicable to the compression of scenario speech blocks and performs batch transmission at the speech segment granularity.

[0102] For the speech communication scenario with low bitrate requirements, a bitrate control mechanism can also be introduced in the residual parameter encoding. By clipping the quantization level or selecting the residual parameter channels, the size of the compressed bitstream is dynamically adjusted to adapt to the transmission requirements of edge devices or low-speed networks.

[0103] In this embodiment, by performing entropy coding on the compressed feature vector, the index and residual information representing the speech features are effectively converted into a highly compressed and structured bitstream, significantly reducing the redundancy and number of bits of the encoding output while ensuring the data reconstruction accuracy. This process adapts to the distribution characteristics of different speech contents, improving the robustness and encoding efficiency of the overall compression system under conditions such as speech complexity, content variation, and uneven transmission bandwidth, and constructing a flexible and controllable speech compression output mechanism.

[0104] S70, recovering the compressed feature vector from the compressed bitstream through entropy decoding;

[0105] In this embodiment, the compressed feature vector is recovered from the compressed bitstream through entropy decoding. The role is to restore the structured bitstream compressed during the encoding process to the basic elements constituting the speech feature representation, providing accurate input for subsequent inverse quantization and speech reconstruction. The compressed bitstream usually contains two logical components: the index bitstream and the residual bitstream, corresponding to the quantization index vector and the residual parameters respectively. Since these two types of information have different coding models and compression strategies, the decoding process needs to perform segmented parsing according to the preset encapsulation structure and restore the original feature representation.

[0106] The first step of the decoding process is to parse the overall structure of the compressed bitstream. This is usually based on the format identification field or boundary indication information pre-embedded in the bitstream, used to separate the index segment and the residual segment. Format parsing can be completed through mechanisms such as fixed-length field reading, frame header field analysis, or synchronization flag detection, ensuring the correctness and stability of the bitstream partitioning.

[0107] Subsequently, an adaptive arithmetic decoding operation is performed on the index bitstream. The arithmetic decoder reversely derives the encoding process of the symbol sequence according to a predefined cumulative probability table or a dynamic context model, and restores the original quantized index vector bit by bit. Since the encoding process uses an adaptive probability model, the decoder needs to synchronously construct an equivalent probability estimator to ensure that the probability states on both the encoding and decoding sides are consistent and avoid index distortion. Each item of this index vector corresponds to the codebook number selected during the residual vectorization process and is the core reference for constructing the basic feature vector.

[0108] Next, the residual bitstream is decoded. The residual bitstream is usually a discrete representation after floating-point quantization. The decoder needs to first parse its quantization level, bit-width distribution, or encoding method (such as the context modeling structure), and then map the codewords back to continuous residual values. For the residual part compressed using context modeling, the decoder also needs to reconstruct the probability dependence structure between the residuals and predict the residual symbols according to the current context to gradually restore the complete residual sequence.

[0109] After the decoding of the two segments of information is completed, they are reassembled into a compressed feature vector, whose structure is consistent with that in the encoding stage, including a discrete index vector and a set of continuous residual parameters. The reconstruction of the compressed feature vector marks the closure of the first stage of the compression-decompression process and lays the foundation for inverse vectorization and final speech generation.

[0110] In one implementation, the compressed bitstream indicates the boundary lengths of the index segment and the residual segment through the frame header field. The decoder loads the index bitstream and the residual bitstream respectively according to the frame header information. The index bitstream uses static table-driven arithmetic decoding, and each codeword corresponds to an index number; the residual bitstream is restored through floating-point value look-up tables or a reconstruction network.

[0111] In a complex speech environment, a context state synchronization mechanism can be introduced to maintain the prediction context of the previous frame during the decoding process and improve the ability to restore the residuals of low-frequency or mutated speech segments. For multi-channel or multi-speech-rate speech scenarios, the speech rate flag and channel information fields embedded in the bitstream can also be decoded to generate branch compressed feature vectors for different decoding paths for subsequent adaptive decoding.

[0112] A Transformer-based decoder structure can also be adopted, inputting the entire compressed bitstream into a pre-trained semantic decoding model. The model automatically completes segmentation, reconstructs the index and residuals, and combines them into a structured feature vector, which is suitable for scenarios of joint processing of multi-task speech reconstruction and semantic recognition.

[0113] In this embodiment, the compressed feature vector is recovered from the compressed bitstream through entropy decoding, realizing the lossless restoration of the information in the encoding stage, and providing a feature input with complete structure and clear semantics for subsequent speech inverse quantization and reconstruction. This process decouples the index and residual information while preserving the feature accuracy, improves the controllability and model compatibility of the decoding process, reduces the restoration error during speech transmission, and enhances the stability of the decoding system in complex speech data scenarios.

[0114] S80, perform inverse quantization processing on the recovered compressed feature vector through the residual inverse quantization layer to generate a reconstructed spliced feature;

[0115] In this embodiment, inverse quantization processing is performed on the recovered compressed feature vector through the residual inverse quantization layer to generate a reconstructed spliced feature. This operation depends on the recovered quantization index vector and the recovered residual parameters contained in the recovered compressed feature vector, aiming to reversely reconstruct a high-dimensional feature representation that conforms to the structure of the spliced feature in the encoding stage, for supporting the subsequent upsampling and spectral restoration of the speech signal.

[0116] The inverse quantization processing first performs a structure analysis on the recovered compressed feature vector to separate the recovered quantization index vector and the recovered residual parameters contained therein. The quantization index vector consists of multiple discrete integers, and each value represents the vector number in the codebook matrix selected in the encoding stage. The residual parameter is a floating-point or quantized floating-point vector, used to compensate for the loss of feature information during the quantization of the index vector.

[0117] According to the recovered quantization index vector, find the corresponding basic vector group in the pre-defined or training-generated accessible codebook matrix and assemble it into a basic feature vector. This codebook matrix is generally a look-up table structure composed of multiple groups of center vectors, which can be generated through adaptive training, and its structure should be consistent with the codebook used in the residual vector quantization layer in the compression stage.

[0118] The found basic feature vector and the recovered residual parameters are subjected to an element-wise addition operation to generate an initial reconstructed feature. This addition operation is essentially to restore the quantized high-dimensional feature content during the compression process, making the initial reconstructed feature closer to the original spliced feature in terms of structure and semantics.

[0119] In order to eliminate the feature space offset caused by quantization and residual accumulation, an anti-projection correction operation is performed on the basis of the initial reconstructed feature. This operation can be implemented through a lightweight neural network, such as a multi-layer perceptron network, for adjusting the projection relationship of the feature vector in a specific dimension to meet the requirements of aligning the frequency band and smoothing the energy distribution before upsampling, and finally obtaining a reconstructed spliced feature with stable structure.

[0120] In one implementation, the restored quantized index vector is input in the form of a two-dimensional tensor. The decoder extracts the corresponding codebook vector combination from the trained codebook matrix through a hash structure or a linear look-up table method, and splices and reconstructs the basic feature vectors according to the index dimension. The residual parameters are restored through displacement reconstruction and floating-point recovery to restore their detailed expressions before encoding.

[0121] A multi-codebook residual superposition structure can also be adopted, that is, multiple codebook indexes correspond to each feature dimension. After generating multiple codebook vectors, weight fusion is performed, and then superimposed with the residual parameters to support the sparse restoration of high-dimensional compressed features, which is suitable for low-bitrate speech scenarios in complex language environments.

[0122] In addition, the back-projection correction module can dynamically switch the correction sub-network according to different speech categories. For example, a speaker-specific feature mapping structure can be used to improve speech consistency, or a normalization layer can be integrated to scale the frequency-sensitive dimensions to prevent the reconstructed features from being distorted in the high-frequency band.

[0123] In this embodiment, the inverse reconstruction of the core features in the encoding compression stage is completed through the residual inverse quantization structure, realizing the structural restoration from the compressed representation to the time-frequency features. It not only retains the structural expression restored through the quantization index, but also integrates the detailed compensation brought by the residual parameters. At the same time, the projection correction enhances the consistency and semantic integrity of the features. This operation not only improves the fidelity of speech reconstruction, but also enhances the generalization ability of the model under complex input speech distributions.

[0124] S90, performing an upsampling process on the reconstructed and spliced features to generate upsampled features;

[0125] In this embodiment, performing an upsampling process on the reconstructed and spliced features to generate upsampled features is a key step in reconstructing low-resolution features into a high-resolution frequency-domain representation in the speech signal compression and decoding process. The core goal of this operation is to restore sufficient resolution of the original speech signal in both the time and frequency dimensions, providing a feature input with complete structure and rich details for the final inverse transform to restore the speech signal.

[0126] The reconstructed and spliced features are the feature representations formed after the compressed features are processed by the residual inverse quantization. Their spatial dimensions are usually downsampled and compressed, containing a lower frequency information distribution and lacking the ability to express high-frequency details. The upsampling process aims to perform a scale-up operation on the feature tensor in terms of feature dimensions, including expanding the number of frequency bands, filling the time-domain sampling gaps, and compensating and aligning the structure of the expanded features for high-frequency details.

[0127] In the implementation process, first, a depthwise separable convolutional network is used to perform multi-scale context-aware feature extraction on the reconstructed and spliced features, forming a multi-scale context feature representation. The depthwise separable convolutional structure can separate the spatial convolution and the channel convolution, capture the feature dimension and semantic features simultaneously while reducing the number of parameters, and can strengthen the non-local information fusion of the band patterns in different speech segments.

[0128] Next, the dimension of the multi-scale context features is expanded through transposed convolution operations, that is, upsampling is achieved. The transposed convolutional layer expands the length of the time step or frequency step of the original feature tensor to several times the original size by setting the reverse stride and padding value, so that the features are restored to the expected size in the time or frequency domain. Although the generated initial upsampled features restore the spatial structure, there may be problems such as discontinuous frequency bands, missing high frequencies, or boundary jumps.

[0129] To further compensate for the hollow features and frequency tomograms caused by structural interpolation, linear interpolation compensation between adjacent frequency bands can be performed on the initial upsampled features. This compensation method takes each frequency band as the center and performs linear regression filling through the numerical relationship between its adjacent upper and lower frequency bands to generate interpolated compensation features, thereby improving the continuity and smoothness of the upsampled features.

[0130] Subsequently, the interpolated compensation features are input into the gated attention mechanism structure to perform band alignment correction and high-frequency detail enhancement processing. The gated attention mechanism can apply different enhancement weights to different frequency band regions, especially enhancing the expression of speech elements such as high-frequency speech components, edge details, pauses, and bursts. The band alignment correction realizes the continuous connection of the spectral structure by adjusting the distribution of the energy centers of different frequency bands, thereby forming a frequency-domain smooth feature.

[0131] Finally, the frequency-domain smooth features and the original reconstructed and spliced features are fused through a residual connection method to enhance the context consistency and high-frequency semantic integrity of the reconstructed features, obtaining the final upsampled features. These features will be directly used for subsequent inverse transformation reduction operations and are the most critical time-frequency expression layer in the speech signal compression and reconstruction process.

[0132] In one implementation, the depthwise separable convolutional network adopts a structure that combines multiple scale receptive fields. Each convolutional kernel covers a different frequency band width, performs multi-scale convolutional stacking on the reconstructed and spliced features in one-dimensional or two-dimensional directions, and after outputting the multi-scale context features, expands them to the original spectral resolution in the frequency dimension through two transposed convolutional modules.

[0133] In another implementation, linear interpolation compensation can adopt a sliding window mechanism. For each frequency band, extract the data in its left and right neighborhoods, perform weighted averaging, and dynamically adjust the interpolation weights to adapt to the energy density of different speech segments. A dynamic interpolation function can also be introduced, such as a local convolutional network or an attention module based on frequency position, to achieve a higher-order spectrum compensation effect.

[0134] In the detail enhancement stage, the gated attention mechanism uses lightweight structures, such as SE channel attention or FFT attention, to map the frequency-domain weight information into a gating factor, and apply enhancement or suppression operations to different frequency bands in the interpolated compensation features. Reverse fusion adopts an identity connection or a 1×1 convolutional residual path to ensure that the spectral features do not lose the key energy distribution due to multiple transformations.

[0135] In this embodiment, through a multi-layer feature reconstruction mechanism, problems such as high-frequency detail loss, frequency band discontinuity, and structural distortion in traditional upsampling methods are overcome, improving the clarity, naturalness, and expression integrity of speech. At the same time, this structure has high modularity and adjustability, and can adapt to the resolution requirements and energy distribution characteristics in different speech scenarios.

[0136] S100, perform an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal.

[0137] In this embodiment, performing an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal aims to restore the high-dimensional features in the frequency domain space to a continuous and audible time-domain audio waveform, realizing the end-to-end reconstruction of the speech signal. The key to this step is to convert the upsampled feature after feature reconstruction and spectrum compensation into a sequence of sample points of the time-domain signal, ensuring the effectiveness of the restored speech signal in terms of semantic integrity, acoustic consistency, and auditory quality.

[0138] The upsampled feature is a frequency-domain feature matrix that combines the processed amplitude information and the restored phase information, and has undergone high-frequency detail compensation, spectrum smoothing processing, and context fusion. Its dimension and frequency distribution are close to the structure before the initial Fourier transform. Each frame of this feature tensor usually contains a composite representation of multiple frequency points, representing the energy distribution of the local speech signal in this frequency band.

[0139] Before performing the inverse Fourier transform, it is necessary to perform an adaptation conversion according to whether the input feature is in a complex form or compressed into an amplitude-phase structure. When the upsampled feature contains components of the amplitude spectrum and the phase spectrum, they should first be recombined into a complex spectrum, that is, through the transformation form from polar coordinates to rectangular coordinates, the amplitude value and the phase angle are jointly constructed into a complex representation for the mathematical calculation of the inverse transform.

[0140] The inverse Fourier transform process usually adopts the fast inverse Fourier transform (IFFT) algorithm, which processes frame by frame depending on the frame-level structure of the input features, and maps the spectral vector back to a sequence of sampling points of the time-domain signal. After the inverse transformation of the spectral data of each frame, a time-domain segment of a fixed length is generated, which corresponds to a continuous waveform segment in the original speech signal.

[0141] To achieve seamless splicing between frames and global reconstruction, it is necessary to introduce the overlap-add technique, that is, when performing the inverse transformation, use the same window function as in the encoding stage to weight the results of each frame, and perform weighted superposition on the overlapping regions of multiple frames according to the frame shift rule to restore a complete and continuous speech waveform.

[0142] Parameters such as the window function type, frame shift ratio, and splicing order in this process must be consistent with those in the initial Fourier transform stage to avoid distortion phenomena such as frequency leakage, boundary artifacts, and time jumps during the speech reconstruction process. The final output result is a complete reconstructed speech signal, which can be used for subsequent playback, recognition, or analysis.

[0143] In one implementation, the input upsampled feature is a combined complex spectrum. The inverse operation function of the FFT library is directly called frame by frame for inverse transformation. A fixed-length real waveform segment is generated for each frame, and then it is combined into the final speech signal through the frame shift superposition strategy.

[0144] In another implementation, if the upsampled feature is still in the amplitude-phase separation structure, it is necessary to first combine it into a complex vector using cosine-sine reconstruction, and then perform the inverse FFT operation. In addition, an improved frequency-domain restoration algorithm can also be used to perform further smoothing processing on the phase spectrum before the inverse transformation to reduce the influence of high-frequency jitter on the reconstruction result.

[0145] It is also possible to adopt the method of neural network fitting for the inverse Fourier process, and obtain a decoding model that maps from the frequency domain to the time domain through training, replacing the traditional FFT / IFFT functions in the edge computing scenario to improve the operation efficiency and stability.

[0146] Example description: In the financial intelligent customer service system, in order to ensure the transmission efficiency and storage compression quality of large-scale voice dialogue data, while maintaining the clarity and recognizability of the user's voice interaction content, the current system can be deployed in the edge node of the voice channel or the cloud voice engine. When the user dials the intelligent customer service through voice, the original voice signal collected by the system is first framed and windowed, and the amplitude spectrum and phase spectrum representing the speaker's voice frequency distribution are obtained after Fourier transform. In the amplitude spectrum path, the system uses a convolutional neural network to perform multi-scale perception of the amplitude spectrum, and uses normalization and forward layers to enhance the frequency band expression, and finally obtains compressible amplitude features through downsampling; in the phase spectrum path, the network extracts the phase change features with a similar structure. The two features are spliced to form a unified representation. Subsequently, the system discretizes and compresses the spliced features through the residual vectorization module. This operation uses the trained codebook matrix to perform matching and residual extraction on the feature space, and encodes the information into quantization indexes and residual parameters. This structure can effectively reduce storage and bandwidth overhead in scenarios with large amounts of financial data. In the downstream entropy coding module, the quantization index is arithmetically coded, and the residual parameters are bit-width compressed through the context model to form the final compressed bitstream for secure storage or network transmission. After receiving the bitstream, the decoding side system uses entropy decoding and residual inverse quantization layer to restore parameters and reconstruct splicing features. Next, through a multi-stage feature upsampling process, combined with the context structure extracted by separable convolution, interpolation compensation mechanism, and gated attention network, the detail features are enhanced to restore the speech resolution before compression. Finally, the system performs an inverse Fourier transform to restore the frequency domain reconstruction result to a high-quality time domain speech signal. Whether it is used for user identity verification (such as voiceprint), semantic recognition (such as command word extraction) or intonation analysis (for identifying anger, questioning, etc.), this solution can provide clear and restorable speech output at low bit rates, avoiding customer service risks such as "inaudible" and "missed recognition" caused by compression.

[0147] In the remote voice medical record and medical follow-up system, doctors record, compress and store the voice descriptions of patients and perform high-quality playback afterwards, which is of great value for identifying potential neurological and respiratory diseases. After deploying the current voice signal compression system, the voice signals input by doctors through mobile terminals or remote recording devices are decomposed into amplitude spectra and phase spectra representing the vocal cord vibration characteristics and the instantaneous dynamics of vocalization through Fourier transform. To avoid the compression loss of high-frequency details such as wheezing and coughing in chronic disease follow-up voices, both the amplitude and phase signals perform deep feature modeling through independent paths. The amplitude part focuses on the pronunciation intensity and intonation structure; the phase part retains the temporal continuity and the consonant change rules. In the compression stage, this method performs residual quantization processing on the spliced dual-channel features, effectively compressing the repetitive structures in long-time voice expressions into codebook indexes and difference expressions. By combining the statistical characteristics such as slow syllable changes and regular breathing intervals in the medical scenario, the residual vector coding model is optimized, making it still have a high retention ability for abnormal structures in disease voices (such as stuttering, pauses, nasal sounds). The compressed bitstream is efficiently encoded through arithmetic coding and context models and can be safely transmitted to the hospital data center or doctor terminals. The decoding stage retains the complete structure reconstruction process, including codebook inverse lookup, residual summation and spectral detail correction modules, which can improve the audibility of pathological features in frequency feature recovery. For example, for patients with Parkinson's disease or throat tumors, their voices have phenomena such as high-frequency mutilation and instantaneous breaks. Using the gated attention mechanism to achieve frequency band alignment and lesion feature enhancement in the upsampling stage helps doctors to identify and assist in algorithmic diagnosis. The finally generated voice waveform is highly similar to the original recording, enabling doctors to accurately capture the throat tension, breath stability and voice pause cycle of patients during the later re-listening process, thereby assisting in judging the progression of the disease.

[0148] In this embodiment, the frequency domain features are restored to the time domain waveform through inverse Fourier transform, which is the basis for realizing the complete reconstruction of voice signals. Through spectrum synthesis, frame-level inverse transform and inter-frame superposition, this processing step retains important parameters such as rhythm, pitch and prosody in the voice signal and controls the compression loss within a perceptible range, significantly improving the intelligibility and restoration quality of the voice.

[0149] The present invention relates to the field of speech processing technology and can be applied to business scenarios such as fintech and healthcare. It discloses a method, device, equipment and medium for compressing speech signals, including: performing Fourier transform on the initial speech signal to extract the amplitude spectrum and phase spectrum, generating features by separately processing the amplitude spectrum and phase spectrum and splicing them, using residual vector quantization to perform quantization to generate a compressed feature vector, performing entropy coding to obtain a compressed bitstream, restoring the compressed feature vector on the decoding side and reconstructing the spliced features through residual inverse quantization, further enhancing the feature expression through upsampling operation, and finally restoring the speech signal through inverse Fourier transform. By constructing a dual-path processing structure for amplitude features and phase features, combining residual vector quantization and entropy coding to achieve feature compression, and synchronously performing inverse quantization and upsampling enhancement in the decoding stage, the present invention effectively preserves the spectral energy structure and phase continuity, and improves the reconstruction accuracy and fidelity of speech signals under low bitrate conditions.

[0150] In one embodiment, the above step S10 includes:

[0151] S101, dividing the initial speech signal into framed signals of a preset length;

[0152] S102, performing windowing processing on each framed signal to generate a windowed signal;

[0153] S103, performing Fourier transform on the windowed signal to generate a complex spectrum;

[0154] S104, determining the modulus value of each frequency point in the complex spectrum to generate an amplitude spectrum matrix;

[0155] S105, determining the phase angle of each frequency point in the complex spectrum to generate an original phase spectrum;

[0156] S106, performing phase unwrapping operation on the original phase spectrum to eliminate phase jumps and generate an unwrapped phase spectrum;

[0157] S107, quantizing the unwrapped phase spectrum into a phase spectrum in a preset precision format.

[0158] In this embodiment, dividing the initial speech signal into framed signals of a preset length is to localize the overall non-stationary speech sequence, so that each frame segment can be considered a short-time stationary signal, thus meeting the prerequisite requirements of the Fourier transform for signal stationarity. The division process usually sets the frame length according to the sampling rate, desired time resolution and subsequent processing requirements. The frame length should be controlled within dozens of milliseconds to balance frequency accuracy and time response. The frame shift parameter determines the overlapping ratio between frames, improving the frequency domain resolution while ensuring time domain continuity.

[0159] Windowing each frame of the segmented signal to generate a windowed signal is to solve the spectral leakage problem caused by the truncated window in the Fourier transform. The windowing operation weights and attenuates the edges of each frame of the signal through a window function to make the transition smooth. Common window functions include the Hamming window, Hanning window, Blackman window, etc. When selecting a window function, the main lobe width and sidelobe attenuation ability need to be considered to control the impact of spectral spreading.

[0160] Performing a Fourier transform on the windowed signal to generate a complex spectrum is the core process of the entire frequency-domain conversion. In this process, each frame of the windowed signal is mapped to a set of complex frequency points, where each frequency point contains a real part and an imaginary part, representing the amplitude projection and phase shift information of the frequency component respectively. The Fourier transform usually adopts the fast Fourier transform algorithm to improve the processing efficiency.

[0161] Determining the modulus value of each frequency point in the complex spectrum to generate an amplitude spectrum matrix is the process of extracting the signal intensity characteristics from the spectrum. The amplitude spectrum is the modulus length of the complex spectrum, representing the energy size of each frequency component in the current frame, and forming a two-dimensional matrix in the frequency dimension. Each column of this matrix represents the spectral distribution at a frame moment, and each row represents the temporal energy trajectory of a certain frequency.

[0162] Determining the phase angle of each frequency point in the complex spectrum to generate an original phase spectrum is to record the time offset information of the frequency component. The phase spectrum is obtained by calculating the arctangent value of the complex frequency point. This spectrum retains the structural continuity and semantic rhythm of the speech signal and is an indispensable part of the signal reconstruction process. Since the phase is a periodic quantity, there may be discontinuous jumps in its numerical change.

[0163] Performing a phase unwrapping operation on the original phase spectrum to eliminate phase jumps and generate an unwrapped phase spectrum is to restore the true change trajectory of the phase between adjacent frames. The phase unwrapping eliminates the phase mutation problem caused by the periodic modulus limitation by judging whether the phase difference exceeds π and performing modulus compensation, thus ensuring the continuity of the phase sequence in the frame-level dimension.

[0164] Quantizing the unwrapped phase spectrum into a phase spectrum in a preset precision format is to control the storage and transmission overhead of the phase information while ensuring the reconstruction quality. The quantization format can adopt fixed-point, floating-point or mixed formats, such as compressing each phase value to 16 bits or 8 bits, and setting it flexibly according to the system transmission capacity and signal quality requirements. This step maps the phase spectrum to a structured matrix within the standard representation range on the basis of controllable precision for subsequent model use.

[0165] In one implementation, the frame length is set to 25 milliseconds, the frame shift is set to 10 milliseconds, and the Hamming window is used as the window function. Each frame of the signal is windowed and then subjected to a 1024-point fast Fourier transform. The resulting complex spectrum is used to calculate the amplitude spectrum matrix and the original phase spectrum. When calculating the phase angle, the four-quadrant arctangent function is used to ensure accurate direction, and phase unwrapping restores continuity by adding or subtracting 2π.

[0166] In another implementation, the phase unwrapping operation is performed by a neural network module. By learning the phase jump pattern, the phase repair strategy is automatically optimized, especially improving phase coherence in high-noise speech scenarios. In addition, the phase quantization format can be dynamically adjusted according to the task. An 8-bit fixed-point representation is used in speech recognition tasks, and it is upgraded to a 32-bit floating-point format in speech synthesis tasks to improve the sound quality.

[0167] A normalization module can also be introduced before the windowing operation to make each frame of the signal have a uniform energy distribution, which is beneficial to the consistent modeling of spectral features. In special scenarios, an asymmetric window function or a frequency-domain masking window can also be selected to optimize the high-frequency energy extraction effect.

[0168] Through the above steps, this embodiment realizes the comprehensive conversion from unstructured speech waveforms to structured frequency-domain features, and extracts the key information in the two dimensions of energy and phase respectively, providing an accurate and complete frequency representation for the compression and reconstruction processes. Through phase unwrapping and quantization processing, the problems of original phase jumps and storage overhead are solved, effectively improving the compression efficiency and the fidelity of subsequent reconstruction.

[0169] In one embodiment, the above step S20 includes:

[0170] S201, performing multi-scale feature extraction on the amplitude spectrum through a convolutional neural network block to generate intermediate amplitude features;

[0171] S202, performing layer normalization processing on the intermediate amplitude features to generate normalized amplitude features;

[0172] S203, performing a non-linear activation transformation on the normalized amplitude features through a forward layer to generate activated amplitude features;

[0173] S204, performing a downsampling operation on the activated amplitude features to generate processed amplitude features.

[0174] In this embodiment, performing multi-scale feature extraction on the magnitude spectrum through a convolutional neural network block means using a convolutional kernel structure with multiple receptive fields to simultaneously capture local frequency band details and global energy trends in the two-dimensional time-frequency space. This process is built on the basis of the two-dimensional matrix of the magnitude spectrum. The convolutional kernel can be designed with small-scale receptive fields in the frequency dimension to extract fine-grained energy textures, and cross-frame convolution in the time dimension to capture the pronunciation persistence and dynamic changes. The convolutional neural network block can include multiple stacked convolutional layers, activation functions, and normalization modules to enhance the perception ability of complex speech features. The convolutional kernel parameters can be obtained through pre-training or end-to-end training methods to adapt to the spectral change structures in different speech scenarios.

[0175] Performing layer normalization on the intermediate magnitude features mainly solves the problem of unstable training caused by inconsistent energy distributions between different speech frames. The layer normalization operation normalizes the feature means and variances of each channel to map the intermediate magnitude features to a unified distribution, which helps to accelerate the model convergence speed and improve the robustness of feature representation. This processing is applicable to all convolutional feature maps and is not affected by changes in frame length and frequency band width.

[0176] Performing a non-linear activation transformation on the normalized magnitude features through the forward layer aims to enhance the model's ability to model non-linear audio components. The forward layer can consist of a set of fully connected mappings, and activation functions such as GELU and Swish, which have smooth boundaries, are commonly used to enhance the response of detailed features. This module enables the normalized magnitude features to have the ability to distinguish categories and recognize speech structures, especially providing additional expression channels at positions such as the start and end of pronunciation and energy mutations.

[0177] Performing a downsampling operation on the activated magnitude features means compressing high-dimensional and redundant features into low-dimensional representations. This operation can be performed using methods such as strided convolution, max pooling, or average pooling, aiming to remove redundant information in the spectrum and retain the main feature structures of the speech. The magnitude features after downsampling retain the spectral energy distribution features in the compressed representation and serve as the basic input for subsequent concatenation and quantization processing, and need to maintain high information density and regular shapes.

[0178] In one implementation, the convolutional neural network block is composed of three ConvNeXt structures. The size of the first convolutional kernel is set to 3×3 to extract energy mutations in the short-time frequency band. The second kernel is set to 5×5 to perceive the resonance region with a longer time span. The third layer introduces a skip connection structure to fuse multi-scale responses. The layer normalization module is embedded after each convolution to ensure balanced distributions between feature channels.

[0179] In another implementation, the forward layer adopts a two-layer fully connected structure, with a GELU activation function inserted in the middle to enhance the high-frequency structure modeling ability through non-linear response. The feature dimension is compressed from the original 128 dimensions to 64 dimensions, and average pooling is used for downsampling to remove unnecessary local oscillations without losing the global frequency distribution.

[0180] It is also possible to fuse the outputs of different convolutional kernels through residual channels, perform downsampling after expanding the feature dimension to a higher-order space, and improve the ability to retain the high-frequency energy boundary. In addition, in specific low-bitrate coding scenarios, the forward layer can use a sparse activation function to compress the subsequent coding volume and improve the coding rate efficiency.

[0181] In this embodiment, while maintaining the integrity of the spectral features, the neural network structure is used to perform targeted structure extraction, normalization adjustment, and non-linear modeling on the magnitude spectrum, and then compact representation is achieved through dimensionality reduction and compression. In the compression stage, the interference of irrelevant information on the coding accuracy is effectively reduced, and at the same time, the modeling accuracy of the subsequent residual quantization module for the energy structure is improved, thereby improving the overall compression efficiency and ensuring the clarity and detail fidelity of speech reconstruction.

[0182] In one embodiment, the above step S50 includes:

[0183] S501, initializing a trainable codebook matrix containing multiple codebook vectors;

[0184] S502, determining the initial residual between the concatenated feature and the codebook vector to generate an initial residual vector;

[0185] S503, performing an iterative optimization operation on the initial residual vector to generate an optimized residual vector;

[0186] S504, generating a quantization index vector and residual parameters according to the optimized residual vector, and outputting the quantization index vector and residual parameters as a compressed feature vector.

[0187] In this embodiment, initializing a trainable codebook matrix containing multiple codebook vectors means constructing a set of vectors for discrete representation of features during the training stage, where each codebook vector is a set of learnable basic representation vectors. This codebook matrix is used to map the concatenated features in the continuous space to the discrete index space to achieve compressed representation. The dimension of each codebook vector should be the same as the dimension of the concatenated features for one-to-one distance calculation and residual calculation. The initialization of the codebook vectors can use a Gaussian random distribution or the center points of K-means clustering as the initial state, and the parameters are updated according to the reconstruction error backpropagation during the training process, so as to continuously adapt to the distribution of the real concatenated features.

[0188] Determining the initial residual between the splicing feature and the codebook vector means that for each splicing feature vector, calculating the error in the form of Euclidean distance, cosine distance, or L1 norm between it and each codebook vector in the codebook matrix, and selecting the codebook vector with the smallest distance as the basic representation. The difference between this codebook vector and the original splicing feature is the initial residual vector. The initial residual vector serves as the starting point for subsequent compression accuracy optimization and contains the high-frequency details in the splicing feature that cannot be captured by the discrete representation.

[0189] Performing iterative optimization operations on the initial residual vector means using a multi-step optimization mechanism to perform compression compensation on the initial residual. This process can gradually approximate the uncoded information in the splicing feature through the residual vector to achieve residual hierarchical quantization. The optimization method can use gradient descent to minimize the mean square error between the splicing feature and the reconstructed feature, or use a sparse optimization strategy to ensure the discreteness of the quantization index matches the codebook capacity. Each round of optimization in the iterative process may update the current optimal codebook vector combination, making the quantization error converge to a set threshold.

[0190] Generating a quantization index vector and residual parameters based on the optimized residual vector means outputting the optimal codebook index and its residual compensation value obtained from the final iteration as components of the compressed feature vector. The quantization index vector is the set of codebook numbers matched by each splicing feature, and the residual parameter is the offset between the splicing feature and the codebook combination representation. These two parts of information together constitute the compressed feature vector, whose representation ability combines discrete efficiency and continuous fineness, which is not only conducive to compressed transmission but also can be used for high-fidelity recovery in the subsequent decoding stage.

[0191] In one implementation, the codebook matrix contains one thousand codebook vectors when initialized, and the vector dimension is the same as the splicing feature. The initialization is randomly generated using a normal distribution and updated by the backpropagation algorithm during the end-to-end training process. During quantization, each splicing feature selects the nearest neighbor codebook by calculating the L2 distance between it and all codebook vectors, and records this codebook index as the initial quantization index.

[0192] In another implementation, the initial residual vector is input into a lightweight feedforward network to predict the optimal residual direction and perform three rounds of residual superposition optimization. The optimized residual vector is normalized and then sent together with the codebook representation into the encoder module to generate the final compressed feature vector.

[0193] A hierarchical residual quantization strategy can also be adopted, which divides the splicing feature into multiple frequency bands, and each frequency band is encoded using a different subset of codebooks to improve the compression adaptability in different frequency ranges. For the dynamically changing regions in the speech signal, such as when the speech rate speeds up or during the transition of consonants, the optimization process will give the residual more expression space to retain the detailed changes.

[0194] In this embodiment, by converting the splicing feature into a joint representation of index and residual, while maintaining a high compression ratio, the detailed structure in the speech signal is effectively retained. In particular, the modeling ability for high-frequency structures and non-steady change regions is significantly enhanced. Through the trainable codebook and residual optimization mechanism, the compression process has an end-to-end adaptive ability, avoiding the expression limitation brought by the fixed representation space in traditional vector quantization. The finally output compressed feature vector can effectively adapt to the entropy coding module, further improving the overall compression efficiency.

[0195] In one embodiment, the above step S60 includes:

[0196] S601, performing adaptive arithmetic coding on the quantization index vector in the compressed feature vector to generate an index bitstream;

[0197] S602, performing bit-width compression and context modeling coding on the residual parameters in the compressed feature vector to generate a residual bitstream;

[0198] S603, splicing the index bitstream and the residual bitstream according to a preset format to generate a compressed bitstream.

[0199] In this embodiment, performing adaptive arithmetic coding on the quantization index vector in the compressed feature vector means constructing a dynamic probability model for the discrete index value sequence, and based on this model, allocating probability intervals to the index vector, so as to generate a highly compressed bitstream. In this process, the encoder dynamically updates the occurrence probability of the current symbol according to the historical distribution of the input index values, and accordingly shrinks the coding interval, so that high-frequency indexes use shorter codewords and low-frequency indexes use longer codewords. Adaptive arithmetic coding has a compression efficiency close to the entropy limit, can effectively improve the coding compression ratio of the index part, and at the same time avoid the instability brought by fixed dictionary coding.

[0200] Performing bit-width compression and context modeling coding on the residual parameters in the compressed feature vector means first performing fixed-point processing on the continuous residual values, and then using the context modeling strategy to differentially encode the quantized residuals. Bit-width compression can compress the numerical expression range through methods such as mean removal, range normalization, and exponential fitting, reducing the effective number of bits required for coding. Context modeling coding constructs a multi-dimensional conditional probability model based on the context state of the current residual (such as neighboring frame residuals, frequency band range, quantization level, etc.), and then encodes based on the prediction of the symbol distribution, thereby improving the compression efficiency of the residual bitstream. This coding method can adapt to the residual distribution characteristics of different speech paragraphs and improve the modeling ability for dynamic feature changes.

[0201] Concatenating the index bitstream and the residual bitstream according to a preset format means organizing and encapsulating the two types of bitstreams according to a defined data structure to generate a complete compressed bitstream that can be parsed by the decoding end. The preset format may include the starting position of the index segment, the code length identifier, the frequency band identifier, or the bit width header, ensuring that the boundaries and mapping relationships between the index and the residual can be accurately identified during decoding. The design of the concatenation format needs to take into account byte alignment, decoding efficiency, and generality, avoiding redundancy caused by repeated or overly nested coding fields.

[0202] In one implementation, the adaptive arithmetic encoder uses a sliding window mechanism to perform frequency statistics on the most recent N index symbols, and constructs the current probability distribution table by accumulating symbol frequencies. When the window is updated, an exponential decay method is used to gradually weaken the historical frequencies to quickly respond to changes in the index distribution.

[0203] In another method, the residual parameters are first normalized, mapping their maximum value to the 16-bit fixed-point number representation space, and independently compressing the bit width for each frequency band. During the context modeling process, joint sliding modeling of the residual is performed in the time dimension and the frequency dimension. A multi-layer perceptron is used to predict the probability distribution of the current residual value, and an entropy encoder based on the CDF model is used to generate the residual bitstream.

[0204] A head nested concatenation format can also be designed. The compressed bitstream consists of a unified identification header, an index code segment, and a residual code segment. The header contains a byte alignment control flag and redundancy check information, improving the synchronization and fault tolerance capabilities of the compressed stream in a distributed environment.

[0205] This embodiment significantly improves the encoding efficiency and the decoding robustness of the stream through probability modeling and structured concatenation. The adaptive modeling of the index bitstream makes full use of the local concentration of the symbol frequency distribution, and the context modeling of the residual bitstream improves the characterization accuracy of the continuous value change pattern. The length of the finally formed compressed bitstream is significantly shortened, while all the structural information required for reconstruction is retained, laying a high-precision foundation for the subsequent speech decoding process.

[0206] In one embodiment, the above step S80 includes:

[0207] S801, separating the recovered quantized index vector and the recovered residual parameters from the recovered compressed feature vector;

[0208] S802, looking up the corresponding codebook vector in the codebook matrix according to the recovered quantized index vector to generate a basic feature vector;

[0209] S803, adding the recovered residual parameters to the basic feature vector to generate an initial reconstructed feature;

[0210] S804, performing an inverse projection correction operation on the initial reconstructed feature to generate a reconstructed concatenated feature.

[0211] In this embodiment, separating the recovered quantization index vector and the recovered residual parameter from the recovered compressed feature vector is a pre-step for structural analysis of the composite feature vector generated during the encoding process. The compressed feature vector consists of a set of structured coding contents, including discrete quantization index values and continuous residual compensation items. At the decoding end, these two parts need to be accurately disassembled for reverse restoration. The quantization index vector is used to identify the original position of the feature in the codebook space, while the residual parameter records the detailed information that is not captured by quantization.

[0212] Searching for the corresponding codebook vector in the codebook matrix based on the recovered quantized index vector means using the decoded index value to locate the corresponding basic feature vector in the trained codebook matrix. This process can be regarded as a query operation, where the index value is directly mapped to the specific position of the multidimensional codebook matrix and a vector with a fixed structure is output. The codebook vector is usually a discretized representation of the features before compression, which is the optimal center point set obtained by clustering in the encoding stage, and supports fast reconstruction.

[0213] Adding the recovered residual parameters to the basic feature vector to generate the initial reconstructed features is the process of superimposing fine-grained error information on the quantized approximate vector to restore a representation closer to the original continuous feature. This addition operation requires that the basic vector pointed to by the quantized index and the residual parameter match exactly in shape. After superposition, the nonlinear perturbations and small changes in the speech features are retained, allowing the decoder to restore the fuzzy parts lost during the training phase.

[0214] Performing back-projection correction on the initial reconstruction features to generate reconstructed splicing features is to further correct the reconstruction deviation caused by residual error accumulation, index mismatch, quantization saturation and other problems. Back-projection correction is usually completed through a shallow network structure or mapping transformation function. It dynamically adjusts the feature energy distribution and frequency band structure according to the offset of the initial reconstruction features in the time-frequency space, making them more consistent with the continuous feature space before encoding, avoiding problems such as frequency band breakage and energy jump after reconstruction.

[0215] In one implementation, the separation of compressed feature vectors can be performed based on field parsing of the byte identifier header, where the index vector is encoded as a low-order integer and the residual parameter is a high-order floating-point compressed value, using fixed-length field positioning for fast disassembly.

[0216] In another implementation, the codebook matrix adopts a multi-resolution nested structure design to map the quantization index to basis vectors at multiple scales, generate cross-scale basic features through weighted fusion, and enhance the stability of the features in different time windows.

[0217] The residual superposition mechanism can also be designed to combine with the position encoding information. After concatenating the residual parameters and the position feature vectors, they are input into the residual correction network, enabling the residual superposition process to have position self - adaptability and adapting to the non - uniform distortion problem in long speech paragraphs.

[0218] In the back - projection correction operation, a single - layer feed - forward network can be used to perform a learnable linear transformation on each frequency point, and the residual connection mechanism is used to retain the backbone information of the input features, strengthening the interpretability and reconstruction accuracy of the correction results.

[0219] In this embodiment, through the combination of index mapping and residual compensation, while maintaining bit - rate control, the high - fidelity feature distribution can be restored. The back - projection correction mechanism further makes up for the weakness that the initial residual cannot fully restore the encoded features, realizing the closed - loop correction of the errors introduced during the compression process, thereby enhancing the structural coherence and perceptual naturalness of the reconstructed speech signal.

[0220] In one embodiment, the above - mentioned step S90 includes:

[0221] S901, extracting multi - scale context features from the reconstructed concatenated features through a depth - wise separable convolutional layer to generate multi - scale context features;

[0222] S902, performing transposed convolutional upsampling on the multi - scale context features to generate initial upsampled features;

[0223] S903, performing linear interpolation compensation on adjacent band features in the initial upsampled features to generate interpolated compensation features;

[0224] S904, performing band - alignment correction and high - frequency detail enhancement processing on the interpolated compensation features through a gated attention mechanism to generate frequency - domain smoothed features;

[0225] S905, fusing the frequency - domain smoothed features and the reconstructed concatenated features through residual connection to generate the final upsampled features.

[0226] In this embodiment, performing upsampling processing on the reconstructed concatenated features to generate upsampled features is a feature resolution improvement link before reconstructing the speech signal, aiming to enhance the structural expressiveness in the time dimension and frequency dimension. First, multi - scale context features are extracted from the reconstructed concatenated features through a depth - wise separable convolutional layer. This process extracts semantic content from the two - dimensional time - frequency feature map within different receptive field ranges, and the channel dimension and spatial dimension in the convolutional operation are separated, thus effectively reducing the parameter scale while retaining the context relationship at different scales. This multi - scale processing constructs a feature representation basis from local short - term energy changes to global semantic structures.

[0227] Next, perform transposed convolution upsampling on the multi-scale context features to generate initial upsampled features. The transposed convolution operation reversely projects and expands the resolution of the feature map according to a set ratio. It is a learnable interpolation mechanism, different from the static linear interpolation method. Its convolution kernel parameters are automatically adjusted during the training process to maximize the reconstruction quality. Although the generated initial upsampled features match the output requirements in terms of size, they often have problems such as boundary artifacts and band energy jumps.

[0228] Therefore, it is necessary to perform linear interpolation compensation on adjacent band features in the initial upsampled features. This operation performs weighted estimation on the vacant or broken parts between adjacent bands based on the values of adjacent bands to smooth the energy distribution in the frequency dimension and make the spectral structure more continuous. Interpolation compensation usually constructs the estimation process using methods such as symmetric window weights, moving average, or multi-point least squares fitting, and the result is the generation of interpolated compensation features.

[0229] Then, perform band alignment correction and high-frequency detail enhancement processing on the interpolated compensation features through a gated attention mechanism to further dynamically adjust the interpolation error. The gated attention mechanism uses attention weights to weight and strengthen important regions in the frequency sub-bands, while performing gated suppression on low-energy regions, thereby achieving noise suppression and detail amplification in the high-resolution spectrum. Band alignment correction mainly solves the semantic offset problem between different channels, while high-frequency detail enhancement focuses on improving the retention quality of high-frequency phonemes, consonants, and other structures in the speech signal, and outputs frequency-domain smoothed features.

[0230] Finally, fuse the frequency-domain smoothed features and the reconstructed concatenated features through residual connection to generate the final upsampled features. Residual connection is an effective structure fusion strategy that allows the model to retain both the original low-resolution information and the new upsampled information simultaneously, improving the expression completeness and reconstruction robustness of the features. This fusion operation introduces detail enhancement information while maintaining the basic energy distribution of the speech, enabling the subsequent time-domain recovery stage to restore the speech waveform structure with higher quality.

[0231] In one implementation, depthwise separable convolution adopts a combination of depthwise convolution and pointwise convolution. The former captures local context, and the latter fuses the interactions between channels. Multiple convolution kernel sizes such as 3×3 and 5×5 can be set to extract in parallel to enhance the multi-scale structure.

[0232] In another way, transposed convolution upsampling adopts a deconvolution operation with a stride of two and is combined with an edge correction padding strategy to prevent the spectral boundary from shifting after upsampling and improve the smoothness of the transition band.

[0233] It is also possible to introduce a band consistency constraint loss in the interpolation compensation stage and optimize the interpolation weight model through backpropagation using a frequency-domain smoothness metric (such as gradient variation) during the training stage to generate compensation features with stronger structural continuity.

[0234] In the gated attention mechanism, a combined model of channel attention and spatial attention can also be introduced to perform weight enhancement separately in the channel dimension of the spectrogram and the two-dimensional space of time frequency, improving the selectivity of detail restoration.

[0235] The fusion operation can also adopt a per-pixel weighted fusion method, introducing a fusion gating function to determine whether to retain the original features or the upsampled and corrected features according to the semantic importance, so as to adapt to application scenarios with different speech styles and clarity requirements.

[0236] Through the above steps, this embodiment can effectively solve problems such as discontinuous spectral structure, missing high-frequency information, and sudden change of boundary energy existing in traditional upsampling. The combination of multi-scale context extraction and transposed convolution provides rich spatial details, and the introduction of linear interpolation and attention mechanism compensates for structural faults and weak high-frequency expressions, enabling speech features to obtain higher spectral quality at the upsampling stage. Retaining the backbone of the original reconstructed features in the feature fusion stage further enhances the overall robustness, providing a high-fidelity frequency basis for the final waveform reconstruction and significantly improving the clarity and perceptual naturalness of the reconstructed speech.

[0237] In one embodiment, a speech signal compression device is provided, and this speech signal compression device corresponds one-to-one to the speech signal compression method in the above embodiment. Refer to Figure 3 , Figure 3 which is a schematic diagram of the functional modules of a preferred embodiment of the speech signal compression device of the present invention. A spectrum extraction module 10, an amplitude feature processing module 20, a phase feature processing module 30, a feature fusion module 40, a vector quantization module 50, an entropy encoding module 60, an entropy decoding module 70, a vector inverse quantization module 80, an upsampling enhancement module 90, and a spectrum restoration module 100. The detailed description of each functional module is as follows:

[0238] The spectrum extraction module 10 is configured to perform a Fourier transform on the initial speech signal to extract an amplitude spectrum and a phase spectrum from the initial speech signal;

[0239] The amplitude feature processing module 20 is configured to process the amplitude spectrum through a first processing path to generate a processed amplitude feature;

[0240] The phase feature processing module 30 is configured to process the phase spectrum through a second processing path to generate a processed phase feature;

[0241] The feature fusion module 40 is configured to splice the processed amplitude feature and the processed phase feature to generate a spliced feature;

[0242] The vector quantization module 50 is configured to perform quantization processing on the spliced feature through a residual vector quantization layer to generate a compressed feature vector;

[0243] An entropy encoding module 60 for performing entropy encoding on the compressed feature vector to generate a compressed bitstream;

[0244] An entropy decoding module 70 for recovering the compressed feature vector from the compressed bitstream through entropy decoding;

[0245] A vector inverse quantization module 80 for performing inverse quantization processing on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed spliced feature;

[0246] An upsampling enhancement module 90 for performing upsampling processing on the reconstructed spliced feature to generate an upsampled feature;

[0247] A spectrum restoration module 100 for performing an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal.

[0248] In one embodiment, the spectrum extraction module 10 is specifically configured to:

[0249] Divide the initial speech signal into framed signals of a preset length;

[0250] Perform windowing processing on each framed signal to generate a windowed signal;

[0251] Perform a Fourier transform on the windowed signal to generate a complex spectrum;

[0252] Determine the modulus value of each frequency point in the complex spectrum to generate an amplitude spectrum matrix;

[0253] Determine the phase angle of each frequency point in the complex spectrum to generate an original phase spectrum;

[0254] Perform a phase unwrapping operation on the original phase spectrum to eliminate phase jumps and generate an unwrapped phase spectrum;

[0255] Quantize the unwrapped phase spectrum into a phase spectrum in a preset precision format.

[0256] In one embodiment, the amplitude feature processing module 20 is specifically configured to:

[0257] Perform multi-scale feature extraction on the amplitude spectrum through a convolutional neural network block to generate an intermediate amplitude feature;

[0258] Perform layer normalization processing on the intermediate amplitude feature to generate a normalized amplitude feature;

[0259] Perform a non-linear activation transformation on the normalized amplitude feature through a forward layer to generate an activated amplitude feature;

[0260] Perform downsampling operation on the activated amplitude feature to generate a processed amplitude feature.

[0261] In one embodiment, the vector quantization module 50 is specifically configured to:

[0262] Initialize a trainable codebook matrix including a plurality of codebook vectors;

[0263] Determine an initial residual between the spliced feature and the codebook vector, and generate an initial residual vector;

[0264] Perform an iterative optimization operation on the initial residual vector to generate an optimized residual vector;

[0265] Generate a quantization index vector and a residual parameter according to the optimized residual vector, and output the quantization index vector and the residual parameter as a compressed feature vector.

[0266] In one embodiment, the entropy coding module 60 is specifically configured to:

[0267] Perform adaptive arithmetic coding on the quantization index vector in the compressed feature vector to generate an index code stream;

[0268] Perform bit-width compression and context modeling coding on the residual parameter in the compressed feature vector to generate a residual code stream;

[0269] Splice the index code stream and the residual code stream according to a preset format to generate a compressed code stream.

[0270] In one embodiment, the vector dequantization module 80 is specifically configured to:

[0271] Separate the recovered quantization index vector and the recovered residual parameter from the recovered compressed feature vector;

[0272] Find the corresponding codebook vector in the codebook matrix according to the recovered quantization index vector to generate a basic feature vector;

[0273] Add the recovered residual parameter and the basic feature vector to generate an initial reconstructed feature;

[0274] Perform a back-projection correction operation on the initial reconstructed feature to generate a reconstructed spliced feature.

[0275] In one embodiment, the upsampling enhancement module 90 is specifically configured to:

[0276] Perform multi-scale context feature extraction on the reconstructed spliced feature through a depthwise separable convolutional layer to generate multi-scale context features;

[0277] Perform transposed convolutional upsampling on the multi-scale context features to generate an initial upsampled feature;

[0278] Perform linear interpolation compensation on adjacent band features in the initial upsampled features to generate interpolated compensation features;

[0279] Perform band alignment correction and high-frequency detail enhancement processing on the interpolated compensation features through a gated attention mechanism to generate frequency-domain smoothed features;

[0280] Fuse the frequency-domain smoothed features and the reconstructed splicing features through residual connection to generate the final upsampled features.

[0281] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 4 The figure. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a voice signal compression method.

[0282] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as shown in Figure 5 The figure. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a voice signal compression method

[0283] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0284] Perform Fourier transform on the initial voice signal to extract the amplitude spectrum and phase spectrum from the initial voice signal;

[0285] Process the amplitude spectrum through a first processing path to generate processed amplitude features;

[0286] Process the phase spectrum through a second processing path to generate processed phase features;

[0287] Concatenate the processed amplitude features and the processed phase features to generate concatenated features;

[0288] Quantize the concatenated features through a residual vector quantization layer to generate a compressed feature vector;

[0289] Perform entropy coding on the compressed feature vector to generate a compressed bitstream;

[0290] Recover the compressed feature vector from the compressed bitstream through entropy decoding;

[0291] Perform inverse quantization on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed concatenated feature;

[0292] Perform upsampling on the reconstructed concatenated feature to generate an upsampled feature;

[0293] Perform an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal.

[0294] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0295] Perform a Fourier transform on an initial speech signal to extract an amplitude spectrum and a phase spectrum from the initial speech signal;

[0296] Process the amplitude spectrum through a first processing path to generate processed amplitude features;

[0297] Process the phase spectrum through a second processing path to generate processed phase features;

[0298] Concatenate the processed amplitude features and the processed phase features to generate concatenated features;

[0299] Quantize the concatenated features through a residual vector quantization layer to generate a compressed feature vector;

[0300] Perform entropy coding on the compressed feature vector to generate a compressed bitstream;

[0301] Recover the compressed feature vector from the compressed bitstream through entropy decoding;

[0302] Perform inverse quantization on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed concatenated feature;

[0303] Perform upsampling on the reconstructed concatenated feature to generate an upsampled feature;

[0304] Perform an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal.

[0305] It should be noted that for the functions or steps that can be achieved by the above computer-readable storage medium or computer device, reference can be made to the relevant descriptions on the server side and the user side in the foregoing method embodiments. To avoid repetition, they will not be described in detail here.

[0306] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0307] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In practical applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0308] It should be noted that if software tools or components of other companies appear in the embodiments of this application, they are only used for illustrative introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for compressing voice signals, characterized in that, It includes the following steps: Perform a Fourier transform on the initial speech signal to extract the amplitude spectrum and phase spectrum from the initial speech signal; Process the amplitude spectrum through a first processing path to generate a processed amplitude feature; Process the phase spectrum through a second processing path to generate a processed phase feature; Concatenate the processed amplitude feature and the processed phase feature to generate a concatenated feature; Perform quantization processing on the concatenated feature through a residual vector quantization layer to generate a compressed feature vector; Perform entropy coding on the compressed feature vector to generate a compressed code stream; Recover the compressed feature vector from the compressed code stream through entropy decoding; Perform inverse quantization processing on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed concatenated feature; Perform upsampling processing on the reconstructed concatenated feature to generate an upsampled feature; Perform an inverse Fourier transform on the upsampled feature to generate a reconstructed speech signal.

2. The voice signal compression method according to claim 1, characterized in that, Performing a Fourier transform on the initial speech signal to extract the amplitude spectrum and phase spectrum from the initial speech signal includes: Divide the initial speech signal into framed signals of a preset length; Perform windowing processing on each framed signal to generate a windowed signal; Perform a Fourier transform on the windowed signal to generate a complex frequency spectrum; Determine the modulus value of each frequency point in the complex frequency spectrum to generate an amplitude spectrum matrix; Determine the phase angle of each frequency point in the complex frequency spectrum to generate an original phase spectrum; Perform a phase unwrapping operation on the original phase spectrum to eliminate phase jumps and generate an unwrapped phase spectrum; Quantize the unwrapped phase spectrum into a phase spectrum in a preset precision format.

3. The voice signal compression method according to claim 1, characterized in that, Processing the amplitude spectrum through a first processing path to generate a processed amplitude feature includes: Perform multi-scale feature extraction on the amplitude spectrum through a convolutional neural network block to generate an intermediate amplitude feature; Perform layer normalization processing on the intermediate amplitude feature to generate a normalized amplitude feature; Perform a non-linear activation transformation on the normalized amplitude feature through a forward layer to generate an activated amplitude feature; Perform downsampling on the activated amplitude feature to generate a processed amplitude feature.

4. The voice signal compression method according to claim 1, wherein, Performing quantization processing on the concatenated feature through a residual vector quantization layer to generate a compressed feature vector includes: Initialize a trainable codebook matrix containing multiple codebook vectors; Determine the initial residual between the concatenated feature and the codebook vector to generate an initial residual vector; Perform an iterative optimization operation on the initial residual vector to generate an optimized residual vector; Generate a quantization index vector and residual parameters according to the optimized residual vector, and output the quantization index vector and residual parameters as a compressed feature vector.

5. The voice signal compression method according to claim 1, wherein, Performing entropy coding on the compressed feature vector to generate a compressed code stream includes: Perform adaptive arithmetic coding on the quantization index vector in the compressed feature vector to generate an index code stream; Perform bit-width compression and context modeling coding on the residual parameters in the compressed feature vector to generate a residual code stream; Concatenate the index code stream and the residual code stream according to a preset format to generate a compressed code stream.

6. The voice signal compression method according to claim 1, characterized in that Performing inverse quantization processing on the recovered compressed feature vector through a residual inverse quantization layer to generate a reconstructed concatenated feature includes: Separate the recovered quantization index vector and the recovered residual parameters from the recovered compressed feature vector; Look up the corresponding codebook vectors in the codebook matrix according to the restored quantization index vector to generate basic feature vectors; Add the restored residual parameters to the basic feature vectors to generate initial reconstructed features; Perform a back-projection correction operation on the initial reconstructed features to generate reconstructed spliced features.

7. The voice signal compression method according to claim 1, characterized in that, Perform upsampling processing on the reconstructed spliced features to generate upsampled features, including: Extract multi-scale context features from the reconstructed spliced features through a depthwise separable convolutional layer to generate multi-scale context features; Perform transposed convolutional upsampling on the multi-scale context features to generate initial upsampled features; Perform linear interpolation compensation on adjacent band features in the initial upsampled features to generate interpolation compensation features; Perform band alignment correction and high-frequency detail enhancement processing on the interpolation compensation features through a gated attention mechanism to generate frequency-domain smoothed features; Fuse the frequency-domain smoothed features and the reconstructed spliced features through residual connection to generate the final upsampled features.

8. A voice signal compression device, characterized in that, The voice signal compression device includes: A spectrum extraction module for performing Fourier transform on an initial voice signal and extracting an amplitude spectrum and a phase spectrum from the initial voice signal; An amplitude feature processing module for processing the amplitude spectrum through a first processing path to generate processed amplitude features; A phase feature processing module for processing the phase spectrum through a second processing path to generate processed phase features; A feature fusion module for splicing the processed amplitude features and the processed phase features to generate spliced features; A vector quantization module for performing quantization processing on the spliced features through a residual vector quantization layer to generate compressed feature vectors; An entropy encoding module for performing entropy encoding on the compressed feature vectors to generate a compressed code stream; An entropy decoding module for restoring the compressed feature vectors from the compressed code stream through entropy decoding; A vector dequantization module for performing dequantization processing on the restored compressed feature vectors through a residual inverse quantization layer to generate reconstructed spliced features; An upsampling enhancement module for performing upsampling processing on the reconstructed spliced features to generate upsampled features; A spectrum restoration module for performing inverse Fourier transform on the upsampled features to generate a reconstructed voice signal.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a voice signal compression program stored in the memory and executable on the processor. When the voice signal compression program is executed by the processor, the steps of the voice signal compression method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, A voice signal compression program is stored on the storage medium. When the voice signal compression program is executed by a processor, the steps of the voice signal compression method according to any one of claims 1-7 are implemented.

Images