A speech reconstruction method and system based on gating re-estimation and route weighting

Abstract

The application discloses a speech reconstruction method and system based on gating re-scaling and routing weighting, introduces an intra-group channel gating re-scaling module at the encoding end of a neural vocoder model, adaptively re-scales the encoding features through channel grouping and a gating mechanism, and improves the effectiveness and quantization efficiency of feature expression. Meanwhile, a routing network is introduced during the training phase of the neural vocoder model, an adaptive weighting reconstruction loss for the Mel filter bank channel segment is constructed based on the encoding feature output probability distribution, and the model focuses on optimizing the reconstruction error of the Mel filter bank channel segment corresponding to the perceptual key frequency area under a low bit rate. The routing network only participates in loss calculation and parameter updating during the training phase, and is removed during the inference phase, so that no additional routing inference calculation overhead is introduced. The application can improve the perceptual quality and stability of speech reconstruction under an extremely low bit rate, and is suitable for bandwidth-limited scenarios such as satellite communication and short-wave communication.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a speech reconstruction method and system based on gating recalibration and routing weighting. Background Technology

[0002] Low-bit-rate speech coding technology has significant practical value and is urgently needed in application scenarios with limited bandwidth and complex channel environments, such as satellite communication, shortwave communication, underwater acoustic communication, and secure communication. Adopting a low-bit-rate speech coding scheme can effectively save bandwidth and transmission resources, improve communication link utilization, and also reserve more space for voice encryption and secure transmission. Therefore, it has become a core supporting technology for voice communication under harsh channel and narrowband conditions.

[0003] In recent years, deep learning-based neural vocoders have achieved good results in medium-to-high bit-rate speech generation tasks. However, there are still obvious bottlenecks in low bit-rate scenarios: the coding features have information redundancy and uneven distribution in the channel dimension, and quantization resources are easily occupied by feature channels with low perceptual contribution, resulting in insufficient ability to express key perceptual details such as high-frequency fricatives and transient changes. The reconstructed speech exhibits high-frequency blurring, boundary blunting, and increased distortion. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a speech reconstruction method and system based on gated recalibration and route weighting. While maintaining the main encoder-quantizer-decoder process unchanged, it improves the clarity, naturalness, and detail fidelity of reconstructed speech through lightweight gated recalibration and route weighting during the training phase, especially improving the problem of easily damaged high-frequency fricative details and transient structures at low bit rates.

[0005] On the one hand, a speech reconstruction method based on gated recalibration and route weighting is provided, including:

[0006] Acquire the raw speech signal and perform preprocessing;

[0007] The preprocessed speech signal is input into the neural vocoder model, and the reconstructed speech signal is output.

[0008] The processing of the preprocessed speech signal in the neural vocoder model includes:

[0009] The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered based on the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech.

[0010] Furthermore, the intra-group channel gating recalibration module includes a feature statistics module, a gating generation module, and a broadcast recalibration module.

[0011] Furthermore, the encoded features enter the feature statistics module, where global average pooling is performed on the encoded features to obtain channel statistical vectors. Then, the channel dimension is divided according to the preset number of groups, and the statistical vectors are rearranged. Mean aggregation is performed on the channel dimension within the group to obtain the group-level description vector.

[0012] The group-level description vector is further input into the gating generation module, which consists of two fully connected layers and is used to generate group gating weights.

[0013] The group gating weights are then input into the broadcast recalibration module to obtain the recalibrated feature representation.

[0014] Furthermore, an adaptive weighted reconstruction loss mechanism based on route guidance is introduced into the neural vocoder model training framework to participate in loss calculation and parameter update during the training phase.

[0015] Furthermore, the adaptive weighted reconstruction loss mechanism based on route guidance includes:

[0016] The recalibrated features are input into a lightweight routing network, which consists of two one-dimensional convolutional layers. The probability distribution on the preset routing state is obtained based on the channel statistics information of each time frame. The probability distribution is used to determine the combination of the weighting coefficients of the reconstruction loss.

[0017] Subsequently, the frame-by-frame probabilities are aggregated along the time dimension to obtain sample-level routing coefficients, and these coefficients are linearly combined with a preset weight template to form a channel segment weight vector acting on the Mel spectrum domain.

[0018] In the reconstruction loss calculation, this weight vector is applied to the Mel spectrum difference to obtain the route-guided adaptive weighted reconstruction loss.

[0019] Furthermore, the preset weight templates are three Mel channel segment weight templates with different weighting characteristics: the first template is a low-to-mid frequency channel segment weighting template; the second template is a smooth transition template; and the third template is a high frequency channel segment weighting template.

[0020] On the other hand, a speech reconstruction system based on gated recalibration and route weighting is provided, including:

[0021] Signal acquisition module: Acquires the raw speech signal and performs preprocessing;

[0022] Speech reconstruction module: Input the preprocessed speech signal into the neural vocoder model and output the reconstructed speech signal;

[0023] The processing of the preprocessed speech signal in the neural vocoder model includes:

[0024] The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered based on the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech.

[0025] Furthermore, an electronic device is also provided, including:

[0026] Memory, used for non-transitory storage of computer-readable instructions; and

[0027] Processor, for executing the computer-readable instructions,

[0028] When the computer-readable instructions are executed by the processor, they perform the method described in the first aspect above.

[0029] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the method described in the first aspect is performed.

[0030] In another aspect, a computer program product is also provided, including a computer program that, when run on one or more processors, is used to implement the method described in the first aspect above.

[0031] The above technical solution has the following advantages or beneficial effects:

[0032] This invention discloses a speech reconstruction method and system based on gated recalibration and route weighting. The method encodes the input speech to obtain coded features, and introduces an intra-channel gated recalibration module at the encoder output to recalibrate the coded features, thereby improving the information effectiveness and quantization efficiency of the coded features. During the training phase, a routing network is introduced to output a routing state probability distribution based on the coded features, and the reconstruction loss is adaptively weighted based on this routing state probability distribution to optimize model parameters, thereby improving the speech reconstruction quality under low bit rate conditions. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1 This is a flowchart of the speech reconstruction method based on gated recalibration and route weighting in Example 1;

[0035] Figure 2 This is a structural diagram of the neural vocoder model in Example 1;

[0036] Figure 3 This is a structural diagram of the intra-group channel gate recalibration module in Example 1;

[0037] Among them, (a) is the in-group channel gate recalibration module; (b) is the feature statistics module; and (c) is the gate generation module.

[0038] Figure 4 This is a diagram of the routing network and weighted loss calculation structure for Example 1. Detailed Implementation

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0040] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] In this embodiment of the invention, "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of this invention, "multiple" refers to two or more.

[0042] Furthermore, to facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0043] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0044] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0045] Example 1

[0046] like Figure 1 As shown, this embodiment provides a speech reconstruction method based on gated recalibration and route weighting, including:

[0047] S1: Acquire the raw speech signal and perform preprocessing;

[0048] S2: Input the preprocessed speech signal into the neural vocoder model and output the reconstructed speech signal;

[0049] The processing of the preprocessed speech signal in the neural vocoder model includes:

[0050] The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered based on the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech.

[0051] As one embodiment, this disclosure presents a speech reconstruction method based on gated recalibration and route weighting. This method introduces an intra-group channel gated recalibration module at the encoding end, and adaptively recalibrates the encoded features through channel grouping and gating mechanisms, thereby suppressing redundant channel groups, enhancing key channel groups, and improving the effectiveness of feature representation and quantization efficiency.

[0052] Meanwhile, a routing network is introduced into the training framework of the neural vocoder model. Based on the output probability distribution of the coding features, an adaptive weighted reconstruction loss is constructed for the Mel filter bank channel segments, so that the model focuses on optimizing the reconstruction error of the Mel filter bank channel segments corresponding to the key frequency regions at low bit rates.

[0053] The routing network only participates in loss calculation and parameter updates during the training phase and is removed during the inference phase, thus avoiding the introduction of additional routing inference computation overhead. This invention can improve the perceptual quality and stability of speech reconstruction at extremely low bit rates and is suitable for bandwidth-constrained scenarios such as satellite communication and shortwave communication.

[0054] Therefore, this embodiment, while keeping the main encoder-quantizer-decoder process unchanged, can improve the clarity, naturalness and detail fidelity of reconstructed speech through lightweight gating recalibration and training stage routing weighting, especially improving the problem of high-frequency fricative details and transient structure being easily damaged at low bit rates.

[0055] This embodiment designs a neural vocoder model, such as Figure 2 As shown, the neural vocoder model includes an encoder, a residual vector quantizer, a decoder, and a discriminator. The encoder uses a convolutional neural network to downsample and extract features from the original speech signal, compressing the temporal length while preserving key acoustic information as much as possible. To improve the effective information density of the encoded features in the subsequent discrete quantization process and reduce the influence of redundant channels or channels that contribute little to speech reconstruction, an intra-group channel gating recalibration module is introduced after the sequence modeling layer. The residual vector quantizer is used to construct a codebook to quantize features, mapping continuous features to discrete codeword indices for transmission and storage.

[0056] The decoder consists of a dequantization end and a decoding end. The dequantization end receives discrete indices and recovers the corresponding quantization features to reconstruct an approximate acoustic representation. The decoding end uses a convolutional neural network to upsample and reconstruct the waveform of the recovered quantization features to generate the final speech signal.

[0057] The discriminator uses adversarial training to constrain the distribution and improve the perceptual quality of the generated speech, prompting the generator to output speech with a more reasonable distribution and higher perceptual quality.

[0058] This disclosure presents a speech reconstruction method based on gated recalibration and route weighting, and the specific implementation process is as follows:

[0059] S1: Acquire the raw speech signal and perform preprocessing;

[0060] The original speech signal has a sampling rate of 8kHz, denoted as... , where T is the number of speech sampling points.

[0061] Preprocessing includes batch processing, which processes the original speech signal in batches to obtain the preprocessed speech signal, represented as... , where B is the batch size.

[0062] S2: The preprocessed speech signal is input into the neural vocoder model, and the reconstructed speech signal is output. The neural vocoder model includes an encoder, a residual vector quantizer, a decoder, and a discriminator. The specific processing steps include:

[0063] S21: First, the encoder is composed of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features, including:

[0064] The speech signal input to the neural vocoder model first enters the encoder, where it passes through a one-dimensional convolutional layer to obtain a basic temporal representation. It then sequentially passes through four multi-scale attention convolutional fusion modules and two sequence modeling layers, gradually reducing the temporal resolution while increasing the channel dimension, ultimately yielding the encoded features. ,in Indicates batch size, Indicates the number of channels. This indicates the number of time frames after downsampling.

[0065] S22: Subsequently, the encoded features are input into the intra-group channel gating recalibration module, the encoded features are grouped by channel and group-level global statistics are extracted to generate group gating weights, and the encoded features are adaptively recalibrated based on the group gating weights to obtain the recalibrated features, including:

[0066] To improve the effective information density of encoded features in subsequent discrete quantization processes and reduce the impact of redundant channels or channels that contribute little to speech reconstruction, an intra-group channel gating recalibration module is introduced after the sequence modeling structure, such as... Figure 3 As shown in (a), the intra-group channel gating recalibration module includes a feature statistics module, a gating generation module, and a broadcast recalibration module.

[0067] First, encode features. Enter Figure 3 The feature statistics module shown in (b) in the time dimension The above coding features Perform global average pooling to obtain the channel statistics vector. Then, according to the preset number of groups For channel dimension Divide the data so that each group has the following number of channels: And rearrange the statistical vectors to In this form, mean aggregation is performed on the within-group channel dimension to obtain the group-level descriptive vector. .

[0068] The group-level description vector Further entry Figure 3 The gating generation module shown in (c) consists of two fully connected layers. The first layer is followed by ReLU activation, and the second layer is followed by Sigmoid activation, generating group gating weights with values ​​ranging from (0,1). .

[0069] Group gating weights Then it enters the broadcast recalibration module, grouping the gating weights. Expand back on the channel dimension The feature representation is generated in the first dimension and broadcast along the time dimension to form a gated tensor of the same size as the encoded features. This gated tensor is then multiplied element-wise with the original encoded features to obtain the recalibrated feature representation. .

[0070] This adaptive recalibration mechanism based on channel group statistics can suppress redundant channel groups and enhance key channel groups while maintaining the overall representation structure, thereby improving the quantifiability of coding features and the ability to express speech perception information.

[0071] Existing models mostly use a globally uniform reconstruction loss as the training objective, which lacks the ability to distinguish the differences in perceptual sensitivity of different Mel channel segments and different speech structures. They cannot dynamically adjust and optimize weights according to spectral characteristics, making it difficult to achieve accurate optimization of Mel channel segments corresponding to easily degraded and highly sensitive key frequency regions under bitrate-limited conditions.

[0072] In one embodiment, to improve the optimization capability for reconstruction error of key Mel filter bank channels under ultra-low bit rate conditions, this disclosure introduces a route-guided adaptive weighted reconstruction loss mechanism into the training framework of the neural vocoder model, such as... Figure 4 As shown, this mechanism introduces a lightweight routing network based on encoder output features to generate routing coefficients, constructs an adaptive weighted reconstruction loss in the Mel spectral domain, and participates in the optimization of the entire neural vocoder model during the training phase. This mechanism only participates in loss calculation and parameter updates during the training phase; it is removed during the inference phase, thus not changing the deployment complexity of the encoder-quantizer-decoder backbone structure.

[0073] In the aforementioned route-guided adaptive weighted reconstruction loss mechanism, the adaptively adjusted parameters are the Mel spectrum domain channel segment weighting parameters in the reconstruction loss. Specifically, the routing network outputs a frame-by-frame probability distribution, which is aggregated along the time dimension to obtain sample-level routing coefficients. Weight vectors acting on Mel spectrum domain channel segments are then generated based on these sample-level routing coefficients. These weight vectors are used to adaptively weight the error terms of different Mel channel segments in the multi-scale Mel reconstruction loss. During the training phase, the weighted loss updates the routing network parameters through backpropagation and adjusts the gradient distribution of the generator model parameters.

[0074] Specifically, the recalibrated feature representation obtained by the encoder Input a lightweight routing network, which consists of two one-dimensional convolutional layers. Output a probability distribution of the preset routing state based on the channel statistics of each time frame. The probability distribution is used to determine the combination of weighting coefficients for reconstruction loss, where the number of states K=3. Three Mel channel segment weighting templates with different weighting characteristics are preset: the first template is a low-to-mid-frequency channel segment-focused template. The low-frequency and mid-frequency channel segments are assigned higher weighting coefficients. The second template is a smooth transition template. Each channel segment is assigned a weighting coefficient with a gradual change. The third template is a template focused on high-frequency channel segments. A higher weighting coefficient is assigned to the high-frequency channel segment.

[0075] The three templates are based on the frequency response preset of the Mel filter bank. The low-mid frequency focused template has a higher response value in the low-frequency and mid-frequency channel segments, the high frequency focused template has a higher response value in the high frequency channel segment, and the smooth transition template has a gradual change in response value in each channel segment.

[0076] The probability distribution is used only to generate the weighting coefficients for the reconstruction loss and does not involve explicit speech category discrimination. Subsequently, the frame-by-frame probabilities are aggregated along the time dimension to obtain sample-level routing coefficients, which are then linearly combined with three preset weight templates to form a channel segment weight vector acting in the Mel spectrum domain. This combination mechanism enables the generated weight vectors to exhibit a rich variety of distribution patterns, including non-monotonic weighting patterns, thereby better adapting to complex and ever-changing speech spectrum structures.

[0077] In the reconstruction loss calculation, this weight vector is applied to the Mel spectrum difference to obtain the route-guided adaptive weighted reconstruction loss. This loss, along with other training loss terms, participates in backpropagation to update the model parameters.

[0078] Through the above method, this disclosure can adaptively adjust the importance allocation of different Mel channel segments in the optimization process without explicit speech state discrimination, thereby improving the subjective perception quality of ultra-low bit rate speech reconstruction without increasing inference complexity.

[0079] S23: The recalibrated features are input into the residual vector quantizer for discretization, resulting in a discrete index sequence and the corresponding quantized feature representation, including:

[0080] Recalibrated features The input residual vector is discretized by a quantizer consisting of multiple cascaded codebooks. Each codebook sequentially performs nearest neighbor matching on the current residual vector to obtain the quantized vector and the corresponding discrete index, and then passes the quantized residual to the next codebook until the entire cascaded quantization process is completed. Finally, a discrete index sequence for transmission or storage is obtained, as well as the quantized feature representation corresponding to the discrete index.

[0081] Under the condition of limited codebook capacity, the system achieves stepwise approximation of continuous coding features, thereby meeting the reconstruction accuracy requirements in ultra-low bit rate speech coding scenarios.

[0082] S3: During decoding, the quantization features are recovered based on the discrete index sequence, and the recovered quantization features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech, including:

[0083] The dequantization end in the decoder recovers the quantization features based on the received discrete index sequence. The recovered quantized features are input to the decoder, and through upsampling and waveform reconstruction, the reconstructed speech is obtained. .

[0084] To further improve the naturalness of the reconstructed speech, a multi-scale discriminator and a multi-cycle discriminator are introduced during the training phase to distinguish between real and fake generated speech. Furthermore, the generator parameters are jointly optimized by adversarial loss and feature matching loss to obtain a more stable and realistic reconstruction effect.

[0085] This invention introduces an intra-group channel gating recalibration module and a routing network-driven adaptive weighting technique for reconstruction loss into the training framework of a low-bit-rate neural vocoder model. This achieves the synergistic effect of selective enhancement at the feature level and adaptive optimization of the Mel filter bank channels at the objective function level, thereby improving the naturalness and clarity of the reconstructed speech under extremely low bit-rate conditions.

[0086] Specifically, the intra-group channel gating recalibration module generates group gating weights by grouping the encoded features into channels and extracting group-level global statistics. It then performs adaptive recalibration of the encoded features, effectively suppressing redundant or less contributing channel groups to reconstruction. At the same time, it enhances effective channel groups related to key speech structures, making the feature representation entering the quantizer more compact and significantly alleviating the problems of quantization error amplification and high-frequency detail loss at low bit rates.

[0087] The routing network outputs a routing state probability distribution for the encoded features during the training phase, and uses this distribution to construct Mel spectrum domain channel segment weights to dynamically weight the reconstruction loss. This allows the model to differentiate the error allocation for different Mel filter bank channel segments during training, especially optimizing high-frequency details and transient components that are more susceptible to low bitrates, thereby effectively improving the perceptual quality of the reconstructed speech. Since the routing network only participates in loss calculation during the training phase and is completely removed during the inference phase, this invention improves speech reconstruction quality without introducing additional routing inference calculations and maintains the encoder-quantizer-decoder main workflow unchanged, exhibiting excellent deployment friendliness. It is particularly suitable for extremely low-bitrate speech encoding and decoding applications in bandwidth-constrained scenarios such as satellite communication, shortwave communication, underwater acoustic communication, and secure communication.

[0088] Example 2

[0089] This embodiment provides a speech reconstruction system based on gated recalibration and route weighting, including:

[0090] Signal acquisition module: Acquires the raw speech signal and performs preprocessing;

[0091] Speech reconstruction module: Input the preprocessed speech signal into the neural vocoder model and output the reconstructed speech signal;

[0092] The processing of the preprocessed speech signal in the neural vocoder model includes:

[0093] The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered based on the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech.

[0094] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0095] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0096] Example 3

[0097] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0098] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0099] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0100] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0101] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0102] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0103] Example 4

[0104] This embodiment also provides a storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.

[0105] Example 5

[0106] This embodiment also provides a computer program product, including a computer program that, when run on one or more processors, implements the method described in Embodiment 1.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A speech reconstruction method based on gated re-estimation and route weighting, characterized in that, include: Acquire the raw speech signal and perform preprocessing; The preprocessed speech signal is input into the neural vocoder model, and the reconstructed speech signal is output. The processing of the preprocessed speech signal in the neural vocoder model includes: The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered from the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech. A route-guided adaptive weighted reconstruction loss mechanism is introduced into the neural vocoder model training framework to participate in loss calculation and parameter updates during the training phase; the route-guided adaptive weighted reconstruction loss mechanism includes: The recalibrated features are input into a lightweight routing network, which consists of two one-dimensional convolutional layers. The probability distribution on the preset routing state is obtained based on the channel statistics information of each time frame. The probability distribution is used to determine the combination of the weighting coefficients of the reconstruction loss. Subsequently, the frame-by-frame probabilities are aggregated along the time dimension to obtain sample-level routing coefficients, and these coefficients are linearly combined with a preset weight template to form a channel segment weight vector acting on the Mel spectrum domain; the preset weight template consists of three Mel channel segment weight templates with different weighting characteristics: the first template is a low-to-mid frequency channel segment emphasis template; the second template is a smooth transition template; and the third template is a high-frequency channel segment emphasis template. In the reconstruction loss calculation, this weight vector is applied to the Mel spectrum difference to obtain the route-guided adaptive weighted reconstruction loss.

2. The voice reconstruction method based on gating and re-estimation and routing weighting according to claim 1, characterized in that, The encoded features enter the feature statistics module, where global average pooling is performed on the encoded features to obtain channel statistical vectors. Then, the channel dimension is divided according to the preset number of groups, and the statistical vectors are rearranged. The mean is aggregated on the channel dimension within the group to obtain the group-level description vector. The group-level description vector is further input into the gating generation module, which consists of two fully connected layers and is used to generate group gating weights. The group gating weights are then input into the broadcast recalibration module to obtain the recalibrated feature representation.

3. A speech reconstruction system based on gated recalibration and route weighting, characterized in that, The speech reconstruction method based on gated recalibration and route weighting as described in any one of claims 1-2 includes: Signal acquisition module: Acquires the raw speech signal and performs preprocessing; Speech reconstruction module: Input the preprocessed speech signal into the neural vocoder model and output the reconstructed speech signal; The processing of the preprocessed speech signal in the neural vocoder model includes: The input speech signal first enters the encoder, which consists of a multi-level one-dimensional convolutional downsampling structure and a sequence modeling structure to obtain encoded features. These encoded features are then input into a group-level channel-gated recalibration module, which groups the encoded features by channel and extracts group-level global statistics to generate group gating weights. Based on these group gating weights, the encoded features are adaptively recalibrated to obtain recalibrated features. The recalibrated features are then input into a residual vector quantizer for discretization, resulting in a discrete index sequence. During decoding, the quantized features are recovered based on the discrete index sequence, and the recovered quantized features are input into the decoder. Decoding is performed through upsampling and waveform reconstruction to obtain the reconstructed speech.

4. An electronic device, characterized in that, include: A memory for non-transitory storage of computer-readable instructions; and a processor for executing the computer-readable instructions. When the computer-readable instructions are executed by the processor, they perform a speech reconstruction method based on gating recalibration and routing weighting as described in any one of claims 1-2.

5. A storage medium, characterized in that, Non-transitory storage computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the speech reconstruction method based on gating recalibration and routing weighting as described in any one of claims 1-2 is performed.

6. A computer program product, characterized in that, The method includes a computer program that, when running on one or more processors, implements the speech reconstruction method based on gating recalibration and routing weighting as described in any one of claims 1-2.

Images