End-to-end flow matching speech enhancement method fusing generative and discriminative learning

Abstract

The present application relates to the fusion end-to-end flow matching speech enhancement method of generative and discriminative learning, belongs to the technical field of speech signal processing, the present application constructs the end-to-end framework including conditional input, vector field network, neural ODE solution and hybrid loss calculation;First, the noisy speech is superimposed with Gaussian noise and spliced to form a conditional input, then the conditional vector field is learned through the NCSN++ architecture, the trajectory integration and enhanced speech generation are realized by using the differentiable neural ODE solver, finally the generative CFM loss and discriminative loss including time domain L1, spectral convergence and logarithmic STFT amplitude are fused for joint optimization.The present application breaks the performance-efficiency trade-off of lightweight model through the generation-discrimination hybrid learning mechanism, improves the fidelity and generalization ability of enhanced speech without increasing the inference overhead, realizes the direct mapping from noisy speech to high-quality enhanced speech, and has important application value in real-time communication, hearing aid, speech recognition and other scenes.

Description

Technical Field

[0001] This invention relates to an end-to-end stream matching speech enhancement method that integrates generative and discriminative learning, belonging to the field of speech signal processing technology. Background Technology

[0002] Speech is one of the most frequently used carriers of information in daily human communication. However, during the generation, transmission, and reception of speech signals, they are highly susceptible to environmental noise (such as the roar of range hoods, noise in public places, and traffic noise) and network transmission interference, leading to a decline in speech quality and intelligibility. With the rapid development of artificial intelligence technology and intelligent communication, people's demand for the accuracy and clarity of speech information transmission is increasing. Speech enhancement (SE), as a core supporting technology in modern communication, speech recognition, hearing aids, and other fields, has seen its model performance optimization and efficiency improvement become a research hotspot.

[0003] Currently, deep learning-based speech enhancement methods have gradually replaced traditional methods. This field has long been dominated by discriminative models, which recover clean speech by learning the mapping relationship from noisy speech to clean speech or predicting masks. However, these models have obvious drawbacks: poor generalization ability for unknown noise types that have not been seen before, and the recovered speech is prone to over-smoothing, resulting in the loss of speech details.

[0004] To overcome the limitations of discriminative models, generative models have been introduced into speech enhancement tasks. Early diffusion frameworks achieved speech enhancement by learning the prior distribution of clean speech, but this required a slow iterative sampling process, resulting in a high number of non-feature evaluations (NFEs) and large inference latency, making it difficult to meet the needs of real-time applications. The emergence of the Stream Matching (FM) method effectively reduced the inference latency of the diffusion framework and became one of the mainstream methods for generative speech enhancement. However, the FM method has a significant performance-efficiency trade-off in lightweight models: when reducing the number of model parameters to improve inference efficiency, the Perceived Speech Quality (PESQ) score will drop significantly, making it impossible to balance lightweight model and speech enhancement performance.

[0005] In existing technologies, researchers have proposed several improvement schemes to address the performance-efficiency problem of stream matching: the FlowSE model introduces stream matching methods into speech enhancement, constructs a probabilistic path based on noisy speech conditions, and models the path evolution through ordinary differential equations (ODE), but its performance degrades significantly under lightweight settings; the StoRM model adopts a cascaded prediction-diffusion architecture, using the prediction model as a pre-denoiser and the diffusion model as a post-refiner, which improves enhancement performance but increases model complexity and inference steps; the CasFlowSE model proposes a cascaded dual-stream matching architecture, which improves the PESQ score of lightweight models through a coarse-to-fine progressive enhancement paradigm, but it relies on a multi-stage pipeline, introduces additional estimation steps, negates the efficiency advantage of generative modeling, and cannot achieve single-channel end-to-end speech enhancement.

[0006] In summary, existing speech enhancement methods still have the following technical challenges: 1. Discriminative models have poor generalization ability, resulting in overly smooth speech recovery and loss of details; 2. Generative flow matching models suffer severe performance degradation under lightweight, low NFE settings, making it difficult to balance efficiency and performance; 3. Existing improvement solutions mostly rely on cascaded architectures or additional estimation steps, which increases inference overhead and cannot achieve efficient single-channel end-to-end processing; 4. Purely generative stream matching training objectives are insufficient to recover speech details in lightweight models and lack accurate supervision of the generated trajectories.

[0007] Therefore, there is an urgent need to propose a new speech enhancement method that can effectively improve speech enhancement performance while achieving lightweight models and low inference latency, enabling efficient end-to-end processing and solving the performance-efficiency trade-off problem of existing technologies. Summary of the Invention

[0008] To address the shortcomings of existing discriminative models, such as poor generalization ability, performance degradation of generative stream matching models under lightweight and low NFE settings, and increased inference overhead of cascaded methods, this invention provides an end-to-end stream matching speech enhancement method that integrates generative and discriminative learning. This method achieves an organic unity of lightweight model, low inference latency, and high enhancement performance, improving the fidelity, intelligibility, and generalization ability of enhanced speech, while reducing the difficulty and resource overhead of model engineering deployment.

[0009] An end-to-end stream matching speech enhancement method that integrates generative and discriminative learning includes the following steps: S1: Construct a conditional input module, which superimposes noisy speech with Gaussian noise to obtain an initial noise sample, and then concatenates the noisy speech as a conditional input with the initial noise sample in the channel dimension to form a complete conditional input feature; S2: Construct a vector field network module based on the NCSN++ architecture to perform multi-scale encoding and feature fusion processing on the complete conditional input features, learn the conditional vector field from noise to clean speech, and output a parameterized vector field. ,in y represents the noise sample at time step t, y represents the noisy speech, and t represents the diffusion time step. S3: Construct a neural ODE solving module. During the training phase, use a differentiable ODE solver to integrate the trajectory of the parameterized vector field to generate enhanced speech estimates. During the inference phase, the ODE is solved in reverse from t=1 to t=0 using a numerical integrator to generate the final enhanced speech. S4: Construct a hybrid loss calculation module to calculate generative loss and discriminative loss in parallel during the training phase. After weighted summation of the two, update the parameters of the vector field network module through gradient backpropagation to complete the end-to-end optimization of the model.

[0010] S1 specifically includes: S11: Superimpose the noisy speech y with Gaussian noise to generate an initial noise sample. ,in The standard deviation of Gaussian noise, It is an identity matrix.

[0011] S12: Use the noisy speech y as a condition, and compare it with the initial noise sample. The input conditions are concatenated at the channel level to form a complete set of conditions.

[0012] The NCSN++ architecture consists of an encoder, a decoder, and skip connections, and S2 specifically includes: S21: Construct an encoder module to extract multi-scale features from the complete conditional input features using downsampled residual blocks containing residual units and attention mechanisms, thereby obtaining multi-scale hierarchical features. L represents the number of layers in the encoder; S22: Construct a decoder module to restore the resolution of the deepest features of the multi-scale hierarchical features by upsampling residual blocks containing residual units and attention mechanisms, and gradually map them back to the original feature resolution; S23: Construct a skip connection module to integrate the features from the encoder's l-th layer. The features corresponding to the decoder layer are concatenated along the channel dimension, and attention weights are calculated using a self-attention mechanism to perform weighted fusion of the concatenated features, preserving fine-grained details, and finally outputting the parameterized vector field. .

[0013] In step S3, the differentiable ODE solver is implemented using the odeint function from the torchdiffeq library, and the numerical integrator is implemented using the Euler method; the final expression for enhanced speech is: Where ODEsolve is a differentiable ODE solver. For parameterized vector field networks, satisfying ordinary differential equations This represents the hidden state at time step t.

[0014] S4 specifically includes: S41: Calculate the generative loss By matching the target vector field, the continuous transformation of the data distribution is learned, expressed as: in, For time step t, clean speech And the expectation of noisy speech y, For the target vector field; S42: Calculate the discriminant loss Includes time-domain L1 loss Spectral convergence loss Logarithmic STFT amplitude loss The direct constraint on the difference between enhanced speech and real clean speech is expressed as: in, , , These are the weighting coefficients for each loss term; S43: Generative Loss With discriminant loss Weighted summation is performed, and the parameters of the vector field network module are updated through gradient backpropagation. The total loss expression is: Where α is the weighting coefficient of the generative loss and β is the weighting coefficient of the discriminative loss.

[0015] The time-domain L1 loss Spectral convergence loss Logarithmic STFT amplitude loss The expressions are as follows: in, For authentic and clean voice recordings, It is a short-time Fourier transform. For amplitude operations of complex spectrum, Let Frobenius norm be denoted, and N be the number of features after STFT transformation.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Breaking the performance-efficiency trade-off to achieve lightweight high performance: This invention innovatively introduces a hybrid generative-discriminative learning paradigm, combining pure generative CFM loss with multi-dimensional discriminative loss. Through dual supervision in the time and frequency domains, it achieves accurate recovery of speech details, solving the performance degradation problem of lightweight flow matching models under low NFE settings. The 5.2 million parameter HyFlowSE(T) model outperforms the 65.6 million parameter FlowSE baseline model in PESQ score when NFE=5, achieving a balance between lightweight model and high enhancement performance.

[0017] 2. Improved speech enhancement quality and generalization ability: This invention constructs a flow matching path based on neural ODE and precisely optimizes the speech generation direction through discriminative constraints, effectively suppressing environmental noise interference, avoiding the over-smoothing problem of discriminative models and the artifact problem of generative models, and significantly improving the fidelity and intelligibility of enhanced speech; under harsh acoustic conditions such as low signal-to-noise ratio ([-4,16] dB) and reverberation, the model's core indicators such as PESQ and eSTOI are significantly better than existing baseline models, and it has stronger generalization ability for unknown noise and complex acoustic environments.

[0018] 3. Achieve efficient end-to-end processing and reduce deployment costs: This invention constructs a single-model end-to-end HyFlowSE framework, which eliminates the complex process of cascading multiple models and adding intermediate inference channels in existing technologies, and realizes direct mapping from raw noisy speech to high-quality enhanced speech; the model inference stage only requires a single pass of ODE solution, without additional inference steps, which improves the real-time performance of the system and reduces the engineering deployment difficulty and hardware resource consumption of the model in embedded devices, hearing aids, real-time communication and other scenarios.

[0019] 4. Novel training mechanism and strong model robustness: This invention utilizes the differentiability of neural ODEs to combine discriminative loss with a differentiable ODE solver, enabling the discriminative loss to directly guide the optimization of the entire generation trajectory, achieving end-to-end joint optimization of generation and discrimination targets; when the model parameters are reduced from 65.6 million to 5.2 million, the SI-SDR and other indicators remain stable, demonstrating excellent robustness. The parameter configuration can be flexibly adjusted according to different application scenarios to adapt to different hardware resource requirements.

[0020] 5. Significant application value: This invention integrates four core technologies: conditional input construction, NCSN++ vector field learning, differentiable neural ODE solution, and hybrid loss function optimization. It forms a complete speech enhancement technology solution from noisy speech input to high-quality enhanced speech output. No additional post-processing steps are required. It has broad practical application value and industrialization prospects in real-time voice communication, smart hearing aids, in-vehicle voice recognition, smart home voice interaction and other fields. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a framework diagram of the end-to-end stream matching speech enhancement method that integrates generative and discriminative learning according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Reference Figure 1 The present invention provides an end-to-end stream matching speech enhancement method that integrates generative and discriminative learning, comprising the following steps: S1: Construct a conditional input module, which superimposes noisy speech with Gaussian noise to obtain an initial noise sample, and then concatenates the noisy speech as a conditional input with the initial noise sample in the channel dimension to form a complete conditional input feature; S1 specifically includes: S11: Superimpose the noisy speech y with Gaussian noise to generate an initial noise sample. ,in The standard deviation of Gaussian noise, It is an identity matrix.

[0025] S12: Use the noisy speech y as a condition, and compare it with the initial noise sample. The input conditions are concatenated at the channel level to form a complete set of conditions.

[0026] S2: Construct a vector field network module based on the NCSN++ architecture to perform multi-scale encoding and feature fusion processing on the complete conditional input features, learn the conditional vector field from noise to clean speech, and output a parameterized vector field. ,in y represents the noise sample at time step t, y represents the noisy speech, and t represents the diffusion time step. The NCSN++ architecture consists of an encoder, a decoder, and skip connections, and S2 specifically includes: S21: Construct an encoder module to extract multi-scale features from the complete conditional input features using downsampled residual blocks containing residual units and attention mechanisms, thereby obtaining multi-scale hierarchical features. L represents the number of layers in the encoder; Step S21 can be further described as follows: S211: Construct downsampling residual blocks, each block containing multiple residual units and an attention mechanism, to extract features and compress space from the input features.

[0027] S212: Feature processing involves passing the conditional input through a series of downsampled residual blocks to extract multi-scale hierarchical features. .

[0028] S22: Construct a decoder module to restore the resolution of the deepest features of the multi-scale hierarchical features by upsampling residual blocks containing residual units and attention mechanisms, and gradually map them back to the original feature resolution; Step S22 can be further described as follows: S221: Construct upsampled residual blocks, each block containing multiple residual units and an attention mechanism, to perform feature recovery and spatial expansion on the input features.

[0029] S222: Process the features by gradually restoring the feature resolution to the original size of the deepest features through a series of upsampled residual blocks.

[0030] S23: Construct a skip connection module to integrate the features from the encoder's l-th layer. The features corresponding to the decoder layer are concatenated along the channel dimension, and attention weights are calculated using a self-attention mechanism to perform weighted fusion of the concatenated features, preserving fine-grained details, and finally outputting the parameterized vector field. .

[0031] Step S22 can be further described as follows: S231: Construct a feature concatenation operation, combining the features of the encoder's l-th layer. The features of the corresponding layer of the decoder are concatenated along the channel dimension.

[0032] S232: Calculate attention weights and perform weighted fusion of the concatenated features through a self-attention mechanism to enhance the transmission of key information.

[0033] After undergoing a series of operations in step S2 above, feature c (conditional input) is represented as follows: Where NCSN++ represents the Noise Conditional Fractional Network++ architecture, Let v represent the noise sample at time step t, c represent the conditional input, and t represent the diffusion time step. The output v is the parameterized vector field. This is used for subsequent neural ODE solving.

[0034] S3: Construct a neural ODE solving module. During the training phase, use a differentiable ODE solver to integrate the trajectory of the parameterized vector field to generate enhanced speech estimates. During the inference phase, the ODE is solved in reverse from t=1 to t=0 using a numerical integrator to generate the final enhanced speech. S3 specifically includes: S31: Construct a differentiable ODE solver using the odeint function from the torchdiffeq library.

[0035] S32: During the training phase, perform a full NFE step integral generation. Used for loss calculation; during the inference phase, numerical integrators such as the Euler method are used to solve the ODE in reverse from t=1 to t=0 to generate enhanced speech.

[0036] S4: Construct a hybrid loss calculation module to calculate generative loss and discriminative loss in parallel during the training phase. After weighted summation of the two, update the parameters of the vector field network module through gradient backpropagation to complete the end-to-end optimization of the model.

[0037] S4 specifically includes: S41: Calculate the generative loss By matching the target vector field, the continuous transformation of the data distribution is learned: S42: Calculate the discriminant loss Includes time-domain L1 loss Spectral convergence loss STFT amplitude loss Directly constrain the difference between enhanced speech and real clean speech: in: S43: Weight the sum of the generative and discriminative losses, and optimize the vector field network end-to-end through gradient propagation. : After undergoing the series of operations described above, the speech signal y is represented as follows: Where ODEsolve represents a differentiable ODE solver. Represents a parameterized vector field network. Let y represent the initial noise state and y represent the conditional input (noisy speech), satisfying the ordinary differential equation: in Let y be the hidden state at time step t, and y be the noisy speech condition. It is a parameterized vector field network.

[0038] This invention employs the VoiceBank+DEMAND dataset, which is a mixture of clean speech from VCTK and noise from DEMAND. Furthermore, using three datasets based on the WSJ0 corpus, WSJ0+CHiME3 is generated using the open-source code of StoRM, including a high SNR (H, [0, 20] dB) version and a low SNR (L, [-4, 16] dB) version, as well as WSJ0+Reverb for evaluating performance under reverberation conditions. This invention generates models with 65.6 million, 27.8 million, 11.7 million, and 5.2 million parameters respectively by adjusting the hyperparameters using four different scale configurations. The proposed models are named HyFlowSE, HyFlowSE(M), HyFlowSE(S), and HyFlowSE(T), corresponding to these four parameter configurations. They are compared with representative flow matching baselines, including FlowSE with 65.6 million parameters, FlowSE(M) with 27.8 million parameters, and CasFlowSE. For fair comparison, baseline results were either cited from the original paper or reproduced using the official open-source code for the unavailable NFE=5 setting. All models were trained using the Adam

[23] optimizer with a learning rate of 1×10-4. The key weights for the mixed loss were: α=2×10-4 for the generator term and (wL1, wSC, wMAG)=(1.0,0.5,0.5) for the discriminant term.

[0039] The evaluation was conducted in two phases: First, to demonstrate the effectiveness and scalability of the framework, performance comparisons were performed on all four HyFlowSE variants on the VoiceBank+DEMAND dataset, focusing on the impact of different model capacities. Second, the smallest model in this patent with 5.2 million parameters (HyFlowSE(T)) was benchmarked against larger competitors on the WSJ0 dataset to verify its competitive performance, which is a core assumption of this invention. For all comparisons, the number of function evaluations (NFE) was fixed at 5. Performance was evaluated using standard metrics, including the number of parameters (M), PESQ score, DNSMOS, wav2vec MOS (WVMOS), eSTOI, DNSMOS P.835 (including SIG, BAK, OVRL), and scale-invariant signal-to-distortion ratio (SI-SDR).

[0040] Table 1 compares the present invention with some representative speech enhancement methods on the VoiceBank+DEMAND dataset.

[0041] First, the performance of the comparison method was evaluated on the VoiceBank+DEMAND dataset. As shown in Table 1, the hybrid paradigm proposed in this patent exhibits superior performance on the VoiceBank+DEMAND dataset: the method achieves significant improvements in objective metrics for measuring signal fidelity and intelligibility: at a comparable 27.8 million parameters, HyFlowSE(M) has a significantly higher PESQ score than FlowSE(M) and CasFlowSE. Meanwhile, in terms of parameter efficiency, the compact HyFlowSE(T) (5.2 million parameters) outperforms the significantly larger FlowSE (65.6 million parameters) in PESQ. Furthermore, the method demonstrates significant robustness to lightweight design; despite a substantial reduction in parameters from 65.6 million in HyFlowSE to 5.2 million in HyFlowSE(T), the model maintains a performance level comparable to the full-size model, with very stable SI-SDR values. On the other hand, while the baseline model has a slight advantage in non-invasive DNSMOS metrics, the hybrid design of this invention prioritizes performance and outperforms in signal-level metrics.

[0042] Table 2 compares the present invention with some representative speech enhancement methods on the WSJ0 dataset. Training and testing on WSJ0+CHiME3(H)

[0043] Training and testing on WSJ0+CHiME3(L)

[0044] Training and testing on WSJ0+Reverb

[0045] Performance was evaluated on the more challenging WSJ0 dataset, and the results are shown in Table 2. The data demonstrate that the superiority of our method is particularly evident under harsh acoustic conditions. In the low SNR scenario of WSJ0+CHiME3(L), ​​the compact HyFlowSE(T) with only 5.2 million parameters achieves a significant improvement in PESQ, significantly outperforming the two baseline models with 27.8 million parameters. Similarly, in the reverberation setting, our method outperforms FlowSE(M) and CasFlowSE in terms of perceptual quality. A significant trade-off occurs in the reverberation task, where CasFlowSE achieves a higher SI-SDR, but our HyFlowSE(T) offers superior PESQ performance. Although the baselines have a slight advantage in DNSMOS across all subsets, HyFlowSE(T) consistently leads in key metrics of speech quality and intelligibility, such as PESQ and eSTOI, highlighting its effectiveness in producing high-fidelity speech under challenging conditions.

[0046] Experimental results confirm that the proposed 5.2 million parameter HyFlowSE(T) model achieves state-of-the-art (SOTA) performance, significantly outperforming baselines with parameters several times larger, especially in challenging low SNR scenarios requiring only 5 NFEs. The HyFlowSE framework provides a new approach to breaking down the trade-off between efficiency and performance in generative speech enhancement.

[0047] This invention proposes a hybrid generative-discriminative end-to-end stream matching speech enhancement method, which significantly improves the performance of lightweight models without increasing inference overhead compared with existing pure generative stream matching methods and cascaded multi-stage methods.

[0048] Existing CFM speech enhancement methods employ purely generative training objectives, resulting in severe performance degradation under lightweight, low NFE settings. This invention innovatively introduces a hybrid generative-discriminative learning paradigm, combining standard CFM loss with multi-resolution STFT discriminative loss. Through joint supervision of temporal L1 loss, spectral convergence loss, and logarithmic magnitude loss, it achieves more accurate restoration of speech details, effectively addressing the performance bottleneck of lightweight flow matching models.

[0049] Currently, there is no research combining differentiable neural ODE solvers with hybrid loss training for end-to-end speech enhancement. This invention utilizes the differentiability of neural ODEs and implements the complete ODE integration process using the `odeint` function from the `torchdiffeq` library, allowing the discriminative loss to directly guide the optimization of the entire generation trajectory. This end-to-end training mechanism avoids the additional inference steps required by traditional cascaded methods, achieving performance improvement while maintaining single-pass inference efficiency.

[0050] Existing cascaded two-stream methods improve performance by stacking two stream matching models, but this contradicts the original intention of stream matching to pursue efficient inference. This invention constructs a single-model end-to-end speech enhancement system that uses only a lightweight network (HyFlowSE(T)) with 5.2M parameters. With an NFE=5 setting, it can achieve or even surpass the performance of a baseline model with 65.6M parameters, especially under harsh acoustic conditions such as low signal-to-noise ratio, achieving the best balance between efficiency and performance.

[0051] This invention integrates differentiable solution of neural ODE, hybrid loss function optimization and lightweight network design to form a complete speech enhancement technology solution. It has a complete processing flow from noisy speech input to high-quality enhanced speech output, and has significant practical value and innovation in application scenarios such as real-time communication, hearing aids, and speech recognition.

[0052] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. An end-to-end stream matching speech enhancement method integrating generative and discriminative learning, characterized in that: Includes the following steps: S1: Construct a conditional input module, which superimposes noisy speech with Gaussian noise to obtain an initial noise sample, and then concatenates the noisy speech as a conditional input with the initial noise sample in the channel dimension to form a complete conditional input feature; S2: Construct a vector field network module based on the NCSN++ architecture to perform multi-scale encoding and feature fusion processing on the complete conditional input features, learn the conditional vector field from noise to clean speech, and output a parameterized vector field. ,in y represents the noise sample at time step t, y represents the noisy speech, and t represents the diffusion time step. S3: Construct a neural ODE solving module. During the training phase, use a differentiable ODE solver to integrate the trajectory of the parameterized vector field to generate enhanced speech estimates. ; During the inference phase, the ODE is solved in reverse from t=1 to t=0 using a numerical integrator to generate the final enhanced speech. S4: Construct a hybrid loss calculation module to calculate generative loss and discriminative loss in parallel during the training phase. After weighted summation of the two, update the parameters of the vector field network module through gradient backpropagation to complete the end-to-end optimization of the model.

2. The end-to-end stream matching speech enhancement method based on the fusion of generative and discriminative learning as described in claim 1, characterized in that: S1 specifically includes: S11: Superimpose the noisy speech y with Gaussian noise to generate an initial noise sample. ,in The standard deviation of Gaussian noise, It is the identity matrix; S12: Use the noisy speech y as a condition, and compare it with the initial noise sample. The input conditions are concatenated at the channel level to form a complete set of conditions.

3. The end-to-end stream matching speech enhancement method based on the fusion of generative and discriminative learning as described in claim 1, characterized in that: The NCSN++ architecture consists of an encoder, a decoder, and skip connections, and S2 specifically includes: S21: Construct an encoder module to extract multi-scale features from the complete conditional input features using downsampled residual blocks containing residual units and attention mechanisms, thereby obtaining multi-scale hierarchical features. L represents the number of layers in the encoder; S22: Construct a decoder module to restore the resolution of the deepest features of the multi-scale hierarchical features by upsampling residual blocks containing residual units and attention mechanisms, and gradually map them back to the original feature resolution; S23: Construct a skip connection module to integrate the features from the encoder's l-th layer. The features corresponding to the decoder layer are concatenated along the channel dimension, and attention weights are calculated using a self-attention mechanism to perform weighted fusion of the concatenated features, preserving fine-grained details, and finally outputting the parameterized vector field. .

4. The end-to-end stream matching speech enhancement method based on the fusion of generative and discriminative learning according to claim 1, characterized in that: In step S3, the differentiable ODE solver is implemented using the odeint function from the torchdiffeq library, and the numerical integrator is implemented using the Euler method; the final expression for enhanced speech is: Where ODEsolve is a differentiable ODE solver. For parameterized vector field networks, satisfying ordinary differential equations , This represents the hidden state at time step t.

5. The end-to-end stream matching speech enhancement method based on the fusion of generative and discriminative learning according to claim 1, characterized in that: S4 specifically includes: S41: Calculate the generative loss By matching the target vector field, the continuous transformation of the data distribution is learned, expressed as: in, For time step t, clean speech And the expectation of noisy speech y, For the target vector field; S42: Calculate the discriminant loss Includes time-domain L1 loss Spectral convergence loss Logarithmic STFT amplitude loss The direct constraint on the difference between enhanced speech and real clean speech is expressed as: in, , , These are the weighting coefficients for each loss term; S43: Generative Loss With discriminant loss Weighted summation is performed, and the parameters of the vector field network module are updated through gradient backpropagation. The total loss expression is: Where α is the weighting coefficient of the generative loss and β is the weighting coefficient of the discriminative loss.

6. The end-to-end stream matching speech enhancement method based on the fusion of generative and discriminative learning according to claim 1, characterized in that: The time-domain L1 loss Spectral convergence loss Logarithmic STFT amplitude loss The expressions are as follows: in, For authentic and clean voice recordings, It is a short-time Fourier transform. For amplitude operations of complex spectrum, Let Frobenius norm be denoted, and N be the number of features after STFT transformation.

Images