A speech enhancement method and system of stream matching sample level adaptive path

Abstract

The application discloses a speech enhancement method and system of a stream matching sample level adaptive path, and belongs to the technical field of speech signal processing. The application adopts a deep coupling technical architecture of a feature encoder, a super network, an adaptive probability path generator and a main enhancement model. The feature encoder extracts a noise feature vector from a noisy speech amplitude spectrum based on a convolutional neural network; the super network adopts a multi-layer perceptron to map the feature vector into path parameters satisfying mathematical constraints; the adaptive probability path generator constructs a nonlinear time reparameterization function and a variance scheduling function based on the path parameters; the main enhancement model executes an iterative enhancement process under the modulation of the adaptive path, and finally outputs enhanced speech through signal post-processing. Thus, the problem of poor noise adaptability and insufficient generalization ability caused by path fixation in the existing stream matching speech enhancement method is solved, and the speech enhancement effect and robustness in a complex noise environment can be significantly improved.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a speech enhancement method and system for stream-matched sample-level adaptive paths. Background Technology

[0002] Speech enhancement technology aims to extract clear, intelligible, and clean speech from noisy audio signals, and is one of the core topics in the field of speech signal processing. Early techniques relied heavily on signal processing and statistical methods, such as spectral subtraction and Wiener filtering. In recent years, with the development of deep learning, methods based on deep neural networks (DNNs), especially discriminative models using recurrent neural networks (RNNs) or Transformer architectures, have become mainstream. To further overcome the performance bottlenecks of discriminative models, academia and industry have begun to explore the application of generative models in speech enhancement. Among these, flow-matching-based generative models are an emerging and highly promising approach. These models do not directly learn mappings, but rather learn a probability flow that smoothly and continuously transforms a noise distribution into a clean speech distribution. This process is described by an ordinary differential equation (ODE), and by solving this equation, the transformation from noise to clean speech can be achieved.

[0003] FlowSE models the speech enhancement process as a continuous transformation process. It defines a transition from... t =0 to t The time variable is equal to 1. t When = 0, the signal is noisy speech (or a Gaussian distribution nearby); t When = 1, the signal is the corresponding clean speech. The goal of this model is to learn a time-dependent vector field vt, which guides how the signal should change at each time point t in order to eventually smoothly transform from noisy speech into clean speech.

[0004] In existing speech signal processing techniques, flow matching-based speech enhancement methods (such as FlowSE) typically employ fixed probability paths (e.g., linear interpolation paths), whose path parameters remain unchanged during training and inference. This fixed-path approach ignores the diversity of noise characteristics (such as type, intensity, signal-to-noise ratio, and non-stationarity) in different speech samples, resulting in decreased enhancement performance, loss of speech details, poor subjective listening experience, and limited generalization ability when facing unknown or complex noise scenarios. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a speech enhancement method based on stream matching sample-level adaptive paths. This method utilizes a time-reparameterized function generated by the adaptive probability path generator that satisfies specific mathematical constraints (monotonicity, boundary conditions). t ( t ;c As a modulation signal, it is deeply embedded into the time query and numerical integration steps of the iterative enhancement process performed by the main enhancement model (vector field network) under the adaptive path. This solves the technical problem that the enhancement process under the fixed path is mechanical and rigid, and cannot be "speeded" and "perturbed" according to the noise characteristics. It achieves a smooth, controllable and sample-specific state transition trajectory, and achieves more efficient and thorough noise separation and speech preservation within a limited number of inference steps.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: a speech enhancement method for stream-matched sample-level adaptive paths, comprising the following steps: S1: performing a short-time Fourier transform (STFT) on the noisy speech Y to obtain the noisy amplitude spectrum |Y| and the noisy phase spectrum ∠Y;

[0007] S2: Extract the noise feature vector of the noisy amplitude spectrum |Y| using a feature encoder. c ;

[0008] S3: Based on the noise feature vector through a hypernetwork c Generate adaptive path parameter set P ;

[0009] S4: Based on the parameter set, an adaptive probability path generator is used. P Constructing a time reparameterization function t ( t ; c ) and variance scheduling function s t ′ ,in, t For continuous time, t ( t ; c )exist t Monotonically increasing on [0,1], and satisfying t (0; c )=0、 t (1; c )=1; s t ′ exist t It is monotonically decreasing on [0,1] and satisfies s 0 ′ >0、 s 1 ′ =1;

[0010] S5: Using the noisy amplitude spectrum |Y| as the initial state through the main enhancement model, and according to the time reparameterization function... t ( t ; cThe determined time point sequence and the variance scheduling function s t ′ Iterative enhancement is performed to obtain the enhanced pure spectrum. X clean |;

[0011] S6: The purified spectrum | X clean | Combine the noisy phase spectrum ∠Y with the synthesized enhanced complex spectrum, and perform inverse short-time Fourier transform (ISTFT) to output enhanced speech.

[0012] Preferably, the feature encoder employs a convolutional neural network, which sequentially includes: an initial convolutional layer using a 3×3 convolutional kernel to map the noisy amplitude spectrum of a single channel to 64 channels; multiple residual blocks, consisting of four residual blocks connected in series, each containing two 3×3 convolutional layers and integrating channel attention and dilated convolutions; a temporal aggregation convolutional layer using a 1×3 convolutional kernel to aggregate along the time axis and adjust the number of channels to 128; a time-frequency attention pooling layer to obtain a global representation from the output of the temporal aggregation convolutional layer; and a fully connected layer to map the global representation output by the time-frequency attention pooling layer into a fixed-dimensional noise feature vector. c ; wherein, the noise feature vector c The noise type, signal-to-noise ratio, and non-stationary characteristics of the noisy amplitude spectrum are encoded.

[0013] Preferably, the supernetwork employs a multilayer perceptron for outputting the parameter set. P The parameter set P Including adaptive time path initial shape parameters α′ Adaptive time path termination shape parameters β′ Adaptive initial noise scale s 0 ′ and adaptive variance decay shape parameters c′ ,in α′ >0 and β′ >0 is used to control the shape of the time reparameterization function. s 0 ′ >0 represents the initial noise scale. c′ >0 is used to control the variance scheduling function. s t ′ The rate of decay.

[0014] Preferably, the time reparameterization function t ( t ; c The formula for ) is:

[0015]

[0016] in, α′ The initial shape parameters for the adaptive time path; β′ The shape parameter is the termination parameter for the adaptive time path; t For continuous time; x It is an integral dummy variable.

[0017] Preferably, the variance scheduling function s t ′ The formula is:

[0018]

[0019] in, s 0 ′ The initial noise scale is adaptive; c′ For adaptive variance decay shape parameters; t ( t ; c ) is the time reparameterization function.

[0020] Preferably, the iterative enhancement step includes: adjusting the step size based on the adaptive step size output by the supernetwork. t i ′ and adaptive step count N′ Generate time-point sequences and use a vector field network Vθ at each step based on... t ( t ; c The modulated time query predicts the update direction for performing numerical integration.

[0021] Preferably, the end-to-end joint training optimizes the main augmentation model through a conditional flow matching loss function, which includes: constructing an intermediate spectrum. x t for: x t = m t + s t ′ × x ;in, t For continuous time, the mean of the conditional probability path m t for: m t = t ( t ; c )× x clean +(1- t ( t ; c ))×Y, x Standard Gaussian noise, x clean Y represents the pure speech spectrum, and Y represents the noisy speech spectrum. s t ′ For variance scheduling function, t ( t ; c ) is the time reparameterization function.

[0022] Preferably, the feature encoder, hypernetwork, adaptive probabilistic path generator, and main enhancement model are jointly trained end-to-end to collaboratively achieve noise-adaptive path generation and speech enhancement.

[0023] A system employing the above-mentioned speech enhancement method includes: a feature encoder for receiving a noisy amplitude spectrum and outputting a noise feature vector. c A hypernetwork, connected to the feature encoder, is used to base the noise feature vector... c Generate adaptive path parameters P An adaptive probabilistic path generator, connected to the hypernetwork, is used to generate paths based on the parameters. P Constructing a time reparameterization function t ( t ; c ) and variance scheduling function s t ′ ,in t ( t ; c The expression is monotonically increasing on t∈[0,1] and satisfies t (0; c )=0、 t (1; c )=1; s t ′ It is monotonically decreasing on t∈[0,1] and satisfies s 0 ′ >0、 s 1 ′ =1; Main enhancement model, connected to the adaptive probability path generator, used to take the noisy amplitude spectrum |Y| as the initial state, and according to... t ( t ; c )and s t ′ Iterative enhancement of the noisy amplitude spectrum |Y| is performed to output the enhanced pure spectrum. X clean|; Signal processing module, connected to the main enhancement model, is used to process the pure spectrum| X clean The enhanced complex spectrum is synthesized with the noisy phase spectrum ∠Y, and then the enhanced speech is output after performing the inverse short-time Fourier transform (ISTFT).

[0024] Preferably, the feature encoder includes a convolutional neural network with multiple residual blocks and a channel attention mechanism for extracting noisy feature vectors. c The dimension is 192.

[0025] By adopting the above technical solution, the present invention has the following beneficial effects.

[0026] (1) This invention solves the technical problems of poor noise adaptability and insufficient generalization ability to unknown noise caused by the use of fixed and linear probability paths in existing stream matching-based speech enhancement methods by constructing a deep coupling technical architecture of “feature encoder - super network - adaptive probability path generator - main enhancement model”. It achieves the technical effect of dynamically generating the optimal denoising path for each noisy speech sample, thereby significantly improving the speech enhancement effect and system robustness in complex and variable noise environments.

[0027] (2) This invention uses a feature encoder based on a convolutional neural network (CNN) and an attention mechanism. It employs a CNN combined with residual blocks, dilated convolution, and time-frequency attention pooling to extract high-dimensional noise feature vectors from the noisy amplitude spectrum. This explicitly encodes multi-dimensional features such as noise type, signal-to-noise ratio, and non-stationarity, achieving accurate extraction and dimensionality compression of noise features (e.g., 192-dimensional vectors). This provides a highly discriminative input for path generation, improving feature extraction accuracy by approximately 15%. Simultaneously, this drives a hyper-network to dynamically generate path parameters, solving the technical problem that traditional flow matching models cannot perform targeted optimization based on the noise characteristics (e.g., type, signal-to-noise ratio, non-stationarity) of input samples. This achieves the technical effect of accurate matching of path generation with noise scenes and personalized enhancement of "one path for one sample".

[0028] (3) This invention constructs a nonlinear time reparameterization function based on the noise condition vector adaptively. t ( t ; c ) and variance scheduling function s t ′Furthermore, it is deeply integrated into the iterative integration process of the main enhancement model (vector field network), which solves the technical problem that the enhancement performance of a fixed linear path is significantly reduced in challenging scenarios such as low signal-to-noise ratio, non-steady-state noise and strong reverberation. It achieves a smoother and more controllable speech state transition trajectory, thereby achieving significant improvements in both objective indicators (PESQ, SI-SDR) and subjective listening experience.

[0029] (4) This invention uses an adaptive probability path generator based on parameters P Constructing a time reparameterization function t ( t ; c ) and variance scheduling function s t ′ By using the aforementioned family of differentiable, sample-level probability path functions and order-preserving functions such as Beta-CDF to ensure the rationality and stability of the path, a single linear interpolation is replaced. This provides a rich and controllable selection of transformation trajectories, enabling the denoising process to "variable speed" and "variable perturbation" according to the noise characteristics, thereby improving the enhancement accuracy and robustness.

[0030] (5) This invention, through deep coupling of the main enhancement model and the adaptive path, enables the dynamically generated t ( t ; c As a modulation signal, it is embedded into the time query and numerical integration steps of the vector field network, realizing real-time and dynamic linkage between the path and the enhancement process. This ensures that the "planned path" is strictly executed, thereby achieving enhancement effects and efficiency far exceeding those of fixed-path methods within a limited number of inference steps.

[0031] (6) This invention uses a supernetwork based on a small MLP to dynamically map high-dimensional noise feature vectors into an adaptive path parameter set P that satisfies mathematical constraints. α′ , β′ , s 0 ′ and c′ And through parameter clipping, i.e. α′ , β′ , c′ The parameter set ∈ [0.5, 10] ensures numerical stability, thereby achieving intelligent mapping from the feature space to the path function space. This supports sample-level path customization, freeing the model from dependence on pre-set fixed paths, enhancing performance and generalization ability, and reducing path parameter generation error to within 5%. P Constructing a time reparameterization function t ( t ; c (Using Beta-CDF to guarantee monotonically increasing and boundary conditions) and variance scheduling function s t ′ The (monotonically decreasing) adaptive probability path generator constructs a nonlinear function to replace fixed linear interpolation, eliminating the technical problem that fixed paths cannot achieve "variable speed" and "variable perturbation" denoising under complex noise, resulting in the loss of speech details. This enables the denoising process to adapt to noise characteristics, improving noise suppression efficiency by 20% in low signal-to-noise ratio scenarios.

[0032] (7) This invention uses an active enhancement model to deeply couple the vector field network and the path, and uses it in the iterative enhancement process. t ( t ; c Modulation time query and s t ′ Control noise disturbances and combine them with adaptive step size t i ′ Numerical integration eliminates the mechanical, rigid, and inefficient iteration problems caused by fixed paths in the enhancement process, thus achieving smooth and controllable state transitions. This results in efficient denoising within a finite number of steps (e.g., 5-10 steps), improving the PESQ metric to 1.722 and the STOI to 0.798. This invention overcomes the limitations of existing stream matching methods that use fixed probability paths. Through deep collaboration between the feature encoder, hypernetwork, path generator, and main enhancement model, it achieves sample-level adaptive path generation for different noise characteristics, significantly improving the enhancement effect, generalization ability, and robustness of the speech enhancement system in complex noise environments. Attached Figure Description

[0033] The following provides a detailed discussion of the manufacture and application of preferred embodiments of the present invention. However, it should be understood that the present invention provides many applicable inventive concepts that can be embodied in various specific environments. The specific embodiments discussed are merely illustrative of specific ways of manufacturing and using the present invention and do not limit the scope of the invention. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0034] Figure 1 This is a schematic flowchart of the speech enhancement method of the present invention.

[0035] Figure 2 This is a comparison chart of the spectral effects of the audio data extracted in this invention.

[0036] Figure 3 This is a comparison chart of the verification effect of the present invention in 140h audio. Detailed Implementation

[0037] The following provides a detailed discussion of the manufacture and application of preferred embodiments of the present invention. However, it should be understood that the present invention provides many applicable inventive concepts that can be embodied in various specific environments. The specific embodiments discussed are merely illustrative of specific ways of manufacturing and using the invention and do not limit the scope of the invention.

[0038] This invention addresses the technical problems of existing stream matching speech enhancement methods, which suffer from poor noise adaptability, insufficient generalization ability, and significant performance degradation in low signal-to-noise ratio and non-stationary noise environments due to the use of fixed probability paths. It employs a deeply coupled architecture of "feature encoder - supernetwork - adaptive probabilistic path generator - main enhancement model," utilizing convolutional neural networks to extract noise features, multilayer perceptrons to generate path parameters, nonlinear time reparameterization functions to construct adaptive paths, and vector field networks to perform iterative enhancement under path modulation. The result is improved objective quality indicators of speech enhancement (PESQ improved by 12.26%, STOI improved by 7.69%), enhanced robustness to unknown noise, and sample-level personalized enhancement.

[0039] like Figure 1 This paper presents a stream-matched sample-level adaptive path speech enhancement method. Addressing the technical problems of poor noise adaptability and insufficient generalization ability in existing stream-matched speech enhancement methods due to the use of fixed probability paths, the following specific implementation scheme is provided.

[0040] Step S1 (Signal Preprocessing): First, perform signal preprocessing, load the noisy speech waveform, and perform a Short Time Fourier Transform (STFT) to obtain the complex spectrum Y= a + b i ,in, a For the real part, b The virtual part, i The imaginary unit, i.e.

[0041]

[0042] in, The input signal represents the speech waveform that changes over time. The window function is a localization function centered at time t, and its effect is to localize time... t Extracting a small segment of the signal from the vicinity allows the Fourier transform to reflect only the frequency characteristics of that local time period. This is the time variable of the signal, used for integration calculations; f For frequency; t This is a time parameter.

[0043] The noisy amplitude spectrum |Y| and the noisy phase spectrum ∠Y are separated from the noisy speech Y, where the noisy phase spectrum ∠Y will be used in the final reconstruction stage. The separation formulas for the noisy amplitude spectrum |Y| and the noisy phase spectrum ∠Y are as follows:

[0044]

[0045]

[0046] in, a and b These are the real and imaginary parts of the complex spectral noisy speech Y obtained after short-time Fourier transform; atan 2 is the four-quadrant arctangent function, a special form of the arctangent function. This function can correctly handle and calculate angles in all four quadrants.

[0047] Step S2 (Feature Encoder): The noisy amplitude spectrum |Y| is processed by inputting it into the feature encoder, which employs a convolutional neural network, including convolutional layers, multiple residual blocks, and attention pooling layers. First, a 3×3 convolution maps the single-channel amplitude spectrum to 64 channels, primarily capturing local time-frequency textures. Then, four residual blocks are concatenated, each containing two 3×3 convolutional layers and integrating channel attention mechanisms and dilated convolutions to expand the receptive field and highlight key frequency bands. Next, a 1×3 convolution is used to aggregate the data along the time axis, adjusting the number of channels to 128. This is equivalent to short-range consolidation of adjacent frames, enhancing the perception of non-stationary (time-varying) noise features while maintaining frequency resolution. Finally, a fully connected layer maps the above global representation into a fixed-dimensional noise feature vector. c That is, a global representation is obtained through time-frequency attention pooling, and a 192-dimensional noisy feature vector is output through a fully connected layer. c This is used as a condition for subsequent adaptive path generation and inference scheduling. c It is composed of several sub-vectors, and the sum of the dimensions of each sub-vector is 192. Each sub-vector dimension can be configured with non-steady-state index, estimated SNR, spectral flatness / sharpness, reverberation index, band statistics and dynamics, noise type embedding, and context embedding. This vector explicitly or implicitly encodes the noise type, signal-to-noise ratio and non-steady-state characteristics.

[0048] Step S3 (Hyper-network): Convert the noise feature vector c The input is a supernetwork, which employs a multilayer perceptron to generate an adaptive set of path parameters based on the input features. P This parameter set includes adaptive time path initial shape parameters. α′ Adaptive time path termination shape parameters β′ Adaptive initial noise scale s 0 ′ and adaptive variance decay shape parameters c′ ,in α′ >0 and β′ >0 is used to control the shape of the time reparameterization function. s 0 ′ >0 represents the initial noise scale. c′ >0 is used to control the decay rate of the variance scheduling function. These parameters are constrained within a stable range, such as... α′ , β′ and c′ The range is defined within [0,1] to ensure the stability of the numerical value.

[0049] Adaptive Probabilistic Path Generator: The adaptive probabilistic path generator receives a set of parameters. P Then, the specific path function is constructed, and the time reparameterization function uses the following formula:

[0050]

[0051] in, α′ The initial shape parameters for the adaptive time path; β′ The shape parameter is the termination parameter for the adaptive time path; t For continuous time; x It is an integral dummy variable.

[0052] Ensure that it is monotonically increasing on t∈[0,1] and satisfies t (0; c )=0、 t (1; c =1.

[0053] The variance scheduling function is defined as:

[0054]

[0055] in, s 0 ′ The initial noise scale is adaptive; c′ For adaptive variance decay shape parameters; t ( t ; c () is a time-reparameterized function; it is guaranteed to be monotonically decreasing and satisfies s 0 ′ >0、 s 1 ′ =1.

[0056] The mean of the conditional probability path is defined during the training phase. u tThe calculation formula is as follows:

[0057]

[0058] in, t ( t ; c ) is the time reparameterization function; x clean Y represents the pure speech spectrum; Y represents noisy speech. s t ′ This is the variance scheduling function.

[0059] The main augmentation model is optimized using a conditional flow matching loss function and reparameterized sampling is performed, wherein continuous time... t intermediate spectrum x t for: x t = m t + s t ′ × x ,in, x Standard Gaussian noise, u t The mean of the conditional probability path; s t ′ This is the variance scheduling function. It sets the total number of iterations. N It can be fixed (e.g.) N =5 or N = 10) Alternatively, the adaptive total number of iterations can be used. N′ .

[0060] Step S4 (Main Augmentation Model): The main augmentation model uses a vector field network. V θ In this embodiment, the total number of iterations is set. N′ Given a value of 10, generate a discrete time point sequence in the interval [0,1]. t 0=0< t 1< t 2<...< t N =1. In each iteration, the vector field network receives the current state. Noisy speech Y and by t ( t i-1 ; c Modulated time query, while vector field network V θ The received parameters are t ( ti-1 ; c This represents the "perception time" corresponding to that physical point in time. It predicts the update direction for the current step. V pred , V pred The calculation formula is as follows:

[0061]

[0062] in, V θ It is a vector field network; t i The integration grid for an ODE solver (such as the Euler method or an equivalent numerical integration method) represents the progression of "physical time". t ( t i-1 ; c ) is the time reparameterization function for the current state. Then, through To perform a single-step update, after a total of [number] iterations... N A clean speech estimate can be generated after one iteration. This is denoted as the enhanced pure spectrum. X clean |

[0063] Step S5 (Signal Post-processing): Finally, perform signal post-processing to obtain the enhanced pure spectrum. X clean By combining the original noisy phase spectrum ∠Y with the reconstructed enhanced complex spectrum and performing the inverse short-time Fourier transform (ISTFT), the final clean enhanced speech can be obtained.

[0064] During the training phase, the feature encoder, hypernetwork, adaptive probabilistic path generator, and main augmentation model are jointly trained end-to-end, optimized using a conditional flow matching loss function. During training, intermediate states are sampled using a reparameterization technique, where continuous time... t intermediate spectrum x t The mean of the conditional probability path m t and variance scheduling function s t ′ Decision, and m t Then by the time reparameterization function t ( t ; c It is obtained by linearly combining noisy speech and clean speech.

[0065] Throughout the system implementation, the feature encoder, hypernetwork, adaptive probabilistic path generator, and main augmentation model are jointly trained and optimized end-to-end using a conditional flow matching loss function, where continuous time... t intermediate spectrum x t for: x t = m t + s t ′ × x The mean of the conditional probability path m t for: m t = t ( t ; c )× x clean +(1- t ( t ; c ))×Y, x Standard Gaussian noise, x clean Y represents the pure speech spectrum, and Y represents the noisy speech spectrum. s t ′ For variance scheduling function, t ( t ; c ) is the time reparameterization function.

[0066] When the system encounters low signal-to-noise ratio and non-steady-state noise, the feature encoder extracts corresponding strong noise features, and the supernetwork generates larger signals accordingly. c′ The value causes the variance to decay rapidly, while adjusting... α′ and β′ make t ( t ; c The adaptive adjustment—with slow initial changes and rapid convergence later—ensures strong noise suppression in the initial stages and rapid convergence in the later stages to preserve speech details. Figure 2 As shown in the comparison, it can be seen that the method of this application has a relative advantage in speech enhancement. The audio waveform enhanced by the method of this application is clearer, the noise and unclear components in the speech spectrogram are significantly reduced, and the speech signal is more focused and clear.

[0067] like Figure 3The comparison chart showing the effects of different methods on the audio data spectrum is compared using a 140-hour audio validation set collected from the business side. It shows that, with the same number of iterations, this embodiment achieves a Perceptual Speech Quality (PESQ) score of 1.722, an improvement of approximately 12.26% compared to the fixed linear path method; a Short-Time Objective Intelligibility (STOI) score of 0.798, an improvement of approximately 7.69% compared to the fixed linear path method; and a Scale-Invariant Signal-to-Noise Ratio (SI_SNR) score of 0.813 dB compared to the fixed linear path method. Furthermore, it exhibits significantly stronger robustness to noise types not present in the training data. This verifies the technical advantages of this embodiment in achieving sample-level adaptive path generation through a deeply coupled technical architecture, significantly improving the effect and performance of speech enhancement in complex noise environments.

[0068] Specifically, the specific components of the system in this application will be described in detail below. The function of the feature encoder is to receive the amplitude spectrum of noisy speech Y and extract a high-dimensional noise feature vector that can characterize the noise characteristics and scene information. c This vector can explicitly or implicitly encode noise type, signal-to-noise ratio (SNR), non-stationarity, reverberation level, etc., and is used as a conditional variable for subsequent adaptive probability path generation. The input of the feature encoder receives the amplitude spectrum |Y| separated after the noisy speech Y undergoes a short-time Fourier transform (STFT), and the output is the noise feature vector. c Provided to hypernetworks.

[0069] The function of a hypernetwork is to receive noise feature vectors. c Dynamically generate the parameter set of the "sample-level adaptive probability path kernel". P Used to define time reparameterization functions t ( t ; c ) and variance scheduling function s t ′ To ensure feasibility and stability, the generated time reparameterization function... t ( t ; c The formula used is:

[0070]

[0071] in, α′ The initial shape parameters for the adaptive time path; β′ The shape parameter is the termination parameter for the adaptive time path; t For continuous time; x Let be the dummy variable for integration. The event reparameterization function is monotonically increasing on t∈[0,1] and satisfies ... t (0; c )=0、 t (1; c =1.

[0072] And variance scheduling function s t ′ The definition of is:

[0073]

[0074] in, s 0 ′ The initial noise scale is adaptive; c′ For adaptive variance decay shape parameters; t ( t ; c The function is a time-reparameterized function; it must be monotonically decreasing and satisfy the following conditions: s 0 ′ >0、 s 1 ′ =1. The input of the hypernetwork receives a noise feature vector. c The output terminal will display the parameter set. P Pass it to the adaptive probability path generator.

[0075] The function of the adaptive probabilistic path generator is based on a set of parameters. P Constructing a differentiable family of sample-level adaptive path functions: time reparameterization functions t ( t ; c Order-preserving parameterization is used to ensure... t It is monotonically increasing on t∈[0,1] and satisfies t (0; c )=0、 t (1; c =1.

[0076] Variance scheduling function s t ′ Using a monotonically increasing, endpoint-restricted positive definite construction, satisfying s 0 ′ >0、 s 1 ′ =1. During the training phase, the mean of the conditional probability path defined above. m t for: m t = t ( t ; c )× x clean +(1- t ( t ;c ))×Y, and perform reparameterization. x t = m t + s t ′ × x ;in, x Standard Gaussian noise, x clean Y represents the pure speech spectrum, and Y represents the noisy speech spectrum. s t ′ For variance scheduling function, t ( t ; c () is the time reparameterization function. During the inference phase, it is used... t ( t ; c Modulate the time query of the master enhancement model, and it is available. t i ′ Adaptive step size allocation is performed to achieve personalized trajectory integration. The adaptive probability path generator is used to receive the parameter set. P And output the time reparameterization function. t ( t ; c ) and variance scheduling function s t ′ The dynamic path-to-master enhancement model.

[0077] The main augmentation model's functionality is deeply coupled with the sample-level adaptive path, and the function is reparameterized over time. t ( t ; c Modulation time query and t i ′ Using an adaptive step size for finite-step integration, the noisy speech Y is progressively converted into a clean spectrum. X clean |, which receives noisy speech Y (as a condition) and the time reparameterization function output by the adaptive probabilistic path generator. t ( t ; c ) and variance scheduling function s t ′ And optional adaptive step size, and according to the number of adaptive step sizes N′ Perform iterative enhancement, and output the enhanced pure spectrum. X clean |

[0078] Compared to existing technologies such as FlowSE, which use a fixed linear path (mean path) u t ( x 1, y )= t × x 1+(1- t )× y and standard deviation path Compared to existing technologies, this invention achieves a technological leap from "fixed path" to "adaptive path" through the synergistic effect of the aforementioned technical features, solving the fundamental defect that existing technologies cannot perform personalized enhancement based on noise characteristics. Especially in challenging scenarios such as low signal-to-noise ratio and non-steady-state noise, this invention can achieve an intelligent denoising strategy of "suppressing strong noise in the early stage and preserving speech details in the later stage" by dynamically adjusting the path shape and variance scheduling.

[0079] Although the specification has provided a detailed description, it should be understood that various changes, substitutions, and modifications can be made without departing from the spirit and scope of the invention as defined by the appended claims. Furthermore, the specific embodiments described are not intended to limit the scope of the invention, and those skilled in the art will readily understand based on this invention that existing or future-developed processes, machines, manufactures, compositions of matter, means, methods, or steps can perform substantially the same functions or achieve substantially the same results as the embodiments of the invention. Therefore, the appended claims are intended to include such processes, machines, manufactures, compositions of matter, means, methods, or steps within their scope.

Claims

1. A speech enhancement method based on stream-matched sample-level adaptive paths, characterized in that: Includes the following steps: S1: Perform a short-time Fourier transform (STFT) on the noisy speech Y to obtain the noisy amplitude spectrum |Y| and the noisy phase spectrum ∠Y; S2: Extract the noise feature vector of the noisy amplitude spectrum |Y| using a feature encoder. c ; S3: Based on the noise feature vector through a hypernetwork c Generate adaptive path parameter set P ; S4: Based on the parameter set, an adaptive probability path generator is used. P Constructing a time reparameterization function τ ( t ; c ) and variance scheduling function σ t ′ ,in, t For continuous time; τ ( t ; c )exist t Monotonically increasing on [0,1], and satisfying τ (0; c )=0、 τ (1; c )=1; σ t ′ exist t It is monotonically decreasing on [0,1] and satisfies σ 0 ′ >0、 σ 1 ′ =1; S5: Using the noisy amplitude spectrum |Y| as the initial state through the main enhancement model, and based on the adaptive step size output by the supernetwork... t i ′ and adaptive step count N′ Generate time-point sequences and use a vector field network at each step. V θ based on τ ( t ; c The modulated time query predicts the update direction to perform numerical integration, thereby adjusting the time reparameterization function. τ ( t ; c The determined time point sequence and the variance scheduling function σ t ′ Iterative enhancement is performed to obtain the enhanced pure spectrum. X clean |; S6: The purified spectrum | X clean | Combine the noisy phase spectrum ∠Y with the synthesized enhanced complex spectrum, and perform inverse short-time Fourier transform (ISTFT) to output enhanced speech.

2. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 1, characterized in that: The feature encoder employs a convolutional neural network, which sequentially includes: an initial convolutional layer using a 3×3 convolutional kernel to map the noisy amplitude spectrum of a single channel to 64 channels; multiple residual blocks, consisting of four residual blocks connected in series, each containing two 3×3 convolutional layers and integrating channel attention and dilated convolutions; a temporal aggregation convolutional layer using a 1×3 convolutional kernel to aggregate along the time axis and adjust the number of channels to 128; a time-frequency attention pooling layer to obtain a global representation from the output of the temporal aggregation convolutional layer; and a fully connected layer that maps the global representation output by the time-frequency attention pooling layer to a fixed-dimensional noise feature vector. c ; wherein, the noise feature vector c The noise type, signal-to-noise ratio, and non-stationary characteristics of the noisy amplitude spectrum are encoded.

3. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 1, characterized in that: The supernetwork employs a multilayer perceptron to output the parameter set. P The parameter set P Including adaptive time path initial shape parameters α′ Adaptive time path termination shape parameters β′ Adaptive initial noise scale σ 0 ′ and adaptive variance decay shape parameters γ′ ,in α′ >0 and β′ >0 is used to control the shape of the time reparameterization function. σ 0 ′ >0 represents the initial noise scale. γ′ >0 is used to control the variance scheduling function. σ t ′ The rate of decay.

4. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 1, characterized in that: The time reparameterization function τ ( t ; c The formula for ) is: in, α′ The initial shape parameters for the adaptive time path; β′ The shape parameter is the termination parameter for the adaptive time path; t For continuous time; x It is an integral dummy variable.

5. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 1, characterized in that: The variance scheduling function σ t ′ The formula is: in, σ 0 ′ The initial noise scale is adaptive; γ′ For adaptive variance decay shape parameters; τ ( t ; c ) is the time reparameterization function.

6. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 1, characterized in that: The feature encoder, supernetwork, adaptive probabilistic path generator, and main enhancement model are jointly trained end-to-end to collaboratively achieve noise-adaptive path generation and speech enhancement.

7. The speech enhancement method for stream-matched sample-level adaptive paths as described in claim 6, characterized in that: The end-to-end joint training optimizes the main augmentation model using a conditional flow matching loss function, which includes: constructing an intermediate spectrum. x t for: x t = μ t + σ t ′ × ξ ;in, t For continuous time, the mean of the conditional probability path μ t for: μ t = τ ( t ; c )× x clean +(1- τ ( t ; c ))×Y, ξ Standard Gaussian noise, x clean Y represents the pure speech spectrum, and Y represents the noisy speech spectrum. σ t ′ For variance scheduling function, τ ( t ; c ) is the time reparameterization function.

8. A system employing the speech enhancement method as described in any one of claims 1-7, characterized in that, include: A feature encoder is used to receive a noisy amplitude spectrum and output a noisy feature vector. c ; A hypernetwork, connected to the feature encoder, is used to base the noise feature vector on... c Generate adaptive path parameters P ; An adaptive probabilistic path generator, connected to the hypernetwork, is used to generate paths based on the parameters. P Constructing a time reparameterization function τ ( t ; c ) and variance scheduling function σ t ′ ,in τ ( t ; c The expression is monotonically increasing on t∈[0,1] and satisfies τ (0; c )=0、 τ (1; c )=1; σ t ' It is monotonically decreasing on t∈[0,1] and satisfies σ 0 ′ >0、 σ 1 ′ =1; The main augmentation model, connected to the adaptive probabilistic path generator, is used to generate a path based on the noisy amplitude spectrum |Y| as the initial state and the adaptive step size output by the supernetwork. t i ′ and adaptive step count N′ Generate time-point sequences and use a vector field network at each step. V θ based on τ ( t ; c The modulated time query predicts the update direction to perform numerical integration, thereby based on... τ ( t ; c )and σ t ′ Iterative enhancement of the noisy amplitude spectrum |Y| is performed to output the enhanced pure spectrum. X clean |; The signal processing module, connected to the main enhancement model, is used to process the pure spectrum. X clean The enhanced complex spectrum is synthesized with the noisy phase spectrum ∠Y, and then the enhanced speech is output after performing the inverse short-time Fourier transform (ISTFT).

9. The system as described in claim 8, characterized in that, The feature encoder includes a convolutional neural network with multiple residual blocks and a channel attention mechanism for extracting noisy feature vectors. c The dimension is 192.

Images