Speech enhancement noise reduction system based on generative adversarial network

Abstract

The invention discloses a speech enhancement and noise reduction system based on a generative adversarial network, and the system comprises a time-frequency alignment module which is used for collecting and preprocessing original noise speech; the three-component coding module is used for extracting voice content features, voiceprint features and noise characterization; the dual-domain collaborative generation module is used for generating candidate enhanced waveform frequency spectrums and performing collaborative correction; the multi-head discrimination module is used for generating a noise correction factor through a time domain discriminator, a frequency domain discriminator and an index discrimination head; the noise re-projection module is used for correcting noise representation, generating heavy noise voice and obtaining a noise self-adaptive enhancement result set; and the consistency reconstruction module is used for spectrum consistency correction and phase reconstruction. According to the method, the double-domain three-decoupling VoiceGAN is used for voice enhancement and noise reduction, and the method has the advantages of being high in naturalness, high in intelligibility and good in generalization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech processing, and in particular to a speech enhancement and noise reduction system based on a generative adversarial network. BACKGROUND

[0002] The existing speech enhancement and noise reduction technology mainly includes traditional signal processing methods and deep learning methods. The traditional methods such as spectral subtraction, Wiener filtering, and short-time least mean square error estimation rely on the assumption of noise statistical characteristics and point-by-point correction of amplitude spectrum, which is easy to produce "musical noise", excessive smoothing and loss of speech details, and has poor adaptability to non-stationary noise and complex sound field. The deep learning method uses DNN, CNN, LSTM or attention model to predict mask or amplitude gain in the frequency domain, and there is also an end-to-end time domain method to directly regress waveform; but the frequency domain path often ignores the phase, causing reconstruction distortion and limited perceptual quality, and the time domain path is difficult to explicitly constrain the harmonic and formant structure, and the cross-scene generalization ability is limited.

[0003] In order to alleviate the above problems, adversarial learning has been introduced into speech enhancement, and representative works can improve naturalness to a certain extent, but most methods only generate and discriminate in a single domain, lacking of time and frequency domain collaborative constraints; at the same time, the common framework does not explicitly separate the speech content, speaker voiceprint and environmental noise, which leads to the speaker features being easily weakened or coupled with noise during the enhancement process, affecting intelligibility and consistency. Further, the existing methods mostly indirectly approximate the perceptual indicators by loss function, lack of discriminant channels for objective and subjective evaluation, and are difficult to feedback the enhancement bias in time; also, there is no closed-loop verification mechanism of "enhancement-noise projection-alignment comparison", which cannot use the re-projection residual to adaptively correct the noise representation. In summary, the existing technology has deficiencies in the aspects of dual-domain collaboration, three-component decoupling, multi-head perceptual discrimination, noise re-projection adaptation, and spectrum consistency and phase joint reconstruction, which makes it difficult to ensure clarity, naturalness and speaker consistency in complex non-stationary noise environment.

[0004] Therefore, how to provide a speech enhancement and noise reduction system based on a generative adversarial network is a problem that those skilled in the art need to solve. SUMMARY

[0005] One object of the present application is to provide a speech enhancement and noise reduction system based on a generative adversarial network. The present application uses a dual-domain three-decoupling VoiceGAN for speech enhancement and noise reduction, which has the advantages of high naturalness, strong intelligibility and good generalization.

[0006] According to the speech enhancement and noise reduction system based on the generative adversarial network of the embodiment of the present application, the system comprises: a time-frequency alignment module configured to collect original noisy speech, uniformly sample, perform endpoint detection, frame, and amplitude normalize the original noisy speech, construct time-domain waveform blocks and spectral amplitude and phase representations, and generate a time-frequency joint representation set; a three-component encoding module configured to input the time-frequency joint representation set into a three-component encoding layer of a dual-domain three-decoupled VoiceGAN network, extract speech content features, voiceprint features, and noise representations, and form a feature fusion input set; a dual-domain collaborative generation module configured to input the feature fusion input set into a time-domain generator and a frequency-domain generator, respectively, obtain candidate enhanced waveforms and candidate enhanced spectrums, and form a candidate enhanced result set through collaborative correction; a multi-head discrimination module configured to input the candidate enhanced result set into a time-domain discriminator, a frequency-domain discriminator, and an index discrimination head, and generate a noise correction factor; a noise re-projection module configured to correct the noise representations according to the noise correction factor, input the candidate enhanced result set and the corrected noise representations into a noise re-projection device to generate re-noisy speech, and align and compare the re-noisy speech with the original noisy speech to obtain a noise adaptive enhancement result set; a consistency reconstruction module configured to perform spectral consistency correction and phase reconstruction on the noise adaptive enhancement result set, obtain a final enhanced waveform and a final enhanced spectrum aligned with an original time axis, and output an enhanced speech signal.

[0007] Optionally, the modules are implemented through the following methods: original noisy speech is collected, uniformly sampled, endpoint detected, framed, and amplitude normalized, time-domain waveform blocks and spectral amplitude and phase representations are constructed, and a time-frequency joint representation set is formed; a dual-domain three-decoupled VoiceGAN network is constructed, the time-frequency joint representation set is input into a three-component encoding layer, speech content features, voiceprint features, and noise representations are extracted through a content encoder, a voiceprint encoder, and a noise encoder, respectively, and a feature fusion input set is obtained through integration; the feature fusion input set is input into a time-domain generator and a frequency-domain generator, respectively, candidate enhanced waveforms and candidate enhanced spectrums are generated, and a candidate enhanced result set is formed through collaborative correction; the candidate enhanced result set is input into a time-domain discriminator, a frequency-domain discriminator, and an index discrimination head in the dual-domain three-decoupled VoiceGAN network, and a noise correction factor is generated based on a discrimination result; the noise representations are corrected according to the noise correction factor to obtain corrected noise representations, the candidate enhanced result set and the corrected noise representations are input into a noise re-projection device to generate re-noisy speech, the re-noisy speech is aligned and compared with the original noisy speech, and a noise adaptive enhancement result set is obtained; Performing spectral consistency correction and phase reconstruction on the noise-adaptive enhancement result set, obtaining the final enhanced waveform and the final enhanced spectrum aligned with the original time axis, and outputting the enhanced speech signal.

[0008] Optionally, the generation of the time-frequency joint representation set specifically comprises: Collecting the original noisy speech, performing unified sampling and channel regularization, merging the multi-channel speech into a single-channel speech in a power-weighted manner, and unifying to a target sampling rate and quantization precision to obtain an original sample sequence; Performing endpoint detection on the original sample sequence, calculating the short-time energy and the zero-crossing rate frame by frame, wherein the short-time energy is defined as the sum of squares of sample values in the frame, and the zero-crossing rate is defined as half the number of adjacent sample sign changes, and removing the silent section and the long-time invalid section according to the joint threshold to obtain an effective speech segment sequence; Performing frame division and frame shift on the effective speech segment sequence, and performing windowing processing using the cosine-weighted Hanning window to obtain a standardized speech frame sequence; Performing amplitude normalization on the standardized speech frame sequence, and performing scale compression based on the peak amplitude of each frame to obtain an amplitude-normalized frame-level sequence; Splicing the amplitude-normalized frame-level sequence in a fixed block length and a fixed block shift on the time axis to construct a time-domain waveform block sequence, establishing a correspondence between each time-domain waveform block and a frame index, and forming the time-domain waveform block and its time mapping index; Calculating the short-time Fourier transform of the standardized speech frame sequence to obtain a frame-frequency complex spectrum organized by frame and frequency point, which maintains consistency with the corresponding frame index; Decomposing the frame-frequency complex spectrum to obtain spectral amplitude and phase representations, and aligning them with the time-domain waveform blocks on the same time axis to form a corresponding relationship between the frequency domain and the time domain, and generating a time-frequency joint representation set.

[0009] Optionally, the generation of the feature fusion input set specifically comprises: Constructing a dual-domain three-decoupling VoiceGAN network composed of a three-component encoding layer, a dual-domain collaborative generator, a three-head discriminator, a noise reprojector, and a consistency correction module; The three-component encoding layer includes a content encoder, a voiceprint encoder, and a noise encoder, the dual-domain collaborative generator is composed of a time-domain generator and a frequency-domain generator, and the three-head discriminator includes a time-domain discriminator, a frequency-domain discriminator, and an index discriminator head. The time-frequency joint representation set is input into the content encoder to perform frame-level convolution extraction and context aggregation and time alignment, the frame-level acoustic features are averaged within a fixed-width aggregation window centered on the target time position to obtain speech content features covering the entire time axis; The time-frequency joint representation set is sent to a voiceprint encoder, frame-level feature extraction and statistical pooling and normalization are sequentially performed, the mean and standard deviation of the frame-level features on the coverage time axis are calculated and norm normalization is performed, and a voiceprint feature is obtained; The time-frequency joint representation set is sent to a noise encoder, the minimum value of the spectral amplitude of each frequency band in a given time window is taken to form a noise amplitude estimation, and the noise representation varying with time is obtained by interpolation of the frequency band to the time position; The speech content feature, the voiceprint feature and the noise representation are aligned and scaled according to the time index, and spliced to obtain a component alignment result sequence; The component alignment result sequence is executed channel rearrangement and block-level caching to maintain a corresponding relationship with the time domain waveform block and the spectral amplitude and phase representation, and is archived in the frame block dimension to generate a feature fusion input set.

[0010] Optionally, the generation of the candidate enhancement result set specifically includes: The feature fusion input set is sent to a time domain generator composed of causal one-dimensional convolution, hollow convolution and residual connection stacking to obtain a waveform residual equal in length to the time domain waveform block, the waveform residual and the corresponding time domain waveform block are added to generate a candidate enhanced waveform; The feature fusion input set is sent to a frequency domain generator, a two-dimensional convolution encoder and decoder are combined with cross-layer jump connection for forward calculation to obtain an amplitude ratio mask and a phase increment candidate with the same size as the spectral amplitude, the amplitude ratio mask is applied to the spectral amplitude to obtain a candidate enhanced amplitude, and the phase increment candidate is superimposed on the phase representation to obtain a candidate phase representation, which are combined to form a candidate enhanced spectrum; The phase continuity of the candidate phase representation of the candidate enhanced spectrum is processed in the frame block dimension, the phase change of adjacent frame blocks is calculated and phase wrapping is unfolded to obtain a phase consistency correction amount, the phase consistency correction amount is applied to the candidate phase representation to obtain a phase sequence after phase consistency correction, and the candidate phase representation of the candidate enhanced spectrum is updated to obtain the candidate enhanced spectrum after phase consistency correction; The energy measure of the candidate enhanced waveform and the candidate enhanced amplitude is calculated in the frame block dimension, and the candidate enhanced waveform is scaled in amplitude based on the energy of the candidate enhanced amplitude to obtain a candidate enhanced waveform after energy conservation correction, and the energy registration parameter for amplitude scaling is recorded to obtain the candidate enhanced waveform after energy conservation correction; The candidate enhanced waveform after energy conservation correction and the candidate enhanced spectrum after phase consistency correction are indexed and aligned in the frame block dimension, the matching of the corresponding frame blocks in time consistency and energy consistency is verified according to the energy registration parameter, and a collaborative correction alignment result containing aligned waveform and aligned spectrum is obtained by combination, which is archived by frame block to generate a candidate enhancement result set.

[0011] Optionally, the generation of the noise correction factor specifically comprises: receiving a candidate enhancement result set, unpacking to obtain a candidate enhancement waveform sequence, a candidate enhancement spectrum sequence, and a frame block index, a time index, and an energy registration parameter corresponding thereto, establishing a discriminant input batch and a reference index table according to the frame block, for parallel discrimination of a time domain discriminator, a frequency domain discriminator, and an index discriminator head, to obtain aligned input for each discriminator; inputting the aligned waveform in the candidate enhancement result set into the time domain discriminator by frame block, outputting a time domain true-false score sequence, and obtaining a time domain adversarial residual sequence by subtracting the corresponding time domain true-false score from one, to form a time domain discrimination result containing the time domain true-false score and the time domain adversarial residual; inputting the aligned spectrum in the candidate enhancement result set into the frequency domain discriminator by frame block, outputting a frequency domain true-false score sequence and a spectral structure consistency score sequence, and obtaining a frequency domain adversarial residual sequence by subtracting the corresponding frequency domain true-false score from one, to form a frequency domain discrimination result containing the frequency domain true-false score, the spectral structure consistency score, and the frequency domain adversarial residual; inputting the candidate enhancement result set into the index discriminator head, outputting a frame block level perceptual index prediction vector, and obtaining an index deviation by subtracting the perceptual index prediction vector from the reference index target vector, to form an index discrimination result containing the perceptual index prediction vector and the index deviation; weighting and fusing the time domain adversarial residual, the frequency domain adversarial residual, and the index deviation, normalizing the weights of the time domain channel, the frequency domain channel, and the index channel on each frame block so that they add up to one, multiplying and summing the normalized weights to obtain a frame block level synthesis feedback vector, generating a frame block level consistency score based on the spectral structure consistency score, and forming a fusion result containing the frame block level synthesis feedback vector and the frame block level consistency score; According to the fusion result, the frame block level consistency score is used as a gate to amplify or suppress the frame block level synthesis feedback vector on the time axis, the energy distribution of the aligned spectrum in each frequency band and the contribution ratio of the spectral structure consistency score in the frequency band dimension are combined, the gated synthesis feedback vector is mapped to a weight map in the time and frequency band dimensions, and sliding window smoothing and boundary constraint are performed on the weight map, different upper limits are set for silent and speech regions, and transition processing is performed on the mutation position, to obtain a constrained weight map. Align the noise representation in the time and frequency band dimensions, extract the frame block level global factor and the frequency band level refined factor, and form the noise correction factor by weighted combination.

[0012] Optionally, the generation of the noise adaptive enhancement result set specifically comprises: performing weighted correction on the noise representation in the frame block dimension and the frequency band dimension according to the noise correction factor, to generate a corrected noise representation; The noise-reprojection input is constructed, the candidate enhanced spectrum is spliced with the corrected noise representation on the time axis and the frequency band axis to form a frequency domain joint-reprojection feature, the candidate enhanced waveform is bound with the energy registration parameter on the time axis to form a time domain re-projection auxiliary information, and a re-projection input set is obtained by merging; The re-projection input set is sent to a noise re-projection device, noise re-injection and mixing, phase promotion and amplitude constraint, and time domain consistency generation process are performed in the order of frame blocks, and a re-noisy speech sequence corresponding to the frame block index is output; Two-stage alignment is performed on the re-noisy speech sequence and the original noise speech, coarse alignment anchors are established according to the endpoint position and the energy peak value in the whole section, and then fine alignment is completed in the frame block according to the short-time window to generate a time alignment mapping table; The re-noisy speech sequence and the original noise speech are compared frame by frame and frequency band by frequency band according to the time alignment mapping table to obtain an amplitude difference distribution, a phase difference distribution and an instantaneous signal-to-noise ratio increment distribution, and a re-projection comparison residual graph and a noise fitting degree graph are formed by merging; The re-projection comparison residual graph is used to perform selective rollback and refinement correction on the candidate enhanced spectrum and the candidate enhanced waveform, the residual high area is rolled back and the amplitude is constrained according to the energy registration parameter, and the residual low area is retained or locally enhanced to obtain a noise adaptive enhancement result set, including noise adaptive enhanced spectrum and noise adaptive enhanced waveform.

[0013] Optionally, the generation of the enhanced speech signal specifically includes: The amplitudes and phases of the noise adaptive enhanced spectrum in the noise adaptive enhancement result set are combined at corresponding frame blocks and frequency band positions to construct an initial complex spectrum, phase reconstruction iteration is performed under the constraint of the time alignment mapping table, inverse short-time Fourier transform is performed on the initial complex spectrum to obtain a temporary time domain signal, new complex spectrum is obtained by performing short-time Fourier transform on the temporary time domain signal, and a phase iteration update sequence is obtained by cycling; The phase iteration update sequence is combined with a preset target amplitude, and is projected into a spectrum consistency space through time domain reconstruction and secondary transformation, in-band smoothing and amplitude upper limit constraint are performed on the high residual frequency band according to the re-projection comparison residual graph, and the current amplitude and phase are maintained for the low residual frequency band, to form a spectrum consistency corrected complex spectrum; The noise adaptive enhanced waveform and the spectrum consistency corrected complex spectrum are registered bidirectionally according to the energy registration parameter, the amplitude is scaled at the frame block level in the frequency domain to meet the energy conservation, the waveform is adjusted at the frame block level in the time domain, and the overlap-and-add consistent with the frame division is adopted to splice to obtain an energy and boundary alignment result, and the continuity of the adjacent frame block start and end overlap area is verified through the time alignment mapping table; Perform phase convergence judgment and harmonic refinement on the energy and boundary alignment results, freeze the phase when the phase change is below the convergence threshold, re-perform phase iteration update when the phase change is above the convergence threshold, enhance harmonic peaks and rectified envelope according to the fundamental frequency correlation in the speech area, suppress residual noise stripes and isolated spectral peaks in the silent area, and generate the final enhanced spectrum; Perform inverse short-time Fourier transform on the final enhanced spectrum and complete time domain reconstruction by windowing and overlapping to obtain the final enhanced waveform aligned with the original time axis, and archive the final enhanced waveform and the final enhanced spectrum according to frame blocks and output the enhanced speech signal.

[0014] The beneficial effects of the present application are: The present application introduces a dual-domain three-decoupling VoiceGAN network, forming a collaborative generation and collaborative constraint in time domain and frequency domain, and cooperating with a three-component coding layer to explicitly separate speech content features, voiceprint features and noise representation, overcoming the problems of excessive smoothing, speaker feature weakening and phase distortion caused by existing single-domain processing and feature coupling. Through the multi-head discrimination mechanism, the noise correction factor is generated and the noise re-projector is driven to complete the closed-loop adaptation of "enhancement-re-noise-alignment comparison", so that the noise representation can be dynamically corrected according to the scene, thereby significantly improving the robustness and generalization ability in non-stationary, multi-source and complex reverberation noise environments. Further, after the candidate enhanced result is corrected by phase continuity and energy registration, it enters the consistency reconstruction module, which outputs the final enhanced waveform and the final enhanced spectrum aligned with the original time axis under the joint constraint of spectral consistency correction and phase reconstruction, effectively reducing musical noise and artifacts, maintaining harmonic structure and formant details, balancing intelligibility and naturalness, and reducing boundary splicing distortion under the constraint of full-link index alignment and energy conservation. Overall, while ensuring real-time performance, the present application realizes comprehensive improvement of speech intelligibility, subjective listening and speaker consistency, and significantly improves objective quality and intelligibility indicators. The system can be used as a general enhancement base for speech recognition front-end, call and conference, vehicle voice interaction, and hearing aid devices. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application and are used to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0016] Fig. 1 A flowchart of a speech enhancement and noise reduction system based on a generative adversarial network is proposed for the present application; Fig. 2 A dual-domain three-decoupling VoiceGAN network diagram of a speech enhancement and noise reduction system based on a generative adversarial network is proposed for the present application. DETAILED DESCRIPTION

[0017] The application will now be described in further detail with reference to the drawings. These drawings are simplified schematic illustrations of the basic structure of the application and therefore only show the components relevant to the application.

[0018] Reference Figs. 1-2 A speech enhancement and noise reduction system based on a generative adversarial network, comprising: A time-frequency alignment module for collecting original noisy speech and performing unified sampling, endpoint detection, framing, and amplitude normalization, constructing time-domain waveform blocks and spectral amplitude and phase representations, and generating a time-frequency joint representation set; A three-component encoding module for inputting the time-frequency joint representation set into a three-component encoding layer of a dual-domain three-decoupled VoiceGAN network, extracting speech content features, voiceprint features, and noise representations, and forming a feature fusion input set; A dual-domain collaborative generation module for inputting the feature fusion input set into a time-domain generator and a frequency-domain generator, respectively, to obtain candidate enhanced waveforms and candidate enhanced spectra and form a candidate enhanced result set through collaborative correction; A multi-head discrimination module for inputting the candidate enhanced result set into a time-domain discriminator, a frequency-domain discriminator, and an index discrimination head to generate a noise correction factor; A noise re-projection module for correcting noise representations based on the noise correction factor, inputting the candidate enhanced result set and the corrected noise representations into a noise re-projection device to generate re-noisy speech and align and compare it with the original noisy speech, and obtaining a noise-adaptive enhanced result set; A consistency reconstruction module for performing spectral consistency correction and phase reconstruction on the noise-adaptive enhanced result set to obtain a final enhanced waveform and a final enhanced spectrum aligned with the original time axis and output an enhanced speech signal.

[0019] In this embodiment, the modules are implemented through the following methods: Original noisy speech is collected, and unified sampling, endpoint detection, framing, and amplitude normalization are performed on the original noisy speech to construct time-domain waveform blocks and spectral amplitude and phase representations and form a time-frequency joint representation set; A dual-domain three-decoupled VoiceGAN network is constructed, and the time-frequency joint representation set is input into a three-component encoding layer to extract speech content features, voiceprint features, and noise representations through a content encoder, a voiceprint encoder, and a noise encoder, respectively, and integrate a feature fusion input set; The feature fusion input set is input into a time-domain generator and a frequency-domain generator, respectively, to generate candidate enhanced waveforms and candidate enhanced spectra, respectively, and form a candidate enhanced result set through collaborative correction; The candidate enhancement result set is input into a time domain discriminator, a frequency domain discriminator and an index discriminator head in a dual-domain three-decoupling VoiceGAN network, and a noise correction factor is generated based on a discrimination result; The noise representation is corrected according to the noise correction factor to obtain a corrected noise representation, the candidate enhancement result set and the corrected noise representation are input into a noise re-projector to generate re-noised speech, and the re-noised speech and the original noisy speech are aligned and compared to obtain a noise adaptive enhancement result set; Spectral consistency correction and phase reconstruction are performed on the noise adaptive enhancement result set to obtain a final enhancement waveform and a final enhancement spectrum aligned with an original time axis, and an enhanced speech signal is output.

[0020] In the embodiment, the generation of the time-frequency joint representation set specifically includes: The original noisy speech is collected, unified sampling and channel regularization are performed, multi-channel speech is combined into single-channel speech in a power weighting manner, and is unified to a target sampling rate and quantization precision to obtain an original sample sequence; Endpoint detection is performed on the original sample sequence, short-time energy and zero-crossing rate are calculated frame by frame, wherein the short-time energy is defined as the sum of squares of sample values in the frame, and the zero-crossing rate is defined as half of the number of times of sign changes of adjacent samples, and according to a joint threshold, silent segments and long-time invalid segments are removed to obtain an effective speech segment sequence; The effective speech segment sequence is subjected to framing and frame shifting, and a cosine weighted Hanning window is used for windowing to obtain a standardized speech frame sequence; The standardized speech frame sequence is subjected to amplitude normalization, and scale compression is performed with the peak amplitude of each frame as a reference to obtain an amplitude-normalized frame-level sequence; The amplitude-normalized frame-level sequence is spliced in a fixed block length and a fixed block shift on the time axis to construct a time-domain waveform block sequence, and a corresponding relationship between each time-domain waveform block and a frame index is established to form a time-domain waveform block and a time mapping index thereof; The short-time Fourier transform is calculated for the standardized speech frame sequence to obtain a frame-frequency complex spectrum organized by frame and frequency point, which is consistent with the corresponding frame index; The spectral amplitude and phase representation are obtained by decomposition from the frame-frequency complex spectrum, and are aligned on the same time axis with the time-domain waveform block to form a corresponding relationship between the frequency domain and the time domain, and a time-frequency joint representation set is generated.

[0021] In the embodiment, the generation of the feature fusion input set specifically includes: The dual-domain three-decoupling VoiceGAN network is constructed and consists of a three-component encoding layer, a dual-domain collaborative generator, a three-head discriminator, a noise re-projector and a consistency correction module; The three-component encoding layer includes a content encoder, a voiceprint encoder, and a noise encoder, the dual-domain collaborative generator is composed of a time-domain generator and a frequency-domain generator, and the three-head discriminator includes a time-domain discriminator, a frequency-domain discriminator, and an index discriminator head. The time-frequency joint representation set is sent to the content encoder, frame-level convolution extraction and context aggregation and time alignment are performed, frame-level acoustic features are averaged in a fixed-width aggregation window centered on a target time position, and speech content features covering the entire time axis are obtained. The time-frequency joint representation set is sent to the voiceprint encoder, frame-level feature extraction and statistical pooling and normalization are sequentially performed, the mean and standard deviation of the frame-level features covering the time axis are calculated and normalized, and the voiceprint features are obtained. The time-frequency joint representation set is sent to the noise encoder, the minimum value of the spectral amplitude of each frequency band in a given time window is taken to form a noise amplitude estimation, and the noise representation varying with time is obtained by interpolation from the frequency band to the time position. The speech content features, voiceprint features, and noise representation are aligned and scaled according to the time index and spliced to obtain a component alignment result sequence. The component alignment result sequence is subjected to channel rearrangement and block-level caching to maintain a corresponding relationship with the time-domain waveform block and the spectral amplitude and phase representation, and is archived in the frame block dimension to generate a feature fusion input set.

[0022] In the embodiment, the generation of the candidate enhancement result set specifically includes: The feature fusion input set is sent to the time-domain generator, which is composed of a causal one-dimensional convolution, a hollow convolution, and a residual connection stack, to obtain a waveform residual equal in length to the time-domain waveform block, the waveform residual is added to the corresponding time-domain waveform block to generate a candidate enhanced waveform; The feature fusion input set is sent to the frequency-domain generator, which performs forward calculation through a two-dimensional convolution encoder and decoder combined with cross-layer jumping, to obtain an amplitude ratio mask and a phase increment candidate equal in size to the spectral amplitude, the amplitude ratio mask is applied to the spectral amplitude to obtain a candidate enhanced amplitude, and the phase increment candidate is added to the phase representation to obtain a candidate phase representation, which are combined to form a candidate enhanced spectrum; The phase continuity of the candidate phase representation of the candidate enhanced spectrum is processed in the frame block dimension, the phase change of adjacent frame blocks is calculated and phase wrapping is unfolded to obtain a phase consistency correction amount, the phase consistency correction amount is applied to the candidate phase representation to obtain a phase sequence after phase consistency correction, and the candidate phase representation of the candidate enhanced spectrum is updated to obtain the candidate enhanced spectrum after phase consistency correction; The energy metric of the candidate enhanced waveform and the candidate enhanced amplitude is calculated in the frame block dimension respectively, the candidate enhanced waveform is scaled in amplitude based on the energy of the candidate enhanced amplitude, an energy-conserved corrected candidate enhanced waveform is obtained, an energy registration parameter used for amplitude scaling is recorded, and an energy-conserved corrected candidate enhanced waveform is obtained; The energy-conserved corrected candidate enhanced waveform and the phase-consistent corrected candidate enhanced spectrum are indexed and aligned in the frame block dimension, the matching of the corresponding frame blocks in time consistency and energy consistency is verified according to the energy registration parameter, and a collaborative correction alignment result containing the aligned waveform and the aligned spectrum is obtained by combination, which is archived by frame blocks to generate a candidate enhanced result set.

[0023] In the embodiment, the generation of the noise correction factor specifically includes: The candidate enhanced result set is received, and a candidate enhanced waveform sequence, a candidate enhanced spectrum sequence, and corresponding frame block indexes, time indexes and energy registration parameters are obtained by unpacking, a discrimination input batch and a reference index table are established by frame blocks, which are used for parallel discrimination of the time domain discriminator, the frequency domain discriminator and the index discriminator head, and an aligned input for each discriminator is obtained; The aligned waveform in the candidate enhanced result set is input into the time domain discriminator by frame blocks, a time domain authenticity score sequence is output, and a time domain adversarial residual sequence is obtained by subtracting the corresponding time domain authenticity score from one, thereby forming a time domain discrimination result containing the time domain authenticity score and the time domain adversarial residual; The aligned spectrum in the candidate enhanced result set is input into the frequency domain discriminator by frame blocks, a frequency domain authenticity score sequence and a spectral structure consistency score sequence are output, and a frequency domain adversarial residual sequence is obtained by subtracting the corresponding frequency domain authenticity score from one, thereby forming a frequency domain discrimination result containing the frequency domain authenticity score, the spectral structure consistency score and the frequency domain adversarial residual; The candidate enhanced result set is input into the index discriminator head, a frame block level perception index prediction vector is output, and an index deviation is obtained by subtracting the perception index prediction vector from the reference index target vector, thereby forming an index discrimination result containing the perception index prediction vector and the index deviation; The time domain adversarial residual, the frequency domain adversarial residual and the index deviation are weighted and fused, the weights of the time domain channel, the frequency domain channel and the index channel are normalized to one on each frame block, a frame block level synthesis feedback vector is obtained by multiplying and summing the normalized weights, a frame block level consistency score is generated based on the spectral structure consistency score, and a fusion result containing the frame block level synthesis feedback vector and the frame block level consistency score is formed; According to the fusion result, a frame block level consistency score is used as a gate to amplify or suppress the frame block level synthesized feedback vector on the time axis, the energy distribution of the aligned spectrum on each frequency band and the contribution ratio of the spectral structure consistency score in the frequency band dimension are combined, the synthesized feedback vector after gating is mapped into a time and frequency band two-dimensional weight map, and sliding window smoothing and boundary constraint are performed on the weight map, different upper limits are set for the silent area and the speech area, and transition processing is performed on the mutation position to obtain a constrained weight map which is aligned with the noise representation in the time and frequency band dimensions, and a frame block level global factor and a frequency band level refined factor are extracted to form a noise correction factor by weighted combination.

[0024] In the embodiment, the generation of the noise adaptive enhancement result set specifically includes: According to the noise correction factor, weighted correction is performed on the noise representation in the frame block dimension and the frequency band dimension to generate a corrected noise representation; A noise re-projection input is constructed, the candidate enhancement spectrum and the corrected noise representation are aligned and spliced on the time axis and the frequency band axis to form a frequency domain joint re-projection feature, the candidate enhancement waveform and the energy registration parameter are aligned and bound on the time axis to form time domain re-projection auxiliary information, and a re-projection input set is obtained by merging; The re-projection input set is sent to the noise re-projection device, and the noise re-injection and mixing, phase advance and amplitude constraint, and time domain consistency generation process are performed in the frame block order, and a re-noise speech sequence corresponding to the frame block index is output; Two-stage alignment is performed on the re-noise speech sequence and the original noise speech, coarse alignment anchor points are established according to the endpoint position and the energy peak in the whole section, and then fine alignment is completed in the short-time window in the frame block to generate a time alignment mapping table; According to the time alignment mapping table, the re-noise speech sequence and the original noise speech are compared frame by frame and frequency band by frequency band to obtain an amplitude difference distribution, a phase difference distribution and an instantaneous signal-to-noise ratio increment distribution, and a re-projection comparison residual graph and a noise fitting degree graph are formed by summarizing; The re-projection comparison residual graph is used to perform selective rollback and refined correction on the candidate enhancement spectrum and the candidate enhancement waveform, the residual high area is rolled back and the amplitude is constrained according to the energy registration parameter, and the residual low area is retained or locally enhanced to obtain a noise adaptive enhancement result set including noise adaptive enhancement spectrum and noise adaptive enhancement waveform.

[0025] In the embodiment, the generation of the enhanced speech signal specifically includes: According to the amplitude and phase of the noise-adaptive enhanced spectrum in the noise-adaptive enhanced result set, the initial complex spectrum is constructed by combining the corresponding frame blocks and frequency band positions, phase reconstruction iteration is performed under the constraint of the time alignment mapping table, the initial complex spectrum is subjected to inverse short-time Fourier transform to obtain a temporary time-domain signal, and the temporary time-domain signal is subjected to short-time Fourier transform to obtain a new complex spectrum, and a phase iteration update sequence is obtained by circulation; The phase iteration update sequence is combined with a preset target amplitude, and is projected into a spectral consistency space through time-domain reconstruction and secondary transformation, the high residual band is subjected to in-band smoothing and amplitude upper limit constraint according to the residual error map obtained by re-projection, and the current amplitude and phase are maintained for the low residual band, to form a complex spectrum after spectral consistency correction; According to the energy registration parameter, the noise-adaptive enhanced waveform and the complex spectrum after spectral consistency correction are subjected to bidirectional registration, the amplitude is scaled at the frame block level in the frequency domain to satisfy the energy conservation, the waveform is subjected to frame block level amplitude fine tuning in the time domain and is subjected to overlap addition consistent with the frame division, and the energy and boundary alignment result is obtained, and the continuity of the adjacent frame block start and end overlap area is verified through the time alignment mapping table; The energy and boundary alignment result is subjected to phase convergence determination and harmonic refinement, the phase is frozen when the phase change is lower than the convergence threshold, and the phase iteration update is re-performed when the phase change is higher than the convergence threshold, the harmonic peak and the rectified envelope are enhanced in the speech area according to the fundamental frequency correlation, and the residual noise stripe and the isolated spectral peak are suppressed in the silent area, to generate a final enhanced spectrum; The final enhanced spectrum is subjected to inverse short-time Fourier transform and is subjected to windowing and overlap to complete time-domain reconstruction, to obtain a final enhanced waveform aligned with the original time axis, the final enhanced waveform and the final enhanced spectrum are archived according to the frame blocks, and an enhanced speech signal is output.

[0026] Embodiment 1 In order to verify the feasibility of the application in implementation, the application is applied to the intercom and broadcast system of a large urban transportation hub. The test environment covers various sound fields such as waiting hall, platform edge, transfer channel and ground entrance, including continuous wind noise, wideband noise generated by vehicle entering the station, superimposed noise of dense crowd talking, and broadcast echo and space reverberation. The collection time spans different time periods in the summer of 2025, including morning peak and night low peak; the location is distributed in the ground and underground scenes of the same operating unit to ensure the difference of sound source, device arrangement and reflection condition. The data used for verification consists of intercom recordings and broadcast segments authorized by the operating unit, including multiple speakers, different microphone distances and various types of sound pickup devices, and the environmental background track and scene annotation are recorded synchronously to reproduce the on-site conditions.

[0027] In this scenario, the application accesses the existing audio link in a popular and available way: the microphone array output first enters the time-frequency alignment module, after uniform sampling, endpoint detection, framing and amplitude normalization, the time-domain waveform block and the spectral amplitude and phase representation strictly aligned with the time axis are obtained, forming a time-frequency joint representation set; this set is sent to the three-component coding module of the dual-domain three-decoupling VoiceGAN network, which generates speech content features, voiceprint features and noise representations respectively, and integrates them into a feature fusion input set under the same time alignment scale. Subsequently, the dual-domain collaborative generation module works simultaneously in the time domain and the frequency domain, outputs candidate enhanced waveforms and candidate enhanced spectra, and completes collaborative correction through phase continuity and energy registration to generate a candidate enhanced result set; the multi-head discriminator module performs time-domain authenticity discrimination, frequency-domain spectral structure consistency evaluation and perception index prediction on this set, and generates a noise-oriented correction factor. The noise projection module corrects the noise representation accordingly, and sends the candidate enhanced result and the corrected noise representation to the re-projection process together, generating a noise-adaptive enhanced result set. Finally, the consistency reconstruction module outputs the final enhanced waveform and the final enhanced spectrum strictly aligned with the original time axis under the joint constraints of spectral consistency and phase reconstruction, and directly backfills into the call and broadcast link, while providing a call for the speech recognition front end.

[0028] The core of this application process is to solve the three types of problems that have long plagued front-line operations in transportation hubs: "incomprehensibility" of human voice in strong non-stationary noise, "metallic feeling and musical noise" after enhancement, and unstable effect when switching between floors and devices. Through the explicit separation of speech content features, voiceprint features and noise representations by the three-component coding module, the system maintains the consistency of the speaker's tone while avoiding misidentifying noise textures as speech details; the dual-domain collaborative generation module allows the transient continuity in the time domain and the harmonic structure in the frequency domain to be constrained simultaneously, significantly reducing distortion and artifacts; the multi-head discriminator module provides joint feedback in the time domain, frequency domain and perception channel, generating a noise correction factor for dynamic correction, and cooperating with the re-projection closed loop, so that the system can adaptively correct the noise distribution of different sites, different devices and different passenger flow densities online, ensuring consistent performance across scenarios.

[0029] In a specific implementation, the data link operates in a real-time manner. After the front-end pickup is collected on site, the time-frequency alignment module processes it with a consistent window and overlap strategy as the identification front-end, so that subsequent consistent reconstruction does not need to repeatedly estimate boundary transitions; the dual-domain collaborative generation module uses a causal structure and block-level caching in the inference stage, without interrupting the original call and broadcast rhythm; the output of the multi-head discrimination module is not only used to generate noise correction factors, but also archived as a synthetic feedback vector and a consistency score, and saved together with the frame block index and time index of the candidate enhancement result set, ensuring that the re-projection and alignment comparison stage is traceable and reproducible. The residual noise speech obtained by re-projection and the original noise speech on site form a residual graph and a fitting graph after two-level alignment, which serves as the basis for subsequent rollback and refinement, thereby continuously suppressing residual noise and correcting over-weakened voice details without changing the call and broadcast schedule.

[0030] To provide evidence of beneficial effects, tests are repeated in different areas and at different times of the same transportation hub. The control method includes traditional spectral subtraction and Wiener filtering, single-domain end-to-end time-domain enhancement, and an adversarial method targeting only perceptual indicators; the comparison is carried out with the same front-end collection, the same playback and listening path, excluding the influence of equipment differences. The subjective evaluation adopts a grading labeling process oriented towards naturalness and intelligibility, and is carried out by an independent listening evaluation group in a soundproof area with unified earphone playback for blind testing; the objective evaluation adopts general indicators related to voice quality and intelligibility, and records the trend of word-level error changes before voice recognition; at the same time, the change trajectory of the call-back situation and the artificial repeated broadcast situation in the platform live broadcast is recorded. All results are recorded, including time annotation, location annotation, speaker anonymous identification, sampling device type and channel configuration, to ensure that the data source is traceable. The evaluation and statistics do not disclose specific numerical values in this specification, only trends and relative conclusions are retained, which are used to prove the comprehensive improvement of naturalness, intelligibility and cross-scene stability of the application.

[0031] From the operation and maintenance feedback and evaluation conclusions, it can be seen that after applying the application, the discontinuity of the intercom in the train station and strong wind environment is significantly alleviated, the start and stop of the human voice and the blast sound are kept coherent, and the broadcast content still has good recognition in long-distance and reflection-rich locations; when switching the pickup and playback devices at different sites and different times, the system can maintain consistent performance without re-labeling or manual parameter adjustment; the stability of the voice recognition front-end in the background dense human voice and strong mechanical noise is improved, the trigger frequency of repeated broadcasting is reduced, and the dependence of the on-duty personnel on call verification is reduced. The above phenomena and the differences formed by the control method show that the application can simultaneously consider audio quality, naturalness, semantic clarity and cross-scene stability in complex non-stationary noise environments through the combined design of dual-domain collaboration, three-component decoupling, multi-head discrimination and noise re-projection closed loop.

[0032] To ensure generalizability, the system continuously runs at ground and underground stations of the same operating unit, spanning weekdays and weekends, including different passenger flow intensities and different weather conditions. During the operation, noise correction factors and noise representations are periodically archived to form a noise adaptive dictionary, providing direct migration prior for subsequent station expansion. This embodiment shows that the invention can be implemented across scenes without relying on manually labeled noise types. Without changing the original scheduling and control process, it significantly alleviates the common metallic and music noise problems after enhancement, improves the synchronization experience of the recognition front end and human subjective listening, and meets the technical solutions and beneficial effects described in the claims.

[0033] Table 1 Performance comparison of speech enhancement and noise reduction systems based on generative adversarial networks and traditional systems

[0034] Table 1 compares the coverage of multiple acoustic areas and time periods in the same transportation hub from July to September 2025. Compared with unenhanced audio, all algorithms improve speech quality and intelligibility, but traditional spectral subtraction and Wiener are limited in improving under complex non-stationary noise, such as strong wind and vehicle entry at the platform edge. Spectral subtraction only improves PESQ to a lower range and STOI to a limited extent, and residual artifacts still maintain a high error rate. The pure discriminative deep model and the end-to-end time domain method bring more stable gains in four areas, especially in the transfer channel and ground entrance, where the PESQ and SDR are simultaneously improved. However, there are still shortcomings in phase continuity and cross-scene consistency, such as the intelligibility of SEGAN at the platform edge not fully recovered. Compared with the strong baseline MetricGAN+, the invention has a consistent advantage in four areas: PESQ, STOI, and DNSMOS are further improved in the waiting hall, and WER is reduced. The advantage is more obvious in the transfer channel and ground entrance, and the SDR improvement shows that the noise energy is more fully suppressed. In the platform edge scenario, WER is further reduced from nineteen to sixteen, showing the invention's ability to handle strong transient and broadband noise. Across scenes, the invention has significantly improved PESQ, STOI, DNSMOS, and SDR compared to MetricGAN+, and reduced the end-to-end latency from more than fifty milliseconds to less than twenty milliseconds, meeting the real-time call and broadcast requirements.

[0035] The improvements come from three synergies: first, the dual-domain synergy generates and corrects the time-domain transient and frequency-domain harmonic structure simultaneously, cooperates with energy registration, reduces excessive smoothing and boundary splicing artifacts, and synchronously improves PESQ, DNSMOS and SDR; second, the three-component coding explicitly decouples the speech content and voiceprint from the noise, avoids misjudging the noise texture as speech details, and maintains a higher STOI and stably reduces WER in crowded and reverberant channels; third, the multi-head discrimination and noise projection closed loop constitute an adaptive mechanism of "enhancement-noise projection-alignment comparison-reinforcement", which dynamically corrects the noise representation in non-stationary scenes such as strong wind at the station and transient at the station, and brings consistent performance across scenes. At the same time, the reasoning path with causal structure and multi-resolution short-time window reduces the time delay and maintains the phase reconstruction quality, ensuring the continuity of the listening experience and the stability of the recognition in real-time link.

[0036] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the present application within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A speech enhancement and noise reduction system based on generative adversarial networks, characterized in that, include: The time-frequency alignment module is used to collect raw noisy speech and perform unified sampling, endpoint detection, framing and amplitude normalization, construct time-domain waveform blocks and spectral amplitude and phase representations, and generate a set of time-frequency joint representations. The three-component coding module is used to input the time-frequency joint representation set into the three-component coding layer of the dual-domain three-decoupled VoiceGAN network to extract speech content features, speaker features and noise representations, forming a feature fusion input set; The dual-domain collaborative generation module is used to input the feature fusion input set into the time-domain generator and the frequency-domain generator respectively to obtain candidate enhancement waveforms and candidate enhancement spectra, and then form a candidate enhancement result set through collaborative correction. The multi-head discrimination module is used to input the candidate enhancement result set into the time-domain discriminator, frequency-domain discriminator and index discrimination head to generate noise correction factors; The noise reprojection module is used to correct the noise representation based on the noise correction factor, input the candidate enhancement result set and the corrected noise representation into the noise reprojector to generate noisy speech, and align and compare it with the original noisy speech to obtain the noise adaptive enhancement result set. The consistency reconstruction module is used to perform spectral consistency correction and phase reconstruction on the noise adaptive enhancement result set, obtain the final enhanced waveform and the final enhanced spectrum aligned with the original time axis, and output the enhanced speech signal.

2. The speech enhancement and noise reduction system based on generative adversarial networks according to claim 1, characterized in that, The modules are connected in the following way: The original noisy speech is collected, and the original noisy speech is uniformly sampled, endpoint detected, framed and normalized. Time-domain waveform blocks and spectral amplitude and phase representations are constructed to form a time-frequency joint representation set. A dual-domain, three-decoupled VoiceGAN network is constructed. The time-frequency joint representation set is input into the three-component coding layer. Speech content features, speaker features and noise representations are extracted by the content encoder, speaker encoder and noise encoder respectively, and then integrated to obtain the feature fusion input set. The feature fusion input set is input into the time domain generator and the frequency domain generator respectively to generate candidate enhancement waveforms and candidate enhancement spectra. The candidate enhancement result set is formed through collaborative correction. The candidate enhancement result set is input into the time domain discriminator, frequency domain discriminator and index discriminator head in the dual-domain three-decoupled VoiceGAN network, and a noise correction factor is generated based on the discrimination result. The noise representation is corrected according to the noise correction factor to obtain the corrected noise representation. The candidate enhancement result set and the corrected noise representation are input into the noise reprojector to generate re-noise speech. The re-noise speech is aligned and compared with the original noise speech to obtain the noise adaptive enhancement result set. Perform spectral consistency correction and phase reconstruction on the noise adaptive enhancement result set to obtain the final enhanced waveform and final enhanced spectrum aligned with the original time axis, and output the enhanced speech signal.

3. The speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the time-frequency joint representation set specifically includes: The original noisy speech was collected, and uniform sampling and channel normalization were performed. The multi-channel speech was merged into a single-channel speech by power weighting and then unified to the target sampling rate and quantization accuracy to obtain the original sample sequence. Endpoint detection is performed on the original sample sequence, and short-time energy and zero-crossing rate are calculated frame by frame. Short-time energy is defined as the sum of squares of sample values ​​in the frame, and zero-crossing rate is defined as half the number of symbol changes of adjacent samples. Silence segments and long-term invalid segments are removed based on the joint threshold to obtain the effective speech segment sequence. Framing and frame shifting are performed on the effective speech segment sequence, and a cosine-weighted Hanning window is used for windowing to obtain a standardized speech frame sequence. Amplitude normalization is performed on the standardized speech frame sequence, and scale compression is performed based on the peak amplitude of each frame to obtain the amplitude-normalized frame-level sequence. On the time axis, the frame-level sequence after amplitude normalization is spliced ​​together with fixed block length and fixed block shift to construct a time-domain waveform block sequence. A correspondence between each time-domain waveform block and the frame index is established to form a time-domain waveform block and its time mapping index. Calculate the short-time Fourier transform of the standardized speech frame sequence to obtain the frame frequency complex spectrum organized by frame and frequency point and consistent with the corresponding frame index; The frequency amplitude and phase representations are obtained by decomposing the frame frequency complex spectrum and aligned with the time domain waveform block on the same time axis to form a correspondence between the frequency domain and the time domain, generating a time-frequency joint representation set.

4. The speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the feature fusion input set specifically includes: A dual-domain, three-decoupled VoiceGAN network is constructed, consisting of a three-component coding layer, a dual-domain collaborative generator, a three-head discriminator, a noise reprojector, and a consistency correction module. The three-component coding layer includes a content encoder, a voiceprint encoder, and a noise encoder; the dual-domain co-generator consists of a time-domain generator and a frequency-domain generator; and the three-head discriminator includes a time-domain discriminator, a frequency-domain discriminator, and an index discriminator. The time-frequency joint representation set is fed into the content encoder, and frame-level convolutional extraction and context aggregation and time alignment are performed. The frame-level acoustic features are averaged within a fixed-width aggregation window centered on the target time position to obtain speech content features covering the entire time axis. The time-frequency joint representation set is fed into the voiceprint encoder, and frame-level feature extraction and statistical pooling and normalization are performed in sequence. The mean and standard deviation of the frame-level features covering the time axis are calculated and norm normalized to obtain the voiceprint features. The time-frequency joint representation set is fed into the noise encoder. Within a given time window, the minimum value of the spectral amplitude of each frequency band is taken to form the noise amplitude estimate. The noise characterization that changes with time is obtained by interpolation of the frequency band and the time position. The speech content features, speaker features, and noise representations are aligned and scaled according to time index and then concatenated to obtain the component alignment result sequence. Channel rearrangement and block-level caching are performed on the component alignment result sequence to maintain its correspondence with the time-domain waveform block and the spectral amplitude and phase representation. The result is then archived at the frame block level to generate a feature fusion input set.

5. A speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the candidate enhancement result set specifically includes: The feature fusion input set is fed into the temporal generator, which is composed of causal one-dimensional convolution, dilated convolution and residual connection stacked together to obtain waveform residuals of the same length as the temporal waveform block. The waveform residuals are added to the corresponding temporal waveform block to generate candidate enhanced waveforms. The feature fusion input set is fed into the frequency domain generator. The encoder and decoder of two-dimensional convolution are combined with cross-layer skip connections to perform forward computation, which yields an amplitude scaling mask and phase increment candidates of the same size as the spectral amplitude. The amplitude scaling mask is applied to the spectral amplitude to obtain the candidate enhanced amplitude, and the phase increment candidates are superimposed on the phase representation to obtain the candidate phase representation. The combination forms the candidate enhanced spectrum. At the frame block level, phase continuity processing is performed on the candidate phase representation of the candidate enhancement spectrum. The phase change of adjacent frame blocks is calculated and phase wrapping is performed to obtain the phase consistency correction amount. The phase consistency correction amount is applied to the candidate phase representation to obtain the phase sequence after phase consistency correction. The candidate phase representation of the candidate enhancement spectrum is then updated to obtain the candidate enhancement spectrum after phase consistency correction. The energy metrics of the candidate enhancement waveform and the candidate enhancement amplitude are calculated at the frame block level. The candidate enhancement waveform is scaled based on the energy of the candidate enhancement amplitude to obtain the candidate enhancement waveform after energy conservation correction. The energy registration parameters used for amplitude scaling are recorded to obtain the candidate enhancement waveform after energy conservation correction. The candidate enhancement waveforms after energy conservation correction and the candidate enhancement spectra after phase consistency correction are indexed and aligned in the frame block dimension and time axis. The matching of the corresponding frame blocks in terms of time consistency and energy consistency is verified according to the energy registration parameters. The results are combined to obtain the co-correction alignment results containing the aligned waveforms and aligned spectra. The results are archived by frame block to generate a set of candidate enhancement results.

6. The speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the noise correction factor specifically includes: Receive the candidate enhancement result set, unpack it to obtain the candidate enhancement waveform sequence, candidate enhancement spectrum sequence and corresponding frame block index, time index and energy registration parameters, establish a discrimination input batch and reference index table according to frame block, and use it for parallel discrimination of time domain discriminator, frequency domain discriminator and index discrimination head to obtain the alignment input for each discriminator; The aligned waveforms in the candidate enhancement result set are input into the time-domain discriminator in frame blocks, and the output time-domain true and false score sequence is obtained by subtracting the corresponding time-domain true and false score from one, thus forming a time-domain discrimination result containing the time-domain true and false score and the time-domain adversarial residual. The aligned spectrum in the candidate enhancement result set is input into the frequency domain discriminator in frame blocks, and the output frequency domain true and false score sequence and spectral structure consistency score sequence are output. The corresponding frequency domain true and false score is subtracted by one to obtain the frequency domain adversarial residual sequence, forming a frequency domain discrimination result containing frequency domain true and false score, spectral structure consistency score and frequency domain adversarial residual. The candidate enhancement result set is input into the indicator discrimination head, which outputs the frame block-level perception indicator prediction vector. The indicator deviation is obtained by subtracting the perception indicator prediction vector from the reference indicator target vector, forming an indicator discrimination result that includes the perception indicator prediction vector and the indicator deviation. The temporal adversarial residual, frequency adversarial residual, and index bias are weighted and fused. In each frame block, the weights of the temporal channel, frequency channel, and index channel are normalized and added together to one. The frame block level synthesis feedback vector is obtained by multiplying the normalized weights and summing them. The frame block level consistency score is generated based on the spectral structure consistency score, forming a fusion result that includes the frame block level synthesis feedback vector and the frame block level consistency score. Based on the fusion results, the frame block-level consistency score is used as a gate to amplify or suppress the frame block-level synthesized feedback vector on the time axis. Combining the energy distribution of the aligned spectrum in each frequency band and the contribution ratio of the spectral structure consistency score in the frequency band dimension, the gated synthesized feedback vector is mapped into a two-dimensional weighted graph in time and frequency band. Sliding window smoothing and boundary constraints are performed on the weighted graph, with different upper limits set for the silence and speech regions. Transition processing is performed on abrupt change positions to obtain a constrained weighted graph, which is aligned with the noise representation in the time and frequency band dimensions. Frame block-level global factors and frequency band-level refinement factors are extracted and formed into noise correction factors through weighted combination.

7. A speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the noise adaptive enhancement result set specifically includes: Based on the noise correction factor, the noise representation is weighted and corrected in both the frame block and frequency band dimensions to generate the corrected noise representation. Construct noise reprojection input, align and stitch the candidate enhanced spectrum and the corrected noise characterization on the time axis and frequency band axis to form frequency domain joint reprojection features, align and bind the candidate enhanced waveform and energy registration parameters on the time axis to form time domain reprojection auxiliary information, and merge to obtain the reprojection input set; The reprojection input set is fed into the noise reprojector, and the generation process of noise re-injection and mixing, phase advancement and amplitude constraint, and time domain consistency is executed in the order of frame blocks, and the output is the re-noise speech sequence corresponding to the frame block index. Two-level alignment is performed between the re-noise speech sequence and the original noisy speech. Coarse alignment anchors are established over the entire range based on the endpoint positions and energy peaks. Then, fine alignment is performed within the frame block according to a short time window to generate a time alignment mapping table. Based on the time alignment mapping table, the re-noise speech sequence and the original noisy speech are compared frame by frame and frequency band by frequency band to obtain the amplitude difference distribution, phase difference distribution and instantaneous signal-to-noise ratio increment distribution. These are then summarized to form the reprojection comparison residual map and the noise matching map. By using the reprojection comparison residual map, selective back-off and refinement corrections are performed on the candidate enhancement spectrum and candidate enhancement waveform. The high residual region is back-off and the amplitude is constrained according to the energy registration parameter. The low residual region is preserved or locally enhanced, resulting in a noise adaptive enhancement result set, including the noise adaptive enhancement spectrum and noise adaptive enhancement waveform.

8. A speech enhancement and noise reduction system based on generative adversarial networks according to claim 2, characterized in that, The generation of the enhanced speech signal specifically includes: Based on the combination of the amplitude and phase of the noise adaptive enhancement spectrum in the noise adaptive enhancement result set at the corresponding frame block and frequency band position, an initial complex spectrum is constructed. Under the constraint of the time alignment mapping table, phase reconstruction iteration is performed. The initial complex spectrum is subjected to short-time Fourier inverse transform to obtain a temporary time domain signal. The temporary time domain signal is subjected to short-time Fourier transform to obtain a new complex spectrum. The phase iteration update sequence is obtained by iterative process. The phase iteration update sequence is combined with the preset target amplitude, and projected onto the spectral consistency space through time-domain reconstruction and secondary transformation. Based on the reprojection comparison residual map, in-band smoothing and amplitude upper limit constraints are performed on the high residual frequency band, while the current amplitude and phase are maintained for the low residual frequency band, forming a complex spectrum after spectral consistency correction. Based on the energy registration parameters, bidirectional registration is performed on the complex spectrum after noise adaptive enhancement waveform and spectrum consistency correction. In the frequency domain, the amplitude is scaled at the frame block level to meet energy conservation. In the time domain, the waveform is fine-tuned at the frame block level and overlapped and added in the same way as the framing. The results are then spliced ​​to obtain the energy and boundary alignment results. The continuity of the start and end overlap areas of adjacent frame blocks is verified by the time alignment mapping table. Phase convergence determination and harmonic refinement are performed on the energy and boundary alignment results. When the phase change is lower than the convergence threshold, the phase is frozen. When the phase change is higher than the convergence threshold, the phase iteration is updated again. In the speech region, the harmonic peaks and rectified envelope are enhanced based on the fundamental frequency correlation. In the silence region, residual noise stripes and isolated spectral peaks are suppressed to generate the final enhanced spectrum. A short-time inverse Fourier transform is performed on the final enhanced spectrum, and windowing and overlap are used to complete the time-domain reconstruction, resulting in a final enhanced waveform aligned with the original time axis. The final enhanced waveform and the final enhanced spectrum are archived in frame blocks and the enhanced speech signal is output.

Images