Artificial intelligence-based speech noise reduction methods, devices, equipment, and storage media

Abstract

This application discloses an artificial intelligence-based speech denoising method, apparatus, device, and storage medium. The method includes: acquiring a denoised spectrogram corresponding to the speech to be denoised; inputting the denoised spectrogram into a preset denoising model for denoising processing to obtain a denoised spectrogram, wherein the denoising model sequentially includes: an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module; the frequency domain denoising module is a module that implements a multi-head self-attention mechanism using adjacent sub-band information; the time domain denoising module includes: at least two time domain denoising sub-modules; and reconstructing the speech signal from the denoised spectrogram to obtain the target clean speech. This method achieves decoupling between the time and frequency domains, which is beneficial for streaming speech denoising and is suitable for application scenarios with limited computing resources and / or high real-time requirements, thus improving the denoising effect.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a speech noise reduction method, apparatus, device and storage medium based on artificial intelligence. Background Art

[0002] Speech often contains noise, which reduces the accuracy of speech applications and affects user experience when applied in real-world scenarios. Existing denoising models trained using convolutional neural networks achieve good noise reduction results, but these models have high computational resource requirements and require long computation times, making them unsuitable for applications with limited computing resources and / or high real-time requirements. Summary of the Invention

[0003] The main purpose of this application is to provide a speech denoising method, apparatus, device and storage medium based on artificial intelligence, which aims to solve the technical problem that denoising models trained based on convolutional neural networks cannot be applied to application scenarios with limited computing resources and / or high real-time requirements.

[0004] To achieve the aforementioned objectives, this application proposes an artificial intelligence-based speech noise reduction method, the method comprising:

[0005] Obtain the denoising spectrum of the speech to be denoised;

[0006] The spectrum to be denoised is input into a preset denoising model for denoising processing to obtain a denoised spectrum. The denoising model includes, in sequence, an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module. The frequency domain denoising module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain denoising module includes at least two time domain denoising sub-modules.

[0007] The denoised spectrogram is used to reconstruct the speech signal to obtain the target clean speech.

[0008] Furthermore, the step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum includes:

[0009] The spectrum to be denoised is input into the encoding module for feature extraction, resulting in multiple single-layer audio coding features and target audio coding features.

[0010] The target audio coding features are input into the frequency domain noise reduction module for frequency domain noise reduction to obtain the audio features after frequency domain noise reduction.

[0011] The target audio coding features and the frequency domain denoised audio features are residually concatenated to obtain the audio features to be processed.

[0012] The audio features to be processed are input into the temporal denoising module for feature grouping, grouped temporal denoising, and feature concatenation to obtain the temporally denoised audio features.

[0013] The audio features after frequency domain denoising and the audio features after time domain denoising are residually concatenated to obtain the audio features to be decoded.

[0014] Each of the single-layer audio coding features and the audio features to be decoded are input into the decoding module for decoding to obtain the spectrum diagram to be analyzed.

[0015] The spectrum to be analyzed is input into the masking gain and subtraction module for masking to obtain the denoised spectrum.

[0016] Furthermore, the step of inputting the target audio encoding features into the frequency domain noise reduction module for frequency domain noise reduction to obtain the frequency domain noise-reduced audio features includes:

[0017] The dimensionality reduction submodule of the frequency domain noise reduction module is used to perform dimensionality reduction processing on the target audio coding features to obtain the dimensionality-reduced coding features.

[0018] The multi-head self-attention submodule of the frequency domain denoising module is used to perform frequency domain denoising on the dimensionality-reduced encoded features to obtain the dimensionality-upgraded encoded features. The multi-head self-attention submodule is a module that implements the multi-head self-attention mechanism. The Query, Key, and Value values ​​of the multi-head self-attention mechanism of the frequency domain denoising submodule are determined based on the preset dependent frequency band width and adjacent sub-band information.

[0019] The dimensionality-upgrading submodule of the frequency domain noise reduction module is used to perform dimensionality-upgrading processing on the encoded features to be upgraded, thereby obtaining the audio features after frequency domain noise reduction.

[0020] Furthermore, the step of inputting the audio features to be processed into the temporal denoising module for feature grouping, grouped temporal denoising, and feature concatenation to obtain temporally denoised audio features includes:

[0021] The feature grouping layer of the time-domain noise reduction module is used to divide the audio features to be processed into multiple single audio features, wherein the number of single audio features is the same as the number of time-domain noise reduction sub-modules.

[0022] The i-th single audio feature is input into the i-th temporal denoising submodule for temporal denoising to obtain the i-th audio feature to be combined, where i is an integer greater than 0;

[0023] The feature combination layer of the time-domain noise reduction module is used to perform feature concatenation on each of the audio features to be combined to obtain the audio features after time-domain noise reduction.

[0024] Furthermore, the step of inputting the spectrogram to be denoised into the encoding module for feature extraction to obtain multiple single-layer audio coding features and target audio coding features includes:

[0025] Using the k-th encoding layer of the encoding module, a pointwise convolution is performed on the input vector of the k-th encoding layer to obtain the first audio feature;

[0026] Obtain the preset depthwise convolution time dimension;

[0027] If the time dimension of the Depthwise convolution is equal to 1, the k-th encoding layer is used to perform a regular convolution on the first audio feature to obtain the k-th single-layer audio encoding feature;

[0028] If the time dimension of the Depthwise convolution is equal to 2, the k-th coding layer is used to perform causal convolution on the first audio feature to obtain the k-th single-layer audio coding feature;

[0029] The nth single-layer audio coding feature is taken as the target audio coding feature;

[0030] Where k is an integer greater than 0, k is less than or equal to n, n is greater than 0, and n is the number of coding layers; when k equals 1, the spectrum to be denoised is used as the input vector of the kth coding layer, and when k is greater than 1, the (k-1)th single-layer audio coding feature is used as the input vector of the kth coding layer.

[0031] Furthermore, the step of inputting each of the single-layer audio coding features and the audio features to be decoded into the decoding module for decoding to obtain the spectrogram to be analyzed includes:

[0032] The (n+1)-mth single-layer audio coding feature is dimensionality reduced to obtain the coding feature to be processed, where m is an integer greater than 0 and m is less than or equal to n;

[0033] The output vector of the (m-1)th decoding layer and the encoded feature to be processed are added together at the same position to obtain the m-th feature to be processed;

[0034] The m-th feature to be processed is deconvolutionally processed to obtain the m-th single-layer decoded feature;

[0035] The nth single-layer decoding feature is taken as the spectrum to be analyzed.

[0036] Wherein, when m equals 1, the audio feature to be decoded is used as the output vector of the (m-1)th decoding layer, and when m is greater than 1, the (m-1)th single-layer decoding feature is used as the output vector of the (m-1)th decoding layer.

[0037] Furthermore, before the step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum, the method further includes:

[0038] Obtain multiple training samples and the model to be trained;

[0039] The model to be trained is trained according to each training sample and the preset objective function until the preset model training termination condition is reached. The model to be trained that has reached the model training termination condition is used as the noise reduction model.

[0040] The objective function S is expressed as: S = SISNR + MSE loss + perceptual loss + regularization term. SISNR is the signal-to-noise ratio loss of speech, MSE loss is the loss calculated based on the mean square error of the real part of the spectrogram, the mean square error of the imaginary part of the spectrogram, and the mean square error of the amplitude spectrum of the spectrogram, and perceptual loss is the perceptual loss of speech.

[0041] This application also proposes an artificial intelligence-based speech noise reduction device, the device comprising:

[0042] The data acquisition module is used to acquire the denoised spectrogram corresponding to the speech to be denoised;

[0043] The noise reduction processing module is used to input the spectrum to be denoised into a preset noise reduction model for noise reduction processing to obtain a denoised spectrum. The noise reduction model includes, in sequence, an encoding module, a frequency domain noise reduction module, a time domain noise reduction module, a decoding module, and a mask gain and loss module. The frequency domain noise reduction module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain noise reduction module includes at least two time domain noise reduction sub-modules.

[0044] The speech signal reconstruction module is used to reconstruct the speech signal from the noise-reduced spectrogram to obtain the target clean speech.

[0045] This application also proposes a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0046] This application also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0047] This application discloses an artificial intelligence-based speech denoising method, apparatus, device, and storage medium. The method inputs the spectrum to be denoised into a preset denoising model for denoising processing to obtain a denoised spectrum. The denoising model sequentially includes: an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module. The frequency domain denoising module is a module that utilizes adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain denoising module includes at least two time domain denoising sub-modules. Speech signal reconstruction is performed on the denoised spectrum to obtain the target clean speech. This system employs a denoising model that sequentially performs feature extraction, frequency domain denoising, time domain denoising, decoding, and mask gain and reduction. Effective denoising is achieved by sequentially performing frequency and time domain denoising, improving the denoising performance. Separating frequency and time domain denoising decouples the two domains, which is beneficial for streaming speech denoising. The time domain denoising module uses at least two sub-modules to achieve grouped time domain denoising, reducing computational load and network parameter count, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain denoising module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing denoising performance. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating an embodiment of an artificial intelligence-based speech noise reduction method according to this application.

[0049] Figure 2 This is a schematic block diagram of the structure of an artificial intelligence-based voice noise reduction device according to an embodiment of this application;

[0050] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application.

[0051] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] Reference Figure 1 This application provides an artificial intelligence-based speech noise reduction method, the method comprising:

[0054] S1: Obtain the spectrum of the speech to be denoised;

[0055] S2: Input the spectrum to be denoised into a preset denoising model for denoising processing to obtain a denoised spectrum. The denoising model includes, in sequence, an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module. The frequency domain denoising module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain denoising module includes at least two time domain denoising sub-modules.

[0056] S3: Reconstruct the speech signal from the denoised spectrogram to obtain the target clean speech.

[0057] This embodiment implements a noise reduction model that sequentially performs feature extraction, frequency domain noise reduction, time domain noise reduction, decoding, and mask gain and loss. Effective noise reduction is achieved by sequentially performing frequency and time domain noise reduction, improving the noise reduction effect. Separating frequency and time domain noise reduction decouples the time and frequency domains, which is beneficial for streaming speech noise reduction. The time domain noise reduction module uses at least two time domain noise reduction sub-modules to achieve grouped time domain noise reduction, reducing the amount of computation and network parameters, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain noise reduction module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing the noise reduction effect.

[0058] For S1, it can obtain the denoised spectrum corresponding to the user-input denoised speech, or obtain the denoised spectrum corresponding to the denoised speech from the database, or obtain the denoised spectrum corresponding to the denoised speech from a third-party application.

[0059] The speech to be denoised is one or more segments of speech that need to be denoised.

[0060] The spectrogram to be denoised is the spectrogram of the speech to be denoised, which is generated based on the Fourier spectrum.

[0061] The denoised spectrum consists of two channels: a real part channel and an imaginary part channel. The real part channel represents the real part of the Fourier spectral characteristics, while the imaginary part channel represents the imaginary part of the Fourier spectral characteristics.

[0062] The process involves performing a Short-Time Fourier Transform (SFT) on the speech to be denoised, and using the resulting spectrogram as the spectrum to be processed. Then, the DC component of the spectrum to be processed is removed, and the resulting spectrum is used as the spectrum to be denoised. Since the DC component has little impact on spectrum reconstruction, removing it from the SFT spectrum does not affect the denoising effect and reduces computational complexity.

[0063] For S2, the spectrum to be denoised is input into a preset denoising model and subjected to feature extraction, frequency domain denoising, time domain denoising, decoding, and mask gain and loss in sequence. The data output by mask gain and loss is used as the denoised spectrum.

[0064] The encoding module is used to perform encoding to extract audio features. The encoding module includes multiple encoding layers, which are linearly connected, and each encoding layer outputs a single-layer audio encoded feature.

[0065] Optionally, to achieve streaming processing, only convolutions in the frequency dimension can be used, that is, the kernel size of the convolutions in the time dimension is 1 to reduce the amount of computation, or causal convolutions can be used in the time domain to achieve streaming processing.

[0066] Optionally, the number of encoding layers in the encoding module is at least three.

[0067] The frequency domain noise reduction module is used to perform noise reduction in the frequency domain, thereby making full use of frequency domain information. The output of the last coding layer of the coding module serves as the input of the frequency domain noise reduction module. The frequency domain noise reduction module is a module that utilizes adjacent sub-band information to implement a multi-head self-attention mechanism.

[0068] The temporal denoising module is used to perform denoising in the time domain, thereby making full use of time-domain information. The output of the frequency-domain denoising module and the output of the last coding layer of the coding module are used as inputs to the temporal denoising module. The temporal denoising module includes at least two temporal denoising sub-modules, thereby enabling grouped temporal denoising. The temporal denoising sub-modules are modules based on Long Short-Term Memory (LSTM) artificial neural networks and / or GRUs (Gated Recurrent Units).

[0069] The decoding module is used to perform decoding to obtain a spectrum after frequency domain denoising and time domain denoising. The decoding module includes multiple decoding layers, which are linearly connected. The input to each decoding layer is data obtained from the output of the previous decoding layer and the output of the encoding layer.

[0070] Optionally, the number of decoding layers in the decoding module is the same as the number of encoding layers in the encoding module.

[0071] Optionally, the decoding module has at least three decoding layers.

[0072] The mask gain and loss module is used to enhance the data corresponding to the desired speech and suppress the data corresponding to the unwanted speech in the spectrogram.

[0073] Optionally, the masking gain and loss module uses 0s and 1s for masking. For example, in a spectrogram, the masking gain and loss module uses 1s for masking to gain the data corresponding to the desired speech, and uses 0s for masking to lose the data corresponding to the unwanted speech.

[0074] Optionally, the mask gain and loss module uses a value of 0 to 1 for masking.

[0075] For S3, a short-time inverse Fourier transform is performed on the denoised spectrogram to obtain the time-domain data to be processed; the Overlapadd method is used to reconstruct the speech signal from the time-domain data to be processed, and the reconstructed clean speech is used as the target clean speech corresponding to the speech to be denoised.

[0076] Overlapadd, also written as Overlap-add, means to add items together by overlapping.

[0077] The method of reconstructing speech signals from the time-domain data to be processed using the Overlapadd method will not be described in detail here.

[0078] In one embodiment, the step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum includes:

[0079] S21: Input the spectrum of the noise to be reduced into the coding module for feature extraction to obtain multiple single-layer audio coding features and target audio coding features;

[0080] S22: Input the target audio coding features into the frequency domain noise reduction module to perform frequency domain noise reduction, and obtain the audio features after frequency domain noise reduction;

[0081] S23: Perform residual concatenation between the target audio coding features and the frequency domain denoised audio features to obtain the audio features to be processed;

[0082] S24: Input the audio features to be processed into the time-domain noise reduction module to perform feature grouping, grouped time-domain noise reduction and feature concatenation respectively to obtain the audio features after time-domain noise reduction;

[0083] S25: Perform residual connection between the frequency domain denoised audio features and the time domain denoised audio features to obtain the audio features to be decoded;

[0084] S26: Input each of the single-layer audio coding features and the audio features to be decoded into the decoding module for decoding to obtain the spectrum diagram to be analyzed;

[0085] S27: Input the spectrum to be analyzed into the mask gain and subtraction module for masking to obtain the denoised spectrum.

[0086] This embodiment employs a denoising model that sequentially performs feature extraction, frequency domain denoising, time domain denoising, decoding, and mask gain and reduction. By sequentially performing frequency and time domain denoising, effective denoising is achieved, improving the denoising performance. Separating frequency and time domain denoising decouples the time and frequency domains, which is beneficial for streaming speech denoising. The time domain denoising module uses at least two time domain denoising sub-modules to achieve grouped time domain denoising, reducing computational load and network parameter count, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain denoising module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing the denoising effect.

[0087] For S21, the spectrum to be denoised is input into the coding module for feature extraction. The audio feature data extracted from each coding layer in the coding module is taken as a single-layer audio coding feature, and the single-layer audio coding feature extracted from the last coding layer in the coding module is taken as the target audio coding feature.

[0088] For S22, the target audio encoding features are input into the frequency domain noise reduction module. Noise reduction is performed in the frequency domain dimension through the multi-head self-attention mechanism of the frequency domain noise reduction module. The audio features after noise reduction in the frequency domain dimension are used as the audio features after frequency domain noise reduction.

[0089] For S23, the target audio coding features and the frequency domain denoised audio features are performed by a residual connection, and the audio features obtained by the residual connection are used as the audio features to be processed.

[0090] The method for residually connecting the target audio coding features and the frequency domain denoised audio features will not be elaborated here.

[0091] For S24, the audio features to be processed are input into the temporal denoising module. The temporal denoising module performs feature grouping, and the grouped data is subjected to temporal denoising in the temporal dimension. The denoised features are then concatenated, and the concatenated data is used as the temporal denoised audio features.

[0092] For S25, the audio features after frequency domain denoising and the audio features after time domain denoising are residually concatenated, and the audio feature vector obtained by the residual concatenation is used as the audio feature to be decoded.

[0093] The method for performing residual connection between the frequency-domain denoised audio features and the time-domain denoised audio features will not be elaborated here.

[0094] For S26, the m-th feature to be decoded is determined based on the (n+1-m)-th single-layer audio coding feature and the output vector of the (m-1)-th decoding layer, where m is an integer greater than 0 and m is less than or equal to n, and n is the number of coding layers in the coding module, which is the same as the number of decoding layers in the decoding module; when m equals 1, the audio feature to be decoded is used as the output vector of the (m-1)-th decoding layer, and when m is greater than 1, the (m-1)-th single-layer decoding feature is used as the output vector of the (m-1)-th decoding layer.

[0095] The data output from the last decoding layer of the decoding module is used as the spectrogram to be analyzed.

[0096] For S27, the spectrum to be analyzed is input into the mask gain and subtraction module; the mask gain and subtraction module uses CRM Mask as the filtering function for noise reduction, so as to increase the data corresponding to the desired speech and subtract the data corresponding to the unwanted speech in the spectrum; the masked spectrum to be analyzed is used as the noise-reduced spectrum.

[0097] Optionally, the following formula can be used for masking: enhancement_real + i*enhance_image = (mask_real + i*mask_image)*(noisy_real + i*noisy_image), where enhancement_real is the real part of the enhanced speech, enhancement_image is the imaginary part after enhancement, mask_real is the masking enhancement coefficient of the real part, mask_image is the masking enhancement coefficient of the imaginary part, noisy_real is the real part of the noise, noisy_image is the imaginary part of the noise, and i is the imaginary unit.

[0098] The mask enhancement factor is a value between 0 and 1; it can be either 0 or 1.

[0099] In one embodiment, the step of inputting the target audio coding features into the frequency domain noise reduction module for frequency domain noise reduction to obtain the frequency domain noise-reduced audio features includes:

[0100] S221: The dimensionality reduction submodule of the frequency domain noise reduction module is used to perform dimensionality reduction processing on the target audio coding features to obtain the dimensionality-reduced coding features;

[0101] S222: The multi-head self-attention submodule of the frequency domain denoising module is used to perform frequency domain denoising on the dimensionality-reduced encoded features to obtain the dimensionality-upgrading encoded features. The multi-head self-attention submodule is a module that implements the multi-head self-attention mechanism. The Query, Key, and Value values ​​of the multi-head self-attention mechanism of the frequency domain denoising submodule are determined based on the preset dependent frequency band width and adjacent sub-band information.

[0102] S223: Using the dimensionality-upgrading submodule of the frequency domain noise reduction module, the dimensionality-upgrading coded features to be upgraded are processed to obtain the frequency domain noise-reduced audio features.

[0103] This embodiment performs dimensionality reduction, frequency domain denoising, and dimensionality increase sequentially, ensuring that the computational load is controllable while maintaining the model's performance. The Query, Key, and Value values ​​of the multi-head self-attention mechanism in the frequency domain denoising submodule are determined based on the preset dependent frequency band width and adjacent sub-band information, which helps improve the frequency domain denoising effect.

[0104] For S221, the dimensionality reduction submodule of the frequency domain noise reduction module is used to perform dimensionality reduction processing on the target audio coding features, and the data obtained from the dimensionality reduction processing is used as the dimensionality-reduced coding features.

[0105] The dimensionality reduction submodule adopts a fully connected layer.

[0106] For S222, the multi-head self-attention submodule of the frequency domain denoising module is used to perform frequency domain denoising on the dimensionality-reduced encoded features, and the data obtained by frequency domain denoising is used as the dimensionality-upgrading encoded features.

[0107] Query, Key, and Value are the elements of the multi-head self-attention mechanism. The basic principle is: given a Query, calculate the relevance between the Query and the Key, and then find the most suitable Value based on the relevance between the Query and the Key.

[0108] Optionally, the frequency band width is set to a preset context dependency width. Specifically, the Query, Key, and Value values ​​of the multi-head self-attention mechanism in the frequency domain noise reduction submodule are determined based on the preset frequency band width and adjacent sub-band information. That is, the current frame, the preceding frame with the preset context dependency width, and the following frame with the preset context dependency width are used as the Query value; the current frame is used as the Key value; a content vector is generated based on the current frame, the preceding frame with the preset context dependency width, and the following frame with the preset context dependency width; and the generated content vector is used as the Value value.

[0109] It is understandable that the current frame, the preceding frame with a preset context-dependent width, and the following frame with a preset context-dependent width are consecutive frames.

[0110] For example, if the bandwidth depends on 4, then the value of the Query is determined by the current frame, the previous 4 frames, and the following 4 frames (a total of 9 frames, i.e., consecutive frames). The current frame is used as the value of the Key. A content vector is generated based on the current frame, the previous 4 frames, and the following 4 frames (a total of 9 frames). The generated content vector is used as the value of the Value.

[0111] For S223, the dimensionality-upgrading submodule of the frequency domain noise reduction module is used to perform dimensionality-upgrading processing on the coded features to be upgraded, and the data obtained from the dimensionality-upgrading processing is used as the audio features after frequency domain noise reduction.

[0112] The upgraded submodule adopts a fully connected layer.

[0113] In one embodiment, the steps of inputting the audio features to be processed into the temporal denoising module for feature grouping, grouped temporal denoising, and feature concatenation to obtain temporally denoised audio features include:

[0114] S241: The feature grouping layer of the time-domain noise reduction module is used to divide the audio features to be processed into multiple single audio features, wherein the number of single audio features is the same as the number of time-domain noise reduction sub-modules;

[0115] S242: Input the i-th single audio feature into the i-th temporal denoising submodule for temporal denoising to obtain the i-th audio feature to be combined, where i is an integer greater than 0;

[0116] S243: Using the feature combination layer of the time-domain noise reduction module, feature splicing is performed on each of the audio features to be combined to obtain the audio features after time-domain noise reduction.

[0117] This embodiment performs feature grouping, time-domain denoising of the groups, and feature concatenation respectively. While ensuring the width of the time-domain denoising submodule, it reduces the amount of computation and network parameters, which is beneficial for application scenarios with limited computing resources and / or high real-time requirements.

[0118] For S241, the feature grouping layer of the time-domain noise reduction module is used to divide the audio features to be processed, and each group of features obtained is taken as a single audio feature.

[0119] The number of individual audio features is the same as the number of time-domain noise reduction submodules, thus preparing input data for each time-domain noise reduction submodule.

[0120] For S242, the i-th single audio feature is input into the i-th temporal denoising submodule for temporal denoising to obtain the i-th audio feature to be combined, thereby realizing that each single audio feature is denoised by one temporal denoising submodule.

[0121] In other words, the number of audio features to be combined is the same as the number of time-domain noise reduction submodules.

[0122] For S243, the feature combination layer of the time-domain noise reduction module is used to perform feature concatenation on each of the audio features to be combined, and the data obtained by feature concatenation is used as the audio feature after time-domain noise reduction.

[0123] In one embodiment, the step of inputting the spectrogram to be denoised into the encoding module for feature extraction to obtain multiple single-layer audio coding features and target audio coding features includes:

[0124] S211: Using the k-th coding layer of the coding module, perform pointwise convolution on the input vector of the k-th coding layer to obtain the first audio feature;

[0125] S212: Obtain the preset Depthwise convolution time dimension;

[0126] S213: If the time dimension of the Depthwise convolution is equal to 1, the k-th coding layer is used to perform a regular convolution on the first audio feature to obtain the k-th single-layer audio coding feature;

[0127] S214: If the time dimension of the Depthwise convolution is equal to 2, the k-th coding layer is used to perform causal convolution on the first audio feature to obtain the k-th single-layer audio coding feature;

[0128] S215: Use the nth single-layer audio coding feature as the target audio coding feature;

[0129] Where k is an integer greater than 0, k is less than or equal to n, n is greater than 0, and n is the number of coding layers; when k equals 1, the spectrum to be denoised is used as the input vector of the kth coding layer, and when k is greater than 1, the (k-1)th single-layer audio coding feature is used as the input vector of the kth coding layer.

[0130] This embodiment uses a sequence of Depthwise convolution followed by Pointwise convolution for feature extraction. However, this sequence does not effectively synthesize information. To address this issue, this embodiment uses a sequence of Pointwise convolution followed by Depthwise convolution for information synthesis. Pointwise convolution is first used to synthesize information in the real part of the channel, followed by Depthwise convolution in the imaginary part. The audio features obtained from this information synthesis are then used for noise reduction, which improves the noise reduction effect. Compared to conventional convolution, Pointwise and Depthwise convolutions have fewer parameters, thus improving speech noise reduction efficiency. This further facilitates the application of this application in scenarios with limited computing resources and / or high real-time requirements. Conventional convolution is used when the time dimension of Depthwise convolution is equal to 1, and causal convolution is used when the time dimension of Depthwise convolution is equal to 2, thereby achieving streaming processing. This further facilitates the application of this application in scenarios with limited computing resources and / or high real-time requirements.

[0131] A regular convolution has a kernel size of 3*3, 64 input channels, and 128 output channels. In this case, the parameters of the regular convolution are 64*128*3*3.

[0132] Pointwise convolution uses a 1×1×M kernel, where M is the number of channels in the previous layer. Therefore, the convolution operation here weights and combines the previous feature map along the depth direction to generate a new feature map. The number of output feature maps corresponds to the number of convolution kernels.

[0133] Depthwise convolution involves one kernel responsible for one channel, with each channel being convolved by only one kernel. A 64×64 pixel, three-channel color image first undergoes the first convolution operation. The difference is that this convolution is performed entirely in a two-dimensional plane, and the number of filters is the same as the number of channels in the previous layer. Therefore, a three-channel image generates three feature maps after the operation.

[0134] In the example above, the parameters of a regular convolution are 64*128*3*3, while the parameters of a depthwise convolution and a pointwise convolution are 64*128 + 128*3*3, which significantly reduces the number of parameters and the amount of computation.

[0135] For S211, the k-th encoding layer of the encoding module is used, and the input vector of the k-th encoding layer is subjected to pointwise convolution, and the data obtained by convolution is used as the first audio feature.

[0136] For S212, a preset Depthwise convolution time dimension can be obtained from the database, or a preset Depthwise convolution time dimension input by the user can be obtained, or the preset Depthwise convolution time dimension can be written into the program implementing this application.

[0137] For S213, if the time dimension of the Depthwise convolution is equal to 1, it means that the temporal convolution kernel is 1. Therefore, the k-th coding layer is used to perform a regular convolution on the first audio feature, and the data obtained by the regular convolution is used as the k-th single-layer audio coding feature.

[0138] For S214, if the time dimension of the Depthwise convolution is equal to 2, it means that the temporal convolution kernel is 2. Therefore, when calculating the current frame in streaming inference, it is necessary to record the information of the previous frame. Thus, the k-th coding layer is used to perform causal convolution on the first audio feature, and the data obtained by causal convolution is used as the k-th single-layer audio coding feature.

[0139] Causal convolutions.

[0140] For S215, the nth single-layer audio coding feature is taken as the target audio coding feature, that is, the audio feature extracted from the last coding layer in the coding module is taken as the target audio coding feature.

[0141] When k equals 1, the spectrum to be denoised is used as the input vector of the kth coding layer, that is, the input vector of the first coding layer is the input vector of the coding module; when k is greater than 1, the (k-1)th single-layer audio feature vector is used as the input vector of the kth coding layer, that is, the input vector of the coding layer after the first one is the output vector of the previous coding layer.

[0142] In one embodiment, the step of inputting each of the single-layer audio coding features and the audio features to be decoded into the decoding module for decoding to obtain the spectrogram to be analyzed includes:

[0143] S261: The (n+1)-mth single-layer audio coding feature is subjected to dimensionality reduction processing to obtain the coding feature to be processed, where m is an integer greater than 0 and m is less than or equal to n;

[0144] S262: Add the output vector of the (m-1)th decoding layer and the element value at the same position of the encoded feature to be processed to obtain the m-th feature to be processed;

[0145] S263: Perform deconvolution processing on the m-th feature to be processed to obtain the m-th single-layer decoded feature;

[0146] S264: The nth single-layer decoding feature is taken as the spectrum to be analyzed;

[0147] Wherein, when m equals 1, the audio feature to be decoded is used as the output vector of the (m-1)th decoding layer, and when m is greater than 1, the (m-1)th single-layer decoding feature is used as the output vector of the (m-1)th decoding layer.

[0148] This embodiment achieves twice the number of channels compared to the Concat method by using dimensionality reduction, adding the element values ​​at the same positions as the output vector of the (m-1)th decoding layer, and deconvolution. This embodiment reduces the number of network parameters and computational load, making it more suitable for applications with limited computing resources and / or high real-time requirements. Compared to the Skip method, this embodiment offers better performance.

[0149] For S261, the (n+1-m)th single-layer audio coding feature is subjected to pointwise convolution to achieve dimensionality reduction, and the data obtained from the dimensionality reduction is used as the coding feature to be processed.

[0150] When performing pointwise convolution on the (n+1)-mth single-layer audio coding features, a 1*1 convolution is used.

[0151] For S262, the output vector of the (m-1)th decoding layer and the element value at the same position of the encoded feature to be processed are added together, and the data obtained by the addition is used as the m-th feature to be processed.

[0152] For example, the element value of the output vector of the (m-1)th decoding layer in the b-th row and c-th column is added to the element value of the encoding feature to be processed in the b-th row and c-th column, and the data obtained by addition is used as the element value of the m-th feature to be processed in the b-th row and c-th column.

[0153] For S263, the m-th feature to be processed is deconvolutionally processed, and the data obtained from the deconvolution process is used as the m-th single-layer decoding feature.

[0154] For S265, the nth single-layer decoding feature is used as the spectrum to be analyzed, thereby realizing the use of the data output from the last decoding layer of the decoding module as the spectrum to be analyzed.

[0155] In one embodiment, before the step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum, the method further includes:

[0156] S0211: Obtain multiple training samples and the model to be trained;

[0157] S0212: Train the model to be trained according to each training sample and the preset objective function until the preset model training termination condition is reached, and use the model to be trained that has reached the model training termination condition as the noise reduction model.

[0158] The objective function S is expressed as: S = SISNR + MSE loss + perceptual loss + regularization term. SISNR is the signal-to-noise ratio loss of speech, MSE loss is the loss calculated based on the mean square error of the real part of the spectrogram, the mean square error of the imaginary part of the spectrogram, and the mean square error of the amplitude spectrum of the spectrogram, and perceptual loss is the perceptual loss of speech.

[0159] The conventional approach is to process the amplitude spectrum and then reconstruct the clean speech by combining it with the phase spectrum of the noisy speech. However, since the phase spectra of the noisy speech and the clean speech are not the same, and the signal-to-noise ratio (SNR) is low, the upper bound of the phase spectrum difference will be larger, which limits the upper limit of the noise reduction effect. To solve this problem, this embodiment uses SNR loss, loss calculated based on the mean square error of the real part of the spectrogram, the mean square error of the imaginary part of the spectrogram, and the mean square error of the amplitude spectrum of the spectrogram, and perceptual loss. The model training fully considers the information and perception of the real part, imaginary part, and amplitude spectrum of the spectrogram, thereby improving the noise reduction capability of the model.

[0160] For S0211, multiple training samples can be obtained from user input, multiple training samples can be obtained from the database, or multiple training samples can be obtained from third-party applications.

[0161] Each training sample includes: a spectrogram, a spectrogram calibration result, a clean speech calibration result, and a perceptual data calibration result. The spectrogram is the spectrogram obtained after performing a short-time Fourier transform on the speech sample. The clean speech calibration result is the accurate clean speech corresponding to the spectrogram. The spectrogram calibration result is the accurate spectrogram corresponding to the accurate clean speech corresponding to the spectrogram. The perceptual data calibration result is the accurate perceptual data corresponding to the accurate clean speech corresponding to the spectrogram.

[0162] The speech sample is obtained by mixing clean speech and noisy speech.

[0163] For S0212, one of the training samples is selected as the target training sample; the spectrogram of the target training sample is input into the model to be trained for noise reduction processing to obtain a spectrogram prediction result; speech signal is reconstructed based on the spectrogram prediction result to obtain a clean speech prediction result; the spectrogram prediction result, the clean speech prediction result, the clean speech calibration result of the target training sample, the spectrogram calibration result, and the perception data calibration result are input into the objective function to calculate the loss value; the calculated loss value is used to update the network parameters of the model to be trained, and the updated model to be trained is used to calculate the spectrogram prediction result in the next iteration; the step of selecting one of the training samples as the target training sample is repeated until the model training termination condition is met; the model to be trained that meets the model training termination condition is used as the noise-reduced model.

[0164] The model training termination conditions include: the loss value of the model to be trained reaches the first convergence condition or the number of iterations of the model to be trained reaches the second convergence condition.

[0165] The first convergence condition means that the loss value of the model to be trained no longer decreases in two consecutive calculations.

[0166] The second convergence condition is when the training metric no longer improves. For example, the training metric is the signal-to-noise ratio loss.

[0167] After speech undergoes a short-time Fourier transform, it will yield real and imaginary components. The real part of the spectrogram refers to the components of the real part. The imaginary part of the spectrogram refers to the components of the imaginary part.

[0168] The Short-Time Fourier Transform (SFT) is a general-purpose tool for speech signal processing. It defines a very useful class of time and frequency distributions, specifying the complex amplitude of any signal as it changes with time and frequency. The amplitude spectrum of the spectrogram is the complex amplitude obtained from the SFT.

[0169] The regularization term is user-defined; it's an L2 norm regularization, applied to the regularization of the weight values ​​in the functions corresponding to the signal-to-noise ratio loss and the MSE loss. By incorporating the regularization term into the objective function, the model tends to select the model with smaller parameters during gradient descent, thus reducing the model's elasticity and mitigating overfitting to some extent.

[0170] The L2 norm is the Euclidean norm.

[0171] SISNR, short for scale-invariant source-to-noise ratio, is the signal-to-noise ratio that is unaffected by changes in the signal. The loss function corresponding to SISNR will not be elaborated here.

[0172] Perceptual loss includes: LMS (Log Mel Spectra) and PMSQE (Perceptual Metric for Speech Quality Evaluation).

[0173] In another embodiment of this application, the objective function S is expressed as: S = SISNR + MSE loss + regularization term.

[0174] Reference Figure 2 This application also proposes an artificial intelligence-based speech noise reduction device, the device comprising:

[0175] The data acquisition module 100 is used to acquire the denoising spectrum corresponding to the speech to be denoised;

[0176] The noise reduction processing module 200 is used to input the spectrum to be denoised into a preset noise reduction model for noise reduction processing to obtain a denoised spectrum. The noise reduction model includes, in sequence, an encoding module, a frequency domain noise reduction module, a time domain noise reduction module, a decoding module, and a mask gain and loss module. The frequency domain noise reduction module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain noise reduction module includes at least two time domain noise reduction sub-modules.

[0177] The speech signal reconstruction module 300 is used to reconstruct the speech signal from the noise-reduced spectrogram to obtain the target clean speech.

[0178] This embodiment implements a noise reduction model that sequentially performs feature extraction, frequency domain noise reduction, time domain noise reduction, decoding, and mask gain and loss. Effective noise reduction is achieved by sequentially performing frequency and time domain noise reduction, improving the noise reduction effect. Separating frequency and time domain noise reduction decouples the time and frequency domains, which is beneficial for streaming speech noise reduction. The time domain noise reduction module uses at least two time domain noise reduction sub-modules to achieve grouped time domain noise reduction, reducing the amount of computation and network parameters, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain noise reduction module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing the noise reduction effect.

[0179] Reference Figure 3 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as artificial intelligence-based speech denoising methods. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an artificial intelligence-based speech denoising method. The AI-based speech denoising method includes: acquiring a denoised spectrogram corresponding to the speech to be denoised; inputting the denoised spectrogram into a preset denoising model for denoising processing to obtain a denoised spectrogram, wherein the denoising model sequentially includes: an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module; the frequency domain denoising module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism; the time domain denoising module includes: at least two time domain denoising sub-modules; and reconstructing the speech signal from the denoised spectrogram to obtain the target clean speech.

[0180] This embodiment implements a noise reduction model that sequentially performs feature extraction, frequency domain noise reduction, time domain noise reduction, decoding, and mask gain and loss. Effective noise reduction is achieved by sequentially performing frequency and time domain noise reduction, improving the noise reduction effect. Separating frequency and time domain noise reduction decouples the time and frequency domains, which is beneficial for streaming speech noise reduction. The time domain noise reduction module uses at least two time domain noise reduction sub-modules to achieve grouped time domain noise reduction, reducing the amount of computation and network parameters, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain noise reduction module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing the noise reduction effect.

[0181] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements an artificial intelligence-based speech denoising method, including the steps of: acquiring a denoised spectrogram corresponding to the speech to be denoised; inputting the denoised spectrogram into a preset denoising model for denoising processing to obtain a denoised spectrogram, wherein the denoising model sequentially includes: an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module; the frequency domain denoising module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism; the time domain denoising module includes: at least two time domain denoising sub-modules; and reconstructing the speech signal from the denoised spectrogram to obtain the target clean speech.

[0182] The aforementioned AI-based speech denoising method employs a denoising model to sequentially perform feature extraction, frequency domain denoising, time domain denoising, decoding, and masking gain and loss. Effective denoising is achieved by sequentially performing frequency and time domain denoising, improving the denoising effect. Separating frequency and time domain denoising decouples the time and frequency domains, facilitating streaming speech denoising. The time domain denoising module uses at least two sub-modules to achieve grouped time domain denoising, reducing computational load and network parameter count, making it suitable for applications with limited computing resources and / or high real-time requirements. Furthermore, the frequency domain denoising module utilizes adjacent sub-band information to implement a multi-head self-attention mechanism, further enhancing the denoising effect.

[0183] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0185] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A speech noise reduction method based on artificial intelligence, characterized in that, The method includes: Obtain the denoising spectrum of the speech to be denoised; The spectrum to be denoised is input into a preset denoising model for denoising processing to obtain a denoised spectrum. The denoising model includes, in sequence, an encoding module, a frequency domain denoising module, a time domain denoising module, a decoding module, and a mask gain and loss module. The frequency domain denoising module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain denoising module includes at least two time domain denoising sub-modules. The denoised spectrogram is used to reconstruct the speech signal to obtain the target clean speech; The step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum includes: The spectrum to be denoised is input into the encoding module for feature extraction, resulting in multiple single-layer audio coding features and target audio coding features. The target audio coding features are input into the frequency domain noise reduction module for frequency domain noise reduction to obtain the audio features after frequency domain noise reduction. The target audio coding features and the frequency domain denoised audio features are residually concatenated to obtain the audio features to be processed. The audio features to be processed are input into the temporal denoising module for feature grouping, grouped temporal denoising, and feature concatenation to obtain the temporally denoised audio features. The audio features after frequency domain denoising and the audio features after time domain denoising are residually concatenated to obtain the audio features to be decoded. Each of the single-layer audio coding features and the audio features to be decoded are input into the decoding module for decoding to obtain the spectrum diagram to be analyzed. The spectrum to be analyzed is input into the masking gain and subtraction module for masking to obtain the noise-reduced spectrum. The masking gain and subtraction module uses 0 and 1 for masking.

2. The artificial intelligence-based speech noise reduction method according to claim 1, characterized in that, The step of inputting the target audio encoding features into the frequency domain noise reduction module for frequency domain noise reduction to obtain the frequency domain noise-reduced audio features includes: The dimensionality reduction submodule of the frequency domain noise reduction module is used to perform dimensionality reduction processing on the target audio coding features to obtain the dimensionality-reduced coding features. The multi-head self-attention submodule of the frequency domain denoising module is used to perform frequency domain denoising on the dimensionality-reduced encoded features to obtain the dimensionality-upgraded encoded features. The multi-head self-attention submodule is a module that implements the multi-head self-attention mechanism. The Query, Key, and Value values ​​of the multi-head self-attention mechanism of the frequency domain denoising module are determined based on the preset dependent frequency band width and adjacent sub-band information. The dimensionality-upgrading submodule of the frequency domain noise reduction module is used to perform dimensionality-upgrading processing on the encoded features to be upgraded, thereby obtaining the audio features after frequency domain noise reduction.

3. The artificial intelligence-based speech noise reduction method according to claim 1, characterized in that, The step of inputting the audio features to be processed into the temporal denoising module for feature grouping, grouped temporal denoising, and feature concatenation to obtain temporally denoised audio features includes: The feature grouping layer of the time-domain noise reduction module is used to divide the audio features to be processed into multiple single audio features, wherein the number of single audio features is the same as the number of time-domain noise reduction sub-modules. The i-th single audio feature is input into the i-th temporal denoising submodule for temporal denoising to obtain the i-th audio feature to be combined, where i is an integer greater than 0; The feature combination layer of the time-domain noise reduction module is used to perform feature concatenation on each of the audio features to be combined to obtain the audio features after time-domain noise reduction.

4. The artificial intelligence-based speech noise reduction method according to claim 1, characterized in that, The step of inputting the spectrum to be denoised into the encoding module for feature extraction to obtain multiple single-layer audio coding features and target audio coding features includes: Using the k-th encoding layer of the encoding module, a pointwise convolution is performed on the input vector of the k-th encoding layer to obtain the first audio feature; Obtain the preset depthwise convolution time dimension; If the time dimension of the Depthwise convolution is equal to 1, the k-th encoding layer is used to perform a regular convolution on the first audio feature to obtain the k-th single-layer audio encoding feature; If the time dimension of the Depthwise convolution is equal to 2, the k-th coding layer is used to perform causal convolution on the first audio feature to obtain the k-th single-layer audio coding feature; The nth single-layer audio coding feature is taken as the target audio coding feature; Where k is an integer greater than 0, k is less than or equal to n, n is greater than 0, and n is the number of coding layers; when k equals 1, the spectrum to be denoised is used as the input vector of the kth coding layer, and when k is greater than 1, the (k-1)th single-layer audio coding feature is used as the input vector of the kth coding layer.

5. The artificial intelligence-based speech noise reduction method according to claim 3, characterized in that, The step of inputting each of the single-layer audio coding features and the audio features to be decoded into the decoding module for decoding to obtain the spectrogram to be analyzed includes: The (n+1)-mth single-layer audio coding feature is dimensionality reduced to obtain the coding feature to be processed, where m is an integer greater than 0 and m is less than or equal to n; The output vector of the (m-1)th decoding layer and the coded feature to be processed are added together at the same position to obtain the m-th feature to be processed; The m-th feature to be processed is deconvolutionally processed to obtain the m-th single-layer decoded feature; The nth single-layer decoding feature is taken as the spectrum to be analyzed. Wherein, when m equals 1, the audio feature to be decoded is used as the output vector of the (m-1)th decoding layer, and when m is greater than 1, the (m-1)th single-layer decoding feature is used as the output vector of the (m-1)th decoding layer.

6. The artificial intelligence-based speech noise reduction method according to claim 1, characterized in that, Before the step of inputting the spectrum to be denoised into a preset denoising model for denoising processing to obtain the denoised spectrum, the method further includes: Obtain multiple training samples and the model to be trained; The model to be trained is trained according to each training sample and the preset objective function until the preset model training termination condition is reached. The model to be trained that has reached the model training termination condition is used as the noise reduction model. The objective function S is expressed as: S = SISNR + MSE loss + perceptual loss + regularization term, where SISNR is the signal-to-noise ratio loss of speech, MSE loss is the loss calculated based on the mean square error of the real part of the spectrogram, the mean square error of the imaginary part of the spectrogram, and the mean square error of the amplitude spectrum of the spectrogram, and perceptual loss is the perceptual loss of speech.

7. An artificial intelligence-based speech noise reduction device, used to perform the method according to any one of claims 1-6, characterized in that, The device includes: The data acquisition module is used to acquire the denoised spectrogram corresponding to the speech to be denoised; The noise reduction processing module is used to input the spectrum to be denoised into a preset noise reduction model for noise reduction processing to obtain a denoised spectrum. The noise reduction model includes, in sequence, an encoding module, a frequency domain noise reduction module, a time domain noise reduction module, a decoding module, and a mask gain and loss module. The frequency domain noise reduction module is a module that uses adjacent sub-band information to implement a multi-head self-attention mechanism. The time domain noise reduction module includes at least two time domain noise reduction sub-modules. The speech signal reconstruction module is used to reconstruct the speech signal from the noise-reduced spectrogram to obtain the target clean speech.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images