Self-supervised speech enhancement method and system based on efficient local attention

By introducing efficient local attention blocks and attention mechanisms into the self-supervised speech enhancement method, the problem of local information loss in speech reconstruction is solved, and high-quality speech enhancement effect is achieved.

CN119028368APending Publication Date: 2024-11-26JIANGSU ZHIHENG INFORMATION TECH SERVICES CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411262421.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing self-supervised speech enhancement method has partial local information loss during speech reconstruction, resulting in limited performance improvement.

Method used

Using a self-supervised speech enhancement method based on efficient local attention, speech features are extracted through convolutional neural networks and Transformer encoders, and an efficient local attention block is introduced into the speech enhancement module, combining horizontal and vertical attention mechanisms to capture the spatial dimension Eigenvectors, retaining more local information.

Benefits of technology

It effectively improves the voice noise reduction effect, achieves high-quality voice enhancement, ensures accurate position information of the area of ​​interest, and enriches the target position characteristics.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028368A_ABST
    Figure CN119028368A_ABST
Patent Text Reader

Abstract

The invention provides a self-supervised speech enhancement method and system based on efficient local attention. The method comprises the following steps: acquiring original noisy speech; inputting the original noisy speech into a pre-trained self-supervised speech enhancement model, and outputting enhanced speech; the processing steps of the speech enhancement model comprise: carrying out feature extraction on the original noisy speech to obtain noisy speech features; performing short-time Fourier transform on the original noisy voice to obtain a frequency spectrum voice feature; splicing the noisy speech features and the spectrum speech features to obtain spliced speech features; performing speech enhancement on the spliced speech features to obtain output speech features; performing feature fusion on the frequency spectrum voice feature and the output voice feature to obtain an enhanced voice feature; and performing short-time inverse Fourier transform on the enhanced speech features to obtain enhanced speech. According to the invention, high-quality speech enhancement can be realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech signal processing, and in particular to a self-supervised speech enhancement method based on efficient local attention. Background Art

[0002] Speech enhancement is an important research branch in the field of speech signal processing. The research on this technology has important theoretical value and application prospects. In the communication process, speech is usually interfered by various background noises, such as environmental noise, background noise and reverberation. These noises will greatly damage the quality and intelligibility of speech, reduce people's auditory perception of speech, and thus make incorrect extraction and judgment of information in speech. The main purpose of speech enhancement is to eliminate the interference components that may exist in the speech signal and restore the clean target speech signal, thereby improving the quality and intelligibility of speech. Common speech enhancement algorithms can be divided into two categories: traditional speech enhancement algorithms and speech enhancement algorithms based on deep learning.

[0003] Most traditional speech enhancement algorithms are based on digital signal processing technology, and mainly use the idea of ​​filtering noise reduction and noise estimation to filter out noise. Common traditional speech enhancement algorithms include spectral subtraction, filtering, spectral estimation algorithm based on minimum mean square error, and subspace-based algorithm. However, the above traditional speech enhancement methods are all unsupervised speech enhancement methods, which are highly dependent on the prediction of noise distribution laws. If under the condition of stationary noise, the above methods can achieve good speech enhancement effects. But in real life, most of the noise is non-stationary noise. When the environment changes, the performance of traditional speech enhancement algorithms decreases significantly, and good speech enhancement effects cannot be obtained.

[0004] In order to overcome the problems of traditional speech enhancement algorithms, supervised speech enhancement methods based on deep learning have made outstanding contributions to practical applications. However, supervised speech enhancement methods rely on a large amount of labeled data, and collecting this data is time-consuming and unrealistic. Self-supervised learning methods aim to use a large amount of unlabeled data to extract meaningful latent representations and use these latent representations as input for downstream tasks. However, there are few studies on applying the characteristics of self-supervised learning models to speech enhancement. The problem is that some local feature information required for speech reconstruction tasks is lost in deeper layers, resulting in very limited performance improvements in speech enhancement. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a self-supervised speech enhancement method based on efficient local attention, which can make up for the performance defects of the self-supervised speech enhancement method caused by the loss of some local information and achieve high-quality speech enhancement.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] On the one hand, the present invention provides a self-supervised speech enhancement method based on efficient local attention, comprising:

[0008] Get the original noisy speech;

[0009] Inputting the original noisy speech into a pre-trained self-supervised speech enhancement model to output enhanced speech;

[0010] The processing steps of the speech enhancement model include:

[0011] Extracting features from the original noisy speech to obtain noisy speech features;

[0012] Performing short-time Fourier transform on the original noisy speech to obtain spectral speech features;

[0013] Concatenating the noisy speech feature and the spectral speech feature to obtain a concatenated speech feature;

[0014] Performing speech enhancement on the concatenated speech features to obtain output speech features;

[0015] Fusing the spectral speech feature with the output speech feature to obtain an enhanced speech feature;

[0016] Performing short-time inverse Fourier transform on the enhanced speech feature to obtain enhanced speech.

[0017] Optionally, a feature extraction module is used to extract features from the original noisy speech, and the feature extraction module includes a convolutional neural network encoder and a Transformer encoder;

[0018] A convolutional neural network encoder is used to extract local features of the original noisy speech, and a Transformer encoder is used to extract global features of the local features.

[0019] Optionally, a speech enhancement module is used to perform speech enhancement on the concatenated speech features, wherein the speech enhancement module includes a first linear layer, a second linear layer, a bidirectional long short-term memory network, an efficient local attention block, and a Sigmoid activation function;

[0020] Using a first linear layer to transform the concatenated speech features into feature dimensions, thereby obtaining speech features after the dimension transformation;

[0021] Using a bidirectional long short-term memory network to capture bidirectional feature information of the speech features after the dimension transformation, so as to obtain bidirectional speech features;

[0022] Using an efficient local attention block to locate the region of interest of the bidirectional speech feature to obtain the speech feature of interest;

[0023] Using a second linear layer to determine the feature dimension of the speech feature of interest, to obtain the speech feature after the dimension is determined;

[0024] A Sigmoid activation function is used to perform probability range mapping on the speech features after the dimension is determined to obtain output speech features.

[0025] Optionally, the bidirectional speech feature locates the region of interest through an efficient local attention block, and the efficient local attention block includes a horizontal attention mechanism and a vertical attention mechanism, specifically:

[0026] In the horizontal attention mechanism, the bidirectional speech feature sequentially extracts horizontal feature vectors, local feature interactions, normalization, and probability mapping to obtain horizontal position features;

[0027] In the vertical attention mechanism, the bidirectional speech feature extracts vertical feature vectors, local feature interactions, normalization and probability mapping in sequence to obtain vertical position features;

[0028] The horizontal position feature and the vertical position feature are multiplied to obtain the speech feature of interest.

[0029] Optionally, the training of the self-supervised speech enhancement model includes: optimizing the parameters of the self-supervised speech enhancement model using a joint loss function synthesized from a mean square error loss function of an audio waveform and a minimum absolute error loss function of an audio spectrum.

[0030] Optionally, the joint loss function is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] In the formula, is the joint loss function; To enhance speech; For pure voice; is the mean square error loss weight coefficient of the audio waveform; The mean square error loss function of the audio waveform between the enhanced speech and the clean speech; The minimum absolute error loss function of the audio spectrum between the enhanced speech and the clean speech; is the sample size; To enhance the real part of the speech spectrum; is the real part of the pure speech spectrum; To enhance the imaginary part of the speech spectrum; is the imaginary part of the pure speech spectrum.

[0035] Optionally, before inputting the original noisy speech into a pre-trained self-supervised speech enhancement model, the original noisy speech is preprocessed to obtain original noisy speech with a fixed sampling rate and length.

[0036] Optionally, the original noisy speech is subjected to short-time Fourier transform and then log1p normalization to obtain spectral speech features.

[0037] In a second aspect, the present invention provides a self-supervised speech enhancement system based on efficient local attention, comprising:

[0038] A speech acquisition module, used to acquire original noisy speech;

[0039] A feature extraction module, used for extracting features from original noisy speech;

[0040] A feature transformation module, used for performing short-time Fourier transformation on the original noisy speech or performing inverse short-time Fourier transformation on the enhanced speech features;

[0041] A feature concatenation module, used for concatenating noisy speech features and spectral speech features;

[0042] A speech enhancement module, used for performing speech enhancement on the concatenated speech features;

[0043] The feature fusion module is used to fuse the spectral speech features and the output speech features.

[0044] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the self-supervised speech enhancement method based on efficient local attention described in the first aspect.

[0045] Beneficial Effects

[0046] Compared with the prior art, the beneficial effects achieved by this aspect are:

[0047] (1) The present invention effectively compensates for the performance defects of existing self-supervised speech enhancement methods caused by the loss of some local information during the speech reconstruction process, retains more local information to meet the needs of speech reconstruction, effectively improves the effect of speech denoising, and achieves high-quality speech enhancement.

[0048] (2) The present invention introduces an efficient local attention block in the speech enhancement module. The efficient local attention block obtains feature vectors in the horizontal and vertical directions in the spatial dimension, while maintaining a narrow kernel shape to capture long-distance dependencies and prevent irrelevant regions from affecting label prediction, thereby generating rich target position features in their respective directions to ensure accurate position information of the region of interest, thereby retaining more local information for speech reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 FIG. 1 is a flow chart of a self-supervised speech enhancement method based on efficient local attention in one embodiment of the present invention;

[0050] Figure 2 FIG. 1 is a schematic diagram of the structure of a feature extraction module in an embodiment of the present invention;

[0051] Figure 3 The figure is a schematic diagram of the structure of a speech enhancement module in one embodiment of the present invention;

[0052] Figure 4 FIG. 4 is a schematic diagram of the structure of an efficient local attention block of the present invention in one embodiment. DETAILED DESCRIPTION

[0053] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0054] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship.

[0055] Example 1

[0056] like Figure 1 As shown, the present invention introduces a self-supervised speech enhancement method based on efficient local attention, which specifically includes the following steps:

[0057] Get the original noisy speech .

[0058] The original noisy speech Input a pre-trained self-supervised speech enhancement model and output enhanced speech .

[0059] In this embodiment, the training corpus comes from the speech data in the VoiceBank-DEMAND corpus, which contains clean speech and its corresponding pre-mixed noisy speech. In the training stage, the clean speech selects 11,572 voices of 28 speakers (14 males and 14 females), and the noisy speech also selects 11,572 voices of 28 speakers (14 males and 14 females), and the sampling frequency is 48kHz. Among them, the noisy speech in the training stage is a mixture of clean speech and 10 types of noise at a signal-to-noise ratio of 0dB, 5dB, 10dB and 15dB, of which 8 are from the DEMAND data set and the other 2 are self-made noises.

[0060] Enhanced Voice and pure voice Together they are used to evaluate the loss, and the parameters of the self-supervised speech enhancement model are optimized using a joint loss function synthesized by the mean square error loss function of the audio waveform and the minimum absolute error loss function of the audio spectrum. By combining these two parts, the model can be better fitted in terms of both waveform and spectrum.

[0061] The joint loss function is as follows:

[0062] ;

[0063] ;

[0064] ;

[0065] In the formula, is the joint loss function; To enhance speech; For pure voice; is the mean square error loss weight coefficient of the audio waveform; The mean square error loss function of the audio waveform between the enhanced speech and the clean speech; The minimum absolute error loss function of the audio spectrum between the enhanced speech and the clean speech; is the sample size; To enhance the real part of the speech spectrum; is the real part of the pure speech spectrum; To enhance the imaginary part of the speech spectrum; is the imaginary part of the pure speech spectrum.

[0066] Through continuous iterative training, the voice is enhanced and pure voice The difference between them is getting smaller and smaller, and the loss function of the self-supervised speech enhancement model is as small as possible until the set number of iterations is reached, thereby obtaining a trained self-supervised speech enhancement model. Due to different specific settings of the neural network and different performances of the experimental equipment, the number of iterations set is also different accordingly. In this embodiment, the number of iterations is set to 100.

[0067] In this embodiment, the test corpus comes from the speech data in the VoiceBank-DEMAND corpus, which contains clean speech and the corresponding premixed noisy speech. In the test phase, the premixed noisy speech selects 824 speech of 2 speakers (1 male and 1 female), and the clean speech also selects 824 speech of 2 speakers (1 male and 1 female), and the sampling frequency is 48kHz. Among them, the premixed noisy speech in the test phase is a mixture of clean speech and other 5 noises from the DEMAND data set with a signal-to-noise ratio of 2.5dB, 7.5dB, 12.5dB and 17.5dB.

[0068] The test corpus was preprocessed and the premixed noisy speech and clean speech were resampled to 16kHz. The test corpus contained 824 pairs of clean speech and premixed noisy speech. Each corpus was randomly sampled into a fixed premixed noisy speech with a length of 20480 (128 frames × 160 hop length).

[0069] The processing steps of the speech enhancement model include:

[0070] The original noisy speech is extracted by a feature extraction module. Perform feature extraction to obtain noisy speech features.

[0071] The feature extraction module includes a convolutional neural network encoder and a Transformer encoder.

[0072] The original noisy speech is extracted using a convolutional neural network encoder The Transformer encoder is used to extract the deeper global features of the local features, thereby obtaining the noisy speech features.

[0073] In this embodiment, the feature extraction module uses WavLM as the speech self-supervised learning model. WavLM is mainly composed of a 7-layer convolutional neural network encoder and a 12-layer Transformer encoder. Its structure is as follows: Figure 2 shown.

[0074] The original noisy speech Perform short-time Fourier transform and then log1p normalization to obtain the spectral speech features.

[0075] The noisy speech feature and the spectral speech feature are concatenated to obtain a concatenated speech feature.

[0076] A speech enhancement module is used to perform speech enhancement on the concatenated speech features to obtain output speech features.

[0077] The speech enhancement module includes a first linear layer, a second linear layer, two bidirectional long short-term memory networks, an efficient local attention block and a Sigmoid activation function, and its structure is as follows: Figure 3 shown.

[0078] The first linear layer is used to perform feature dimension transformation on the concatenated speech features to obtain speech features after dimension transformation.

[0079] A bidirectional long short-term memory network is used to capture bidirectional feature information of the speech features after the dimensional transformation to obtain bidirectional speech features. As the main framework of the speech enhancement module, the bidirectional long short-term memory network can capture bidirectional information in the input sequence to form long-term dependencies to obtain a more comprehensive acoustic feature representation, and has good performance.

[0080] An efficient local attention block is used to locate the region of interest of the bidirectional speech feature to obtain the speech feature of interest, which can accurately locate the region of interest, thereby retaining more local information.

[0081] The bidirectional speech feature locates the region of interest through an efficient local attention block, and the efficient local attention block includes a horizontal attention mechanism and a vertical attention mechanism, specifically:

[0082] In the horizontal attention mechanism, the bidirectional speech features sequentially extract horizontal feature vectors, local feature interactions, normalization, and probability mapping to obtain horizontal position features.

[0083] In the vertical attention mechanism, the bidirectional speech features sequentially extract vertical feature vectors, local feature interactions, normalization, and probability mapping to obtain vertical position features.

[0084] That is, firstly, strip pooling is used in the spatial dimension to obtain feature vectors in the horizontal and vertical directions while maintaining a narrow kernel shape. Then, a one-dimensional convolution is applied to locally interact with the two feature vectors respectively, and the obtained feature vectors are subjected to group normalization and Sigmoid activation function to generate position attention features in two directions.

[0085] In this embodiment, both the horizontal attention mechanism and the vertical attention mechanism include a strip pooling layer, a one-dimensional convolution, a group normalization, and a Sigmoid activation function, and their structures are as follows: Figure 4As shown in Figure 2. Narrower kernel shapes are used to effectively capture long-range dependencies while preventing irrelevant regions from affecting label predictions, thereby producing rich target location features in their respective directions. One-dimensional convolutions can choose to adjust the kernel size to indicate the coverage of local interactions.

[0086] The horizontal position feature and the vertical position feature are multiplied to obtain the speech feature of interest.

[0087] The second linear layer is used to determine the feature dimension of the speech feature of interest to obtain the speech feature after the dimension is determined.

[0088] The sigmoid activation function is used to perform probability range mapping on the speech features after the dimension is determined, and the output speech features can be mapped to the probability range of [0, 1], and then a prediction mask is generated as the output of the speech enhancement module.

[0089] The Sigmoid activation function formula is:

[0090] ;

[0091] in, represents the input of the activation guidance layer; represents the hyperparameter, set to 0.1; Represents an exponential function with the natural constant e as the base.

[0092] The spectral speech feature and the output speech feature are multiplied to perform feature fusion to obtain enhanced speech features.

[0093] Perform short-time Fourier inverse transform on the enhanced speech feature to restore it to the original waveform to obtain the enhanced speech .

[0094] Example 2

[0095] This embodiment introduces a self-supervised speech enhancement system based on efficient local attention, including:

[0096] The speech acquisition module is used to acquire original noisy speech.

[0097] The feature extraction module is used to extract features from the original noisy speech.

[0098] The feature transformation module is used to perform short-time Fourier transform on the original noisy speech or inverse short-time Fourier transform on the enhanced speech features.

[0099] The feature concatenation module is used to concatenate noisy speech features and spectral speech features.

[0100] The speech enhancement module is used to perform speech enhancement on the concatenated speech features.

[0101] The feature fusion module is used to fuse the spectral speech features and the output speech features.

[0102] Example 3

[0103] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the self-supervised speech enhancement method based on efficient local attention described in Example 1 are implemented.

[0104] Example 4

[0105] This embodiment introduces a computer device, including:

[0106] Memory, for storing computer instructions;

[0107] A processor is used to execute the computer instructions to implement the steps of the self-supervised speech enhancement method based on efficient local attention described in Example 1.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A self-supervised speech enhancement method based on efficient local attention, It is characterized in that include: Get the original noisy speech; Inputting the original noisy speech into a pre-trained self-supervised speech enhancement model to output enhanced speech; The processing steps of the speech enhancement model include: Extracting features from the original noisy speech to obtain noisy speech features; Performing short-time Fourier transform on the original noisy speech to obtain spectral speech features; Concatenating the noisy speech feature and the spectral speech feature to obtain a concatenated speech feature; Performing speech enhancement on the concatenated speech features to obtain output speech features; Fusing the spectral speech feature with the output speech feature to obtain an enhanced speech feature; Performing short-time inverse Fourier transform on the enhanced speech feature to obtain enhanced speech.

2. The self-supervised speech enhancement method based on efficient local attention according to claim 1, It is characterized in that Using a feature extraction module to extract features from the original noisy speech, the feature extraction module includes a convolutional neural network encoder and a Transformer encoder; A convolutional neural network encoder is used to extract local features of the original noisy speech, and a Transformer encoder is used to extract global features of the local features.

3. The self-supervised speech enhancement method based on efficient local attention according to claim 1, It is characterized in that A speech enhancement module is used to perform speech enhancement on the concatenated speech features, wherein the speech enhancement module includes a first linear layer, a second linear layer, a bidirectional long short-term memory network, an efficient local attention block, and a Sigmoid activation function; Using a first linear layer to transform the feature dimension of the concatenated speech features to obtain speech features after the dimension transformation; Using a bidirectional long short-term memory network to capture bidirectional feature information of the speech features after the dimension transformation, to obtain bidirectional speech features; Using an efficient local attention block to locate the region of interest of the bidirectional speech feature to obtain the speech feature of interest; Using a second linear layer to determine the feature dimension of the speech feature of interest, to obtain the speech feature after the dimension is determined; A Sigmoid activation function is used to perform probability range mapping on the speech features after the dimension is determined to obtain output speech features.

4. The self-supervised speech enhancement method based on efficient local attention according to claim 3, It is characterized in that The bidirectional speech feature locates the region of interest through an efficient local attention block, and the efficient local attention block includes a horizontal attention mechanism and a vertical attention mechanism, specifically: In the horizontal attention mechanism, the bidirectional speech feature sequentially extracts horizontal feature vectors, local feature interactions, normalization, and probability mapping to obtain horizontal position features; In the vertical attention mechanism, the bidirectional speech feature extracts vertical feature vectors, local feature interactions, normalization and probability mapping in sequence to obtain vertical position features; The horizontal position feature and the vertical position feature are multiplied to obtain the speech feature of interest.

5. The self-supervised speech enhancement method based on efficient local attention according to claim 1, It is characterized in that The training of the self-supervised speech enhancement model includes: optimizing the parameters of the self-supervised speech enhancement model by using a joint loss function synthesized by a mean square error loss function of an audio waveform and a minimum absolute error loss function of an audio spectrum.

6. According to the self-supervised speech enhancement method based on efficient local attention as described in claim 5, It is characterized in that The joint loss function is as follows: ; ; ; In the formula, is the joint loss function; To enhance speech; For pure voice; is the mean square error loss weight coefficient of the audio waveform; The mean square error loss function of the audio waveform between the enhanced speech and the clean speech; The minimum absolute error loss function of the audio spectrum between the enhanced speech and the clean speech; is the sample size; To enhance the real part of the speech spectrum; is the real part of the pure speech spectrum; To enhance the imaginary part of the speech spectrum; is the imaginary part of the pure speech spectrum.

7. The self-supervised speech enhancement system based on efficient local attention according to claim 1, It is characterized in that Before inputting the original noisy speech into the pre-trained self-supervised speech enhancement model, the original noisy speech is preprocessed to obtain the original noisy speech with fixed sampling rate and length.

8. The self-supervised speech enhancement system based on efficient local attention according to claim 1, It is characterized in that After performing short-time Fourier transform on the original noisy speech, log1p normalization is performed to obtain spectral speech features.

9. A self-supervised speech enhancement system based on efficient local attention, It is characterized in that include: A speech acquisition module, used to acquire original noisy speech; A feature extraction module, used for extracting features from original noisy speech; A feature transformation module, used for performing short-time Fourier transformation on the original noisy speech or performing inverse short-time Fourier transformation on the enhanced speech features; A feature concatenation module, used for concatenating noisy speech features and spectral speech features; A speech enhancement module, used for performing speech enhancement on the concatenated speech features; The feature fusion module is used to fuse the spectral speech features and the output speech features.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the self-supervised speech enhancement method based on efficient local attention described in any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Underwater acoustic signal noise reduction method and device, computer equipment and readable storage medium

    CN120496487A

  • Underwater acoustic signal denoising method and device, computer device and readable storage medium

    CN120496487B

  • Robust noise reduction processing method and system for sound wave signal self-supervised learning enhancement

    CN121415799A