Multi-self-attention domain method and system for speech enhancement

By adopting the multi-self-attention domain method in the speech enhancement technology, using the self-attention module and trainable coefficients of parallel input, the problem of convolutional layer receptive field limitation in the existing technology is solved, and more efficient speech enhancement is achieved Effect.

CN119479621AInactive Publication Date: 2025-02-18SUZHOU UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510040343.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-10
Publication Date
2025-02-18
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing speech enhancement technologies, especially SEGAN-based variants, relying on convolutional layers limits the ability to capture remote dependencies of input sequences, resulting in limited performance.

Method used

Using the multi-self-attention domain method, by constructing the network architecture of the MSAF model, voice information is input in parallel to multiple self-attention modules, and the output of each module is multiplied by the trainable coefficients to obtain the enhanced Clean voice.

Benefits of technology

By fusing the self-attention module of the self-attention layer at different locations, clean features in speech can be learned from multiple dimensions to achieve better speech enhancement effects, which is significantly improved in multiple evaluation indicators compared with the SASEGAN method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119479621A_ABST
    Figure CN119479621A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of voice enhancement, and discloses a multi-self-attention domain method and system for voice enhancement, and the method comprises the steps: collecting voice information containing noise; constructing a network architecture of the MSAF model; inputting the voice information into N self-attention modules in parallel; and multiplying the output of each self-attention module by a trainable coefficient # imgabs0 # which is randomly sampled from Gaussian distribution to obtain a finally output enhanced clean voice. By fusing the self-attention modules of the self-attention layers at different positions, clean speech features in the speech containing noise can be learned from multiple dimensions, so that a better speech enhancement effect is achieved. The method has comparability in the aspect of improving voice quality, and is excellent in the aspects of segmented SNR and voice clarity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech enhancement, and in particular to a multi-self-attention domain method and system for speech enhancement. Background Art

[0002] In recent years, more and more voice-related applications have become popular, including controlling smart devices through voice commands, supporting voice search in various scenarios, real-time meeting recording, etc. In real-world scenarios, this will affect the performance of voice-related applications. Speech enhancement (SE) is used to attenuate noise in speech signals, thereby reducing the impact of noise on speech-related applications.

[0003] Recently, SEGAN-based variants have improved SEGAN in various aspects. In terms of input types, raw waveforms and spectrograms have been exploited. However, these SEGAN variants still rely heavily on convolutional layers. Receptive field, as the convolution operation limits the ability of SEGAN and SEGAN variants to capture long-range dependencies of input sequences. To address this issue, several solutions have been proposed. The results show that in all objective evaluation metrics, the performance of SASEGAN is affected by the position of the self-attention layer. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] To solve the above technical problems, the present invention provides the following technical solution: a multi-self-attention domain method for speech enhancement, comprising: Collecting voice information containing noise; Construct the network architecture of the MSAF model; Inputting the speech information into N self-attention modules in parallel; The output of each self-attention module is combined with a trainable coefficient randomly sampled from a Gaussian distribution Multiply them together to get the final output of enhanced and clean speech.

[0006] As a preferred solution of the multi-self-attention domain method for speech enhancement described in the present invention, wherein: the network architecture of the MSAF model is composed of multiple self-attention speech enhancement generative adversarial networks and weight coefficients of multiple different domain models; The self-attention speech enhancement generative adversarial network includes a generator and a discriminator.

[0007] As a preferred solution of the multi-self-attention domain method for speech enhancement described in the present invention, wherein: the generator is composed of an encoder and a decoder, the input noisy audio passes through a convolution layer, and the deconvolution layer outputs the enhanced speech; by inserting a convolution layer after a specific convolution f layer, a self-attention domain module in the overall architecture is formed, The field label representing the self-attention layer; the specific convolutional layer f can be any convolutional layer; The specific convolution f layer includes, in N self-attention speech enhancement generation adversarial networks, each self-attention speech enhancement generation adversarial network has a convolution layer set as a specific convolution layer, and the specific convolution layers in the N self-attention speech enhancement generation adversarial networks are respectively at different positions; Each convolutional layer of the encoder is jump-connected to the convolutional layer of the decoder, and the self-attention layer can be placed after any convolutional layer in the encoder; In the decoder, the positions of the self-attention layers mirror those of the encoder: for every convolutional f layers in the encoder, there are corresponding deconvolutional f layers in the decoder; The network architecture of the discriminator is the same as the encoder of the generator. The input of the discriminator is a pair of audio , the output is 0 / 1, where 0 means the discriminator determines that the input audio is noisy speech, and 1 means the discriminator determines that the input audio is clean audio; in, represents the speech generated by the generator, Represented as real and clean speech.

[0008] As a preferred solution of the multi-self-attention domain method for speech enhancement described in the present invention, wherein: the network architecture of the MSAF model further includes placing the self-attention layer after different convolutional layers of N self-attention speech enhancement generative adversarial networks respectively; SASEGAN-N means that the self-attention layer is placed after the Nth convolutional layer in the self-attention speech enhancement generative adversarial network, resulting in N input channels.

[0009] As a preferred solution of the multi-self-attention domain method for speech enhancement described in the present invention, wherein: the self-attention speech enhancement generative adversarial network includes improving the adversarial training of GAN by improving the objective functions of the discriminator and the generator.

[0010] As a preferred embodiment of the multi-self-attention domain method for speech enhancement described in the present invention, the objective function includes the objective function of the discriminator: , The objective function of the generator is: , in, represents the discriminator output; Represents the output of the self-attention layer at the generator of the lth convolutional layer; Represents a clean audio signal; , represents an audio signal corrupted by noise from noise-corrupted audio signals; Indicates the length of time; Indicates that x follows the (x, z) distribution; represents the objective function of minimizing the optimization generator G; Denote the objective function to minimize the optimization generator D; the noise-corrupted audio signal Defined as ; represents the additive background noise signal; Indicates SASEGAN- The weight coefficient of the output of MSAF to the output of ; is a hyperparameter used to adjust the proportion of loss in the full objective loss function; is a matrix randomly sampled from a Gaussian distribution, The size of is the same as the encoder output size.

[0011] As a preferred solution of the multi-self-attention domain method for speech enhancement described in the present invention, wherein: a noisy speech signal is input into the network architecture of the MSAF model, and each self-attention module in the architecture performs data processing in parallel; The noisy speech signal is simultaneously input into N self-attention modules, and the outputs of N self-attention modules are obtained at the same time, and then the N outputs are combined with the coefficients Multiply and then sum to get the final enhanced clean speech signal.

[0012] A multi-self-attention domain system for speech enhancement using any method as described in the present invention, characterized in that: A collection unit, collecting voice information containing noise; An analysis unit constructs a network architecture of the MSAF model; inputs the speech information into N self-attention modules in parallel; The output unit multiplies the output of each self-attention module by a trainable coefficient to obtain the final output of the enhanced and clean speech.

[0013] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0014] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0015] Beneficial effects of the present invention: The multi-self-attention domain method for speech enhancement provided by the present invention can learn clean speech features in noisy speech from multiple dimensions by fusing self-attention modules of self-attention layers at different positions, thereby achieving better speech enhancement effects. In terms of speech quality, the multi-self-attention domain method for speech enhancement proposed by the present invention is improved by 8.22%, 8.52%, 9.28% and 9.40% respectively in CBAK, CSIG, COVL and PESQ indicators compared to the SASEGAN method. In terms of SSNR and STOI indicators, absolute gains of 1.71 and 0.77 are obtained respectively compared to the SASEGAN method. Moreover, the multi-self-attention domain method for speech enhancement provided by the present invention is superior to the mainstream speech enhancement method based on GAN. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 An overall flow chart of a multi-self-attention domain method for speech enhancement provided by the first embodiment of the present invention; Figure 2 A schematic diagram of MSAF in a multi-self-attention domain method for speech enhancement provided by the first embodiment of the present invention; Figure 3 A network architecture diagram of a multi-self-attention domain method for speech enhancement provided by the first embodiment of the present invention, wherein (a) is a generator, and (b) is a discriminator; Figure 4 An expression diagram of N channels of a MSAF of a multi-self-attention domain method for speech enhancement provided by the first embodiment of the present invention; Figure 5A comparison chart of PESQ, CSIG, CBAK and COVL results of a multi-self-attention domain method for speech enhancement and a comparative method provided in the second embodiment of the present invention; Figure 6 A comparison chart of SSNR and STOI results of a multi-self-attention domain method and a comparison method for speech enhancement provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0019] Example 1, reference Figure 1-Figure 4 , as an embodiment of the present invention, provides a multi-self-attention domain method for speech enhancement, comprising: like Figure 2 As shown in the figure, the input is a noisy speech, which is then input into N SA modules (self-attention modules) in parallel. The output of these N modules is multiplied by a trainable coefficient to obtain the final output of enhanced clean speech. Each model inputs the same training set when using different algorithms. Taking N models as an example, it represents the weight t coefficient of the model output in the final output. It allows the basic model to be trained in parallel. Noisy represents an audio signal corrupted by noise, and clean represents a clean audio signal. The SA field indicates that the model has different feature receptive fields for noise. Represents the weight coefficients of different field models. By using a model with multiple fields, the noisy signal is enhanced into a clean signal.

[0020] S1: Collecting speech information containing noise.

[0021] S2: Network architecture for building the MSAF model.

[0022] What we want to say is that for speech enhancement, the goal is to find a function to restore the clean audio signal ,in Represents a clean audio signal and Represents an audio signal corrupted by noise Audio signal corrupted by noise . Indicates the length of time. Noise-corrupted audio signal Defined as ,in Represents the additive background noise signal. In order to improve the network's attention on the correlation of the time dimension, a self-attention speech enhancement generative adversarial network is proposed. The network architecture of the MSAF basic model is shown in Figure 3.

[0023] The network architecture of the MSAF model is composed of multiple self-attention speech enhancement generation adversarial networks and weight coefficients of multiple different domain models. The self-attention speech enhancement generation adversarial network includes a generator and a discriminator.

[0024] The generator consists of an encoder and a decoder, such as Figure 3 As shown in (a), each convolutional layer of the encoder is skip-connected to the convolutional layer of the decoder. is a matrix randomly sampled from a Gaussian distribution, The size of is the same as the size of the encoder output. On this basis, the robustness of the network is improved. The gray blocks represent self-attention layers, which can be placed after any convolutional layer in the encoder. In the decoder, the position of the self-attention layer reflects the position of the encoder. represents the field label of the self-attention layer. The network structure of the discriminator is as follows Figure 3 (b) The discriminator consists of several convolutional layers and a self-attention layer.

[0025] The input noisy audio passes through the convolution layer, and the deconvolution layer outputs the enhanced speech. By inserting a convolution layer after a specific convolution layer f, the self-attention domain module in the overall architecture is formed. The field label representing the self-attention layer; where the specific convolutional layer f can be any convolutional layer.

[0026] The specific convolution f layer includes, in N self-attention speech enhancement generative adversarial networks, each self-attention speech enhancement generative adversarial network has a convolution layer set as a specific convolution layer, and the specific convolution layers in the N self-attention speech enhancement generative adversarial networks are respectively at different positions.

[0027] Each convolutional layer of the encoder is jump-connected to the convolutional layers of the decoder, and the self-attention layer can be placed after any convolutional layer in the encoder.

[0028] In the decoder, the positions of the self-attention layers mirror those of the encoder: for every convolutional f layers in the encoder, there are corresponding deconvolutional f layers in the decoder.

[0029] The network architecture of the discriminator is the same as the encoder of the generator. The input of the discriminator is a pair of audio , the output is 0 / 1, where 0 means the discriminator determines that the input audio is noisy speech, and 1 means the discriminator determines that the input audio is clean audio. represents the speech generated by the generator, Represented as real and clean speech.

[0030] The self-attention layer is placed after different convolutional layers of N self-attention speech enhancement generative adversarial networks. Figure 4 As shown in the figure, SASEGAN-1 means that the self-attention layer is placed after the first convolutional layer in SASEGAN, and SASEGAN-N means that the self-attention layer is placed after the Nth convolutional layer in the self-attention speech enhancement generative adversarial network, resulting in N input channels.

[0031] For the discriminator, the network architecture is similar to the encoder of the generator. The difference is that the input of the discriminator is a pair of audio .

[0032] Due to the uncertainty of the position of the self-attention layer in SASEGAN on the speech enhancement effect, we observe that the self-attention layer learns different features at different positions in SASEGAN. As shown in Figure 4, the base models with self-attention layers at different positions are fused through bagging. Assume that N is set to 11.

[0033] By improving the objective functions of the discriminator and generator, the adversarial training of GAN can be improved. Here, the least squares loss is used. We make some improvements to it, and the objective functions of the discriminator and generator are written as: The objective function of the discriminator is: , The objective function of the generator is: , in, represents the discriminator output; Represents the output of the self-attention layer at the generator of the lth convolutional layer; Represents a clean audio signal; , represents an audio signal corrupted by noise from noise-corrupted audio signals; Indicates the length of time; Indicates that x follows the (x, z) distribution; represents the objective function of minimizing the optimization generator G; Denote the objective function to minimize the optimization generator D; the noise-corrupted audio signal Defined as ; represents the additive background noise signal; Indicates SASEGAN- The weight coefficient of the output of MSAF to the output of ; is a hyperparameter used to adjust the proportion of loss in the full objective loss function; is a matrix randomly sampled from a Gaussian distribution, The size of is the same as the encoder output size.

[0034] S3: Input the speech information into N self-attention modules in parallel. Compare the output of each self-attention module with a trainable coefficient randomly sampled from a Gaussian distribution. Multiply them together to get the final output of enhanced and clean speech.

[0035] The noisy speech signal is input into the network architecture of the MSAF model, and each self-attention module in the architecture performs data processing in parallel.

[0036] The noisy speech signal is simultaneously input into N self-attention modules, and the outputs of N self-attention modules are obtained at the same time, and then the N outputs are combined with the coefficients Multiply and then sum to get the final enhanced clean speech signal. Among them, the coefficient It is trainable; the coefficients are initialized by random sampling from a Gaussian distribution ;coefficient Updated during the training process. During the training process, the training set and validation set of the self-attention module and coefficient are the same; the specific training steps are: first train the N self-attention modules separately, and then fix the self-attention module after training; then multiply the output of the self-attention module by the coefficient, and then train each coefficient to obtain the trained coefficient.

[0037] On the other hand, this embodiment also provides a multi-self-attention domain system for speech enhancement, which includes: The collection unit collects voice information containing noise.

[0038] The analysis unit constructs the network architecture of the MSAF model; and inputs the speech information into N self-attention modules in parallel.

[0039] The output unit multiplies the output of each self-attention module by a trainable coefficient to obtain the final output of the enhanced and clean speech.

[0040] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0041] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0042] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0043] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0044] Example 2, reference Figure 5 , Figure 6 , which is an embodiment of the present invention, provides a multi-self-attention domain method for speech enhancement. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0045] The proposed method is compared with the baseline method SASEGAN and the performance is evaluated in terms of PESQ (speech quality), CBAK (intrusiveness of background noise), CSIG (speech signal distortion), COVL (overall effectiveness), SSNR (segmental signal to noise ratio), and STOI (short-time objective intelligibility).

[0046] The results of MSAF and mainstream speech enhancement methods are shown in Table 1. The higher the value in the table, the better the performance of the corresponding method. The compared methods are divided into two parts: GAN-based methods (including single-channel algorithm structures) and non-GAN-based methods (deep neural networks and TSN). In the GAN-based methods, SASEGAN-N represents SASEGAN with self-attention layers placed at different positions (the data comparison of GAN-based SE methods is, in order, single-channel algorithm structures without self-attention layers and self-attention layers placed at the 1st to 11th layers), where N is the index of the self-attention layer position. The proposed method outperforms the baseline models of CBAK, CSIG, COVL and PESQ, as shown in Figure 5 As shown in Figure 2. Compared with the baseline SASEGAN, the proposed method achieves an average improvement of 9.40%, 8.52%, 8.22%, and 9.28% in PESQ, CSIG, CBAK, and COVL, respectively. Similarly, the proposed method outperforms the GAN-based SE methods ISEGAN and DSEGAN. For non-GAN-based SE methods, the proposed method outperforms DNN. TSN slightly outperforms the proposed method on PESQ, COVL, and CSIG, but the STOI and SSNR of TSN are close to the noise, as shown in Figure 2. Figure 6 shown.

[0047] For a more intuitive comparison, Figure 6 SSNR and STOI are normalized in . The higher the bar, the better the performance of the method. It can be seen that DNN and TSN perform much worse than other methods in SSNR and STOI, even worse than noise-corrupted speech. This shows that although TSN can improve speech quality, it performs poorly in terms of SSNR and speech intelligibility. We found that the GAN-based SE method can improve the STOI and SSNR of noise-corrupted speech while ensuring improved speech quality. For SSNR, MSAF achieved an average absolute gain of 1.71 compared to the baseline SASEGAN. In terms of speech intelligibility STOI, MSAF achieved an average absolute gain of 0.77.

[0048] Table 1: Comparison of objective results between MSAF and mainstream speech enhancement (SE) methods.

[0049] , The results show that the proposed method MSAF outperforms the mainstream SEGAN-based method. For non-GAN-based methods, the proposed method MSAF is comparable in improving speech quality and excels in segmented SNR and speech clarity. In addition, this method conducts ablation experiments to study the impact of the position of the self-attention layer on the results, and has obvious improvements and advantages in various indicators compared with the method with only a single channel.

[0050] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-self-attention domain method for speech enhancement, characterized in that include: Collecting voice information containing noise; Construct the network architecture of the MSAF model; Inputting the speech information into N self-attention modules in parallel; The output of each self-attention module is combined with a trainable coefficient randomly sampled from a Gaussian distribution Multiply them together to get the final output of enhanced and clean speech.

2. The multi-self-attention domain method for speech enhancement according to claim 1, characterized in that: The network architecture of the MSAF model is composed of N self-attention speech enhancement generative adversarial networks and weight coefficients of N different domain models; The self-attention speech enhancement generative adversarial network includes a generator and a discriminator.

3. The multi-self-attention domain method for speech enhancement as claimed in claim 2, characterized in that: The generator consists of an encoder and a decoder. The input noisy audio passes through the convolution layer, and the deconvolution layer outputs the enhanced speech. By inserting a convolution layer after a specific convolution layer f, the self-attention domain module in the overall architecture is formed. The field label representing the self-attention layer; the specific convolutional layer f can be any convolutional layer; The specific convolution f layer includes, in N self-attention speech enhancement generation adversarial networks, each self-attention speech enhancement generation adversarial network has a convolution layer set as a specific convolution layer, and the specific convolution layers in the N self-attention speech enhancement generation adversarial networks are respectively at different positions; Each convolutional layer of the encoder is jump-connected to the convolutional layer of the decoder, and the self-attention layer can be placed after any convolutional layer in the encoder; In the decoder, the positions of the self-attention layers mirror those of the encoder: for every convolutional f layers in the encoder, there are corresponding deconvolutional f layers in the decoder; The network architecture of the discriminator is the same as the encoder of the generator. The input of the discriminator is a pair of audio , the output is 0 / 1, where 0 means the discriminator determines that the input audio is noisy speech, and 1 means the discriminator determines that the input audio is clean audio; in, represents the speech generated by the generator, Represented as real and clean speech.

4. The multi-self-attention domain method for speech enhancement as claimed in claim 3, characterized in that: The network architecture of the MSAF model also includes placing the self-attention layer after different convolutional layers of N self-attention speech enhancement generative adversarial networks respectively; SASEGAN-N means that the self-attention layer is placed after the Nth convolutional layer in the self-attention speech enhancement generative adversarial network, resulting in N input channels.

5. The multi-self-attention domain method for speech enhancement as claimed in claim 4, characterized in that: The self-attention speech enhancement generative adversarial network includes improving the adversarial training of GAN by improving the objective functions of the discriminator and the generator.

6. The multi-self-attention domain method for speech enhancement as claimed in claim 5, characterized in that: The objective function includes the objective function of the discriminator: , The objective function of the generator is: , in, represents the discriminator output; Represents the output of the self-attention layer at the generator of the lth convolutional layer; Represents a clean audio signal; , represents an audio signal corrupted by noise from audio signals corrupted by noise; Indicates the length of time; Indicates that x follows the (x, z) distribution; represents the objective function of minimizing the optimization generator G; Denote the objective function to minimize the optimization generator D; the noise-corrupted audio signal Defined as ; represents the additive background noise signal; Indicates SASEGAN- The weight coefficient of the output of MSAF to the output of ; is a hyperparameter used to adjust the proportion of loss in the full objective loss function; is a matrix randomly sampled from a Gaussian distribution, The size of is the same as the encoder output size.

7. The multi-self-attention domain method for speech enhancement as claimed in claim 6, characterized in that: Inputting the noisy speech signal into the network architecture of the MSAF model, each self-attention module in the architecture performs data processing in parallel; The noisy speech signal is simultaneously input into N self-attention modules, and the outputs of N self-attention modules are obtained at the same time, and then the N outputs are combined with the coefficients Multiply and then sum to get the final enhanced clean speech signal.

8. A multi-self-attention domain system for speech enhancement using the method according to any one of claims 1 to 7, characterized in that: A collection unit, collecting voice information containing noise; An analysis unit constructs a network architecture of the MSAF model; inputs the speech information into N self-attention modules in parallel; Output unit, which combines the output of each self-attention module with a trainable coefficient randomly sampled from a Gaussian distribution Multiply them together to get the final output of enhanced and clean speech.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the multi-self-attention domain method for speech enhancement are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-self-attention domain method for speech enhancement are implemented.

Citation Information

Patent Citations

  • Systems and methods for speech enhancement using attention masking and end to end neural networks

    US20230162758A1