Voice noise reduction method and device, earphone, medium and program product

Abstract

The invention discloses a voice noise reduction method and device, an earphone, a medium and a program product. The method comprises the following steps: acquiring voice sequence data; performing at least one self-attention operation on the voice sequence data; obtaining the output of the self-attention mechanism; noise reduction is carried out according to the output of the self-attention mechanism; the sg attention operation comprises the following steps: carrying out linear processing on the input sequence data by utilizing a preset query-key matrix and a preset value matrix to obtain a query-key vector and a value vector; the query-key matrix corresponds to a product between a preset query matrix and a preset key matrix; obtaining a target attention score according to the transpose of the voice sequence data and the query-key vector; carrying out Softmax operation on the target attention score to obtain an attention weight; and performing matrix multiplication on the attention weight and the value vector to obtain the output of the self-attention operation. In this way, the harsh requirement of the embedded device for high real-time performance in a voice interaction scene can be met.

Description

Technical Field

[0001] This application relates to the field of speech noise reduction technology, and more specifically, to a speech noise reduction method, device, earphone, medium, and program product. Background Technology

[0002] Self-attention networks are widely used in speech signal processing tasks, especially speech denoising, due to their powerful sequence modeling capabilities. They can effectively capture the long-range dependencies of speech signals in the time and frequency domains, thereby more accurately distinguishing noise and speech components and achieving better denoising results than traditional methods.

[0003] However, when deploying complex noise reduction models based on self-attention networks to embedded devices with limited computing, storage, and power consumption, such as TWS earphones, smartwatches, and smart speakers, the self-attention module involves multiple large-scale matrix multiplication operations, making it difficult to meet the high real-time requirements of embedded devices in voice interaction scenarios. Summary of the Invention

[0004] Therefore, the purpose of this application is to provide a speech noise reduction method, device, earphone, medium, and program product to solve the problem that existing self-attention modules are difficult to meet the high real-time requirements of embedded devices in voice interaction scenarios.

[0005] This application provides a speech noise reduction method, including: The speech data to be denoised is processed by convolution to obtain speech sequence data; Perform at least one self-attention operation on the speech sequence data; obtain the output of the self-attention mechanism; The speech data to be denoised is denoised according to the output of the self-attention mechanism; Each self-attention operation includes: The input sequence data is linearized using a preset query-key matrix and value matrix to obtain a query-key vector and a value vector; the product of the query-key matrix and the preset query matrix and the transpose of the preset key matrix corresponds to each other; when performing self-attention operation for the first time, the input sequence data is the speech sequence data; when performing self-attention operation for other times, the input sequence data is the output of the previous self-attention operation. The target attention score is obtained based on the transpose of the speech sequence data and the query-key vector. Perform a Softmax operation on the target attention score to obtain the attention weights; The attention weights are multiplied by the value vector to obtain the output of the self-attention operation.

[0006] In the above implementation, the query-key matrix is ​​mapped to the product of a preset query matrix and the transpose of a preset key matrix. The input sequence data is then linearized using the preset query-key matrix to obtain the query-key vector. This merges key matrix operations and query matrix operations into a single matrix multiplication operation. This reduces the number of matrix multiplication operations, resulting in a faster output of the self-attention operation.

[0007] Based on this, since the output of self-attention operation can be obtained more efficiently, the output of the self-attention mechanism can be used to achieve faster noise reduction of the speech data to be denoised, effectively reducing algorithm latency and meeting the stringent real-time requirements of embedded devices in voice interaction scenarios.

[0008] Optionally, the query-key matrix is ​​equal to the product of a preset query matrix and the transpose of a preset key matrix; Correspondingly, the target attention score is obtained based on the transpose of the speech sequence data and the query-key vector, including: Matrix multiplication is performed on the query-key vector and the transpose of the speech sequence data to obtain the original attention score; The original attention score is scaled using a preset scaling factor to obtain the target attention score.

[0009] In the above implementation, since the preset query-key matrix is ​​equal to the product of the preset query matrix and the transpose of the preset key matrix, using this query-key matrix to perform a linear transformation on the input sequence data to obtain the query-key vector is equivalent to performing linear transformations on the input sequence data using the query matrix and the key matrix respectively to obtain the query vector and the key vector. Simultaneously, obtaining the original attention score by performing matrix multiplication on the transpose of the speech sequence data and the query-key vector is equivalent to obtaining the attention score by multiplying the query vector and the transpose of the key vector.

[0010] Therefore, in the above implementation, the two matrix multiplication operations based on the query vector and the key vector can be simplified into a single matrix multiplication operation, which reduces the computational complexity and latency, while also obtaining an attention score equivalent to the original self-attention mechanism. This allows the high real-time requirements of embedded devices in voice interaction scenarios to be met without sacrificing noise reduction performance.

[0011] Optionally, the query-key matrix is ​​equal to the product of a preset query matrix, the transpose of a preset key matrix, and a preset scaling factor; Correspondingly, the target attention score is obtained based on the transpose of the speech sequence data and the query-key vector, including: The target attention score is obtained by performing matrix multiplication on the query-key vector and the transpose of the speech sequence data.

[0012] In the above implementation, the query-key matrix is ​​equal to the product of a preset query matrix, the transpose of a preset key matrix, and a preset scaling factor. Therefore, using this query-key matrix to perform a linear transformation on the input sequence data to obtain the query-key vector is equivalent to generating the query vector and key vector respectively from the query matrix and key matrix, and scaling their dot product results accordingly. This not only simplifies two matrix multiplication operations based on the query vector and key vector into a single matrix multiplication operation, but also allows this single matrix multiplication operation to include scaling. Thus, feature projection and attention scale normalization can be completed simultaneously in a single linear transformation, significantly reducing the number of operations. This, in turn, meets the stringent real-time requirements of embedded devices in voice interaction scenarios.

[0013] Optionally, denoising the speech data to be denoised based on the output of the self-attention mechanism includes: The output of the self-attention mechanism is deconvolved to obtain the probability of speech presence in the speech data to be denoised. The speech data to be denoised is filtered according to the probability of speech presence to obtain denoised speech data.

[0014] In the above implementation, by performing deconvolution processing on the output of the self-attention mechanism, a high-resolution speech presence probability can be obtained; then, based on the speech presence probability, the speech data to be denoised is filtered to achieve fine local noise suppression and effectively reduce speech distortion caused by "one-size-fits-all" filtering.

[0015] Optionally, the speech data to be denoised is filtered according to the probability of speech presence to obtain denoised speech data, including: Perform a short-time Fourier transform on the speech data to be denoised to obtain the original frequency domain speech data; The original frequency domain speech data is weighted using the speech presence probability to obtain weighted frequency domain speech data. The weighted frequency domain data is subjected to inverse short-time Fourier transform to obtain the denoised speech data.

[0016] Since the human ear's perception of sound is essentially based on frequency domain analysis, in other words, the basilar membrane in the cochlea decomposes the sound signal according to frequency. At the same time, speech and noise are usually more obviously separable in the frequency domain. Therefore, in the above implementation method, by using the speech existence probability to perform time-frequency weighting on the speech data to be denoised after short-time Fourier transform, speech that is more in line with the psychoacoustic characteristics of human hearing can be obtained.

[0017] Optionally, the speech data to be denoised is subjected to convolution processing to obtain speech sequence data, including: Perform a short-time Fourier transform on the speech data to be denoised to obtain the original frequency domain speech data; The original frequency domain speech data is convolved to obtain speech sequence data.

[0018] In the above implementation, by performing convolution processing on the original frequency domain speech data obtained by short-time Fourier transform, the separability of speech and noise at the feature level can be effectively enhanced, thereby achieving separation of the two more efficiently.

[0019] Optionally, the original frequency domain speech data is convolved to obtain speech sequence data, including: Obtain the amplitude spectrum from the original frequency domain speech data; The amplitude spectrum is normalized to obtain a normalized amplitude spectrum; The normalized amplitude spectrum is convolved to obtain speech sequence data.

[0020] Since the intelligibility and perceived quality of speech mainly depend on its amplitude spectrum, while the phase spectrum has a relatively small impact on speech in traditional cognition, the above implementation method extracts the amplitude spectrum from the original frequency domain speech data for convolution processing, which can effectively reduce the model input dimension, reduce computational complexity, and make the convolutional network more focused on learning the discriminative features of speech and noise in energy distribution, thereby improving noise reduction performance and training efficiency.

[0021] Furthermore, factors such as different speakers, recording devices, or volume levels can lead to significant differences in the global energy of the speech amplitude spectrum. If the original amplitude values ​​are directly input, the model is easily affected by scale variations and struggles to generalize. Therefore, by normalizing the amplitude spectrum, all samples can be mapped to a similar numerical range, thereby improving data consistency and facilitating the stable and efficient extraction of local time-frequency features related to noise reduction by subsequent convolutional layers.

[0022] Secondly, embodiments of this application provide a speech noise reduction device, comprising: The convolution module is configured to perform convolution processing on the speech data to be denoised to obtain speech sequence data; The self-attention mechanism module is configured to perform at least one self-attention operation on the speech sequence data and obtain the output of the self-attention mechanism. The parsing module is configured to denoise the speech data to be denoised based on the output of the self-attention mechanism; The self-attention mechanism module includes one or more cascaded self-attention layers; For each self-attention layer, the self-attention layer includes: The first linear layer is configured to linearize the speech sequence data using a preset query-key matrix to obtain a query-key vector; the query-key matrix corresponds to the product of the preset query matrix and the transpose of the preset key matrix. The second linear layer is configured to linearize the speech sequence data using a preset value matrix to obtain a value vector. An attention score calculation unit is configured to obtain a target attention score based on the transpose of the speech sequence data and the query-key vector. The Softmax unit is configured to perform a Softmax operation on the target attention score to obtain attention weights; The output unit is configured to perform matrix multiplication on the attention weights and the value vector to obtain the output of the self-attention mechanism.

[0023] Thirdly, embodiments of this application provide an earphone, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-described voice noise reduction method.

[0024] Fourthly, embodiments of this application provide a computer storage medium storing computer-executable instructions. When the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions cause the processor to implement the aforementioned speech noise reduction method.

[0025] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the above-described speech noise reduction method. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the self-attention operation process of the existing self-attention mechanism; Figure 2 A schematic flowchart of a speech noise reduction method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a speech noise reduction device provided in an embodiment of this application; Figure 4 This application provides a schematic diagram of the self-attention operation process of a self-attention layer. Figure 5 This is a schematic diagram of the structure of an earphone provided in an embodiment of this application. Detailed Implementation

[0027] The self-attention operation process of existing self-attention mechanisms can be as follows: Figure 1 As shown: The input sequence data X is input into the linear layer LinearQ, which then uses a preset query matrix to linearize the input sequence data X, obtaining the query vector R1. For example, , Wq represents the query matrix, which is also the weight of the linear layer (Linearq). . , where m represents the sequence length and n represents the input dimension.

[0028] Input sequence data X into the linear layer Lineark, which then uses a preset key matrix to linearize the input sequence data X, obtaining the key vector R2. , Wk represents the bond matrix, which is also the weight of the linear layer Lineark. .

[0029] Input sequence data X into the linear layer Linearv, and Linearv uses a preset value matrix Wv to linearize the input sequence data X to obtain the value vector R3. , Wv is the representation matrix, which is also the weight of the linear layer Linearv. .

[0030] The original attention score calculation unit performs matrix multiplication on the transpose R2T of the query vector R1 and the key vector R2 to obtain the original attention score R4. Correspondingly, .

[0031] The scaling unit utilizes a preset scaling factor. Scaling the original attention score R4 yields the target attention score R5. Correspondingly, .

[0032] The Softmax unit performs a Softmax operation on the target attention score R5 to obtain the attention weight R6.

[0033] In other words, the attention weight R6 satisfies the formula: R6 = Softmax(R5) = Softmax( =Softmax( )= .

[0034] The output unit performs matrix multiplication on the attention weight R6 and the value vector R3 to obtain the output Y of the self-attention operation.

[0035] It is evident that the existing self-attention computation process involves a total of 5 large matrix multiplications, which are time-consuming under limited computing power at the edge, becoming the speed bottleneck of the entire noise reduction network and making it difficult to meet the requirements of low-latency real-time processing.

[0036] To address the aforementioned problems, this application provides a speech noise reduction method. See also... Figure 2 As shown, Figure 2 This is a flowchart illustrating the speech noise reduction method provided in the embodiments of this application, including: Step S201: Perform convolution processing on the speech data to be denoised to obtain speech sequence data.

[0037] In one optional implementation of this application, the speech data to be denoised can be directly convolved to obtain speech sequence data.

[0038] In another optional implementation of this application, a short-time Fourier transform can be performed on the speech data to be denoised to obtain the original frequency domain speech data; convolution processing can be performed on the original frequency domain speech data to obtain speech sequence data.

[0039] In another optional implementation of this application, a short-time Fourier transform can be performed on the speech data to be denoised to obtain the original frequency domain speech data; an amplitude spectrum can be obtained from the original frequency domain speech data; the amplitude spectrum can be normalized to obtain a normalized amplitude spectrum; and the normalized amplitude spectrum can be convolved to obtain speech sequence data.

[0040] In some embodiments, the amplitude spectrum can be obtained from the original frequency domain speech data and quantized into fixed-point data, such as int16 type, so that subsequent convolutional networks can process it efficiently to obtain speech sequence data.

[0041] Step S202: Perform at least one self-attention operation on the speech sequence data; obtain the output of the self-attention mechanism.

[0042] The self-attention operation includes: linearizing the input sequence data using a preset query-key matrix and value matrix to obtain a query-key vector and a value vector; the product of the query-key matrix with the transpose of the preset query matrix and the preset key matrix corresponds to the input sequence data; in the case of the first self-attention operation, the input sequence data is speech sequence data; in the case of subsequent self-attention operations, the input sequence data is the output of the previous self-attention operation; obtaining the target attention score based on the transpose of the speech sequence data and the query-key vector; performing a softmax operation on the target attention score to obtain the attention weights; and performing matrix multiplication between the attention weights and the value vector to obtain the output of the self-attention operation.

[0043] In one optional implementation of this application, the query-key matrix can be equal to the product of a preset query matrix and the transpose of a preset key matrix. Correspondingly, matrix multiplication can be performed on the query-key vector and the transpose of the speech sequence data to obtain the original attention score; and the original attention score can be scaled using a preset scaling factor to obtain the target attention score.

[0044] In another optional implementation of this application, the query-key matrix can be equal to the product of a preset query matrix, the transpose of a preset key matrix, and a preset scaling factor. Correspondingly, matrix multiplication can be performed on the query-key vector and the transpose of the speech sequence data to obtain the target attention score.

[0045] Step S203: Denoise the speech data to be denoised based on the output of the self-attention mechanism.

[0046] In one optional implementation of this application, the output of the self-attention mechanism can be deconvolved to obtain the probability of speech presence in the speech data to be denoised; the speech data to be denoised can then be filtered according to the probability of speech presence to obtain the denoised speech data.

[0047] In this embodiment, the probability of speech presence can be directly used to filter the speech data to be denoised, thereby obtaining the denoised speech data.

[0048] In this embodiment, a short-time Fourier transform can be performed on the speech data to be denoised to obtain the original frequency domain speech data; the original frequency domain speech data can be weighted using the speech presence probability to obtain weighted frequency domain speech data; and an inverse short-time Fourier transform can be performed on the weighted frequency domain data to obtain the denoised speech data.

[0049] The speech denoising method provided in this application obtains a query-key vector by mapping the product of the query-key matrix to a preset query matrix and the transpose of a preset key matrix, and by linearizing the input sequence data using the preset query-key matrix. This integrates key matrix operations and query matrix operations into a single matrix multiplication operation. This reduces matrix multiplication operations, resulting in faster acquisition of the output of the self-attention operation.

[0050] Based on this, since the output of self-attention operation can be obtained more efficiently, the output of the self-attention mechanism can be used to achieve faster noise reduction of the speech data to be denoised, effectively reducing algorithm latency and meeting the stringent real-time requirements of embedded devices in voice interaction scenarios.

[0051] Example 2: Based on the same inventive concept, this application provides a structural schematic diagram of a speech noise reduction device 300, combined with... Figure 3 As shown, the speech noise reduction device 300 may include a convolution module 301, a self-attention mechanism module 302, and a parsing module 303.

[0052] Optionally, the convolution module 301 may include two convolutional layers, the self-attention mechanism module 302 may include four self-attention layers, and the parsing module may include two deconvolutional layers and one sigmoid layer.

[0053] Convolution module 301 is configured to perform convolution processing on the speech data to be denoised, obtaining speech sequence data. Self-attention mechanism module 302 is configured to perform at least one self-attention operation on the speech sequence data, obtaining the output of the self-attention mechanism. Parsing module 303 is configured to denoise the speech data to be denoised based on the output of the self-attention mechanism. The self-attention mechanism module includes one or more cascaded self-attention layers.

[0054] Optionally, combined Figure 4 As shown, each self-attention layer may include a first linear layer 331, a second linear layer 332, an attention score calculation unit 333, a softmax unit 334, and an output unit 335.

[0055] The first linear layer 331 is configured to linearize the speech sequence data using a preset query-key matrix to obtain a query-key vector; the product of the query-key matrix and the transpose of the preset query matrix and key matrix corresponds to this. The second linear layer 332 is configured to linearize the speech sequence data using a preset value matrix to obtain a value vector. The attention score calculation unit 333 is configured to obtain a target attention score based on the transpose of the speech sequence data and the query-key vector. The Softmax unit 334 is configured to perform a Softmax operation on the target attention score to obtain attention weights. The output unit 335 is configured to perform a matrix multiplication operation between the attention weights and the value vector to obtain the output of the self-attention mechanism.

[0056] In this embodiment, the query-key matrix can be equal to the product of a preset query matrix and the transpose of a preset key matrix. Correspondingly, the attention score calculation unit can include a raw attention score calculation unit and a scaling unit. Specifically, the raw attention score calculation unit can perform matrix multiplication on the query-key vector and the transpose of the speech sequence data to obtain the raw attention score. The scaling unit can specifically scale the raw attention score using a preset scaling factor to obtain the target attention score.

[0057] In some embodiments, the query-key matrix Wqk can satisfy the formula: Where Wq represents the preset query matrix, This represents the transpose of the preset key matrix Wk. Correspondingly, the query-key matrix Wqk is the weight of the first linear layer 331.

[0058] Optionally, the query-key matrix can be equal to the product of a preset query matrix, the transpose of a preset key matrix, and a preset scaling factor. Correspondingly, the attention score calculation unit can be specifically configured to perform matrix multiplication on the transpose of the speech sequence data and the query-key vector to obtain the target attention score.

[0059] In some embodiments, the query-key matrix Wqk can satisfy the formula: ,in, Characterizes the scaling factor.

[0060] Since the self-attention network has already been trained during the deployment phase, the network weights of the first linear layer are fixed. Therefore, the preprocessing steps can be calculated in advance and will not consume time in the inference process.

[0061] In this embodiment, the operation mechanism of the self-attention layer of this application embodiment is further illustrated by taking the example of the query-key matrix being equal to the product of a preset query matrix, the transpose of a preset key matrix, and a preset scaling factor.

[0062] The input sequence data X is input into the first linear layer 331, which then uses a preset query-key matrix to linearize the speech sequence data, obtaining the query-key vector R1'. Correspondingly, Simultaneously, the input sequence data X can be input into the second linear layer 332, allowing the second linear layer 332 to linearize the speech sequence data using a preset value matrix, obtaining the value vector R2'. Wherein, .

[0063] By performing matrix multiplication on R1' and the transpose XT of the input sequence data X, we can obtain .

[0064] Performing a softmax operation on R3' yields the attention weights R4', where R4' = Softmax(R3') = Softmax( =Softmax( =Softmax( ).

[0065] As can be seen, the attention weight R4' calculated using the method provided in this application is the same as the attention weight R6 calculated using existing self-attention mechanisms. Therefore, the speech denoising method provided in this application can meet the stringent real-time requirements of embedded devices in voice interaction scenarios without sacrificing denoising performance. In addition, each self-attention layer can reduce the storage of the weights of a linear layer, thereby reducing the static storage overhead and runtime memory usage of the model.

[0066] Optionally, the parsing module 303 can be specifically configured to perform deconvolution processing on the output of the self-attention mechanism to obtain the probability of speech presence in the speech data to be denoised. Based on the speech presence probability, the speech data to be denoised is then filtered to obtain the denoised speech data.

[0067] Optionally, the parsing module 303 can be specifically configured to perform a short-time Fourier transform on the speech data to be denoised to obtain the original frequency domain speech data; weight the original frequency domain speech data using the speech existence probability to obtain weighted frequency domain speech data; and perform an inverse short-time Fourier transform on the weighted frequency domain data to obtain the denoised speech data.

[0068] Optionally, the convolution module 301 can be specifically configured to perform a short-time Fourier transform on the speech data to be denoised to obtain the original frequency domain speech data; and to perform convolution processing on the original frequency domain speech data to obtain speech sequence data.

[0069] Optionally, the convolution module 301 can be specifically configured to obtain the amplitude spectrum from the original frequency domain speech data; normalize the amplitude spectrum to obtain a normalized amplitude spectrum; and perform convolution processing on the normalized amplitude spectrum to obtain speech sequence data.

[0070] The voice noise reduction device 300 may include at least one software function module that can be stored in a memory or embedded in the operating system of the device 300 in the form of software or firmware.

[0071] It should be understood that, for the sake of brevity, some of the content described in Embodiment 1 will not be repeated in this embodiment.

[0072] Example 3: Based on the same inventive concept, this embodiment provides an earphone, see [link to relevant documentation]. Figure 5 As shown, it includes a processor 501 and a memory 502. Wherein: The processor 501 is used to execute one or more programs stored in the memory 502 to implement the above-described speech noise reduction method.

[0073] It is understandable that processor 501 can be a processor core or processor chip, or other circuitry capable of program configuration and execution. Memory 502 can be RAM (Random Access Memory), ROM (Read-Only Memory), flash memory, etc., but this is not a limitation.

[0074] It's understandable. Figure 5 The structure shown is for illustrative purposes only; the headphones may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 Different configurations are shown. For example, it may also have an internal communication bus for communication between the processor 501 and the memory 502; or it may have an external communication interface, such as a USB (Universal Serial Bus) interface, a CAN (Controller Area Network) bus interface, etc.; or it may have an information display component such as a display screen, but this is not a limitation.

[0075] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium, such as a floppy disk, optical disk, hard disk, flash memory, USB flash drive, SD (Secure Digital Memory Card), MMC (Multimedia Card), etc., in which one or more programs implementing the above steps are stored. These one or more programs can be executed by one or more processors to implement the above-described speech noise reduction method. Further details will not be elaborated here.

[0076] Based on the same inventive concept, this embodiment also provides a computer program product, which includes a computer program that, when executed by a processor, implements the speech noise reduction method as described above.

[0077] Computer program products can be installation packages or program packages.

[0078] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0079] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0081] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0082] In this article, "multiple" refers to two or more.

[0083] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A voice noise reduction method, characterized by, The method comprises the following steps: performing convolution processing on the to-be-noise-reduced speech data to obtain speech sequence data; performing at least one self-attention operation on the speech sequence data; obtaining an output of the self-attention mechanism; performing noise reduction on the to-be-noise-reduced speech data according to the output of the self-attention mechanism; each self-attention operation comprises the following steps: performing linearization processing on input sequence data by using a preset query-key matrix and a value matrix to obtain a query-key vector and a value vector; the query-key matrix corresponds to the product of a preset query matrix and the transpose of a preset key matrix; in the case of performing the self-attention operation for the first time, the input sequence data is the speech sequence data; in the case of performing the self-attention operation for the second time, the input sequence data is the output of the previous self-attention operation; obtaining a target attention score according to the transpose of the speech sequence data and the query-key vector; performing Softmax operation on the target attention score to obtain an attention weight; performing matrix multiplication operation on the attention weight and the value vector to obtain the output of the self-attention operation.

2. The method of claim 1, wherein, The query-key matrix is equal to the product of the preset query matrix, the transpose of the preset key matrix, and a preset scaling factor. Correspondingly, the step of obtaining a target attention score according to the transpose of the speech sequence data and the query-key vector comprises the following steps: performing matrix multiplication operation on the query-key vector and the transpose of the speech sequence data to obtain an original attention score; scaling the original attention score by using a preset scaling factor to obtain the target attention score.

3. The method of claim 1, wherein, The query-key matrix is equal to the product of the preset query matrix, the transpose of the preset key matrix, and a preset scaling factor. Correspondingly, the step of obtaining a target attention score according to the transpose of the speech sequence data and the query-key vector comprises the following steps: performing matrix multiplication operation on the query-key vector and the transpose of the speech sequence data to obtain the target attention score.

4. The method of claim 1, wherein, The step of performing noise reduction on the to-be-noise-reduced speech data according to the output of the self-attention mechanism comprises the following steps: performing deconvolution processing on the output of the self-attention mechanism to obtain a speech presence probability in the to-be-noise-reduced speech data; performing filtering on the to-be-noise-reduced speech data according to the speech presence probability to obtain noise-reduced speech data.

5. The method of claim 4, wherein, The step of performing filtering on the to-be-noise-reduced speech data according to the speech presence probability to obtain noise-reduced speech data comprises the following steps: performing short-time Fourier transform on the to-be-noise-reduced speech data to obtain original frequency domain speech data; performing weighting on the original frequency domain speech data by using the speech presence probability to obtain weighted frequency domain speech data; performing inverse short-time Fourier transform on the weighted frequency domain data to obtain noise-reduced speech data.

6. The method according to any one of claims 1 to 5, characterized in that, The step of performing convolution processing on the to-be-noise-reduced speech data to obtain speech sequence data comprises the following steps: performing short-time Fourier transform on the to-be-noise-reduced speech data to obtain original frequency domain speech data; performing convolution processing on the original frequency domain speech data to obtain speech sequence data.

7. The method of claim 6, wherein, The step of performing convolution processing on the original frequency domain speech data to obtain speech sequence data comprises the following steps: obtaining an amplitude spectrum from the original frequency domain speech data; The amplitude spectrum is normalized to obtain a normalized amplitude spectrum; The normalized amplitude spectrum is convolved to obtain speech sequence data.

8. A voice noise reduction device, characterized by, Comprise: A convolution module configured to convolve the noise reduction speech data to obtain speech sequence data; A self-attention mechanism module configured to perform at least one self-attention operation on the speech sequence data; and obtain an output of the self-attention mechanism; A parsing module configured to reduce noise in the noise reduction speech data according to the output of the self-attention mechanism; The self-attention mechanism module comprises one or more cascaded self-attention layers; For each self-attention layer, the self-attention layer comprises: A first linear layer configured to linearize the speech sequence data using a preset query-key matrix to obtain a query-key vector; the query-key matrix corresponds to the product of a preset query matrix and the transpose of a preset key matrix; A second linear layer configured to linearize the speech sequence data using a preset value matrix to obtain a value vector; An attention score calculation unit configured to obtain a target attention score according to the transpose of the speech sequence data and the query-key vector; A Softmax unit configured to perform a Softmax operation on the target attention score to obtain an attention weight; An output unit configured to perform a matrix multiplication operation on the attention weight and the value vector to obtain an output of the self-attention mechanism.

9. An earphone, characterized by A processor and a memory, the memory storing computer executable instructions executable by the processor, the processor executing the computer executable instructions to implement the speech noise reduction method of any one of claims 1-7.

10. A computer storage medium, characterized in that, The storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by the processor, the computer executable instructions enable the processor to implement the speech noise reduction method of any one of claims 1-7.

11. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the speech noise reduction method of any one of claims 1-7.

Images