A single-channel speech noise reduction method and apparatus

Abstract

The application discloses a single-channel speech noise reduction method and device, and the method comprises the following steps: extracting signal features of a speech time domain signal and inputting the signal features to an encoding model; dividing the signal features output by the encoding model into sub-frequency bands respectively and inputting the signal features to a frequency GRU model, and inputting the signal features divided by the sub-frequency bands and the signal features output by the frequency GRU model to a sub-frequency band processing model after splicing; the sub-frequency band processing model comprises a GRU layer, a linear layer and a one-dimensional conventional convolution layer in sequence; inputting the signal features output by the sub-frequency band processing model to a time GRU model and then to a decoding model; and outputting a speech time domain signal after optimizing and calculating the signal features output by the decoding model. The application utilizes an encoding-decoding framework and frequency point correlation, combines a recurrent neural network model, realizes speech noise reduction performance optimization, can be operated in real time on a local device, and achieves a good speech noise reduction effect.

Description

Technical Field

[0001] This specification relates to the field of audio processing technology, and in particular to a single-channel speech noise reduction method, apparatus, electronic device, and storage medium. Background Technology

[0002] In remote audio and video communication, various types of noise are prevalent in the environment, severely impacting voice communication quality. Therefore, noise suppression while maintaining voice quality is crucial for remote conferencing systems. Noise suppression techniques can reduce steady-state and non-steady-state noise in speech signals, thereby improving the signal-to-noise ratio, speech clarity, and reducing listening fatigue. Traditional signal processing-based denoising algorithms are unsuitable for handling the prevalent non-steady-state noise in real-world applications. Currently available deep neural network-based noise suppression methods suffer from limitations such as unsuitable model structures for real-time inference and excessively large model sizes that prevent low-power operation on local devices. Therefore, achieving a speech denoising technique with low performance consumption, small model size, and real-time operation on local devices based on deep neural networks is a pressing technical challenge. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to address the above-mentioned problems by providing a single-channel speech noise reduction method, apparatus, electronic device, and storage medium.

[0004] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0005] Firstly, a single-channel speech noise reduction method is proposed, including:

[0006] For the collected speech time domain signal Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal ;

[0007] The signal features are input into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer.

[0008] The signal features output by the coding model are divided into sub-bands and input into the frequency GRU model. The signal features divided into sub-bands are then concatenated with the signal features output by the frequency GRU model and input into the sub-band processing model. The sub-band processing model includes, in sequence, one GRU layer, one linear layer, and one one-dimensional conventional convolutional layer.

[0009] The signal features output by the sub-band processing model are processed by a time GRU model and then input into the decoding model, which includes three sets of one-dimensional transposed convolutional layers.

[0010] After optimizing the signal features output by the decoding model, the speech time-domain signal is output. .

[0011] Furthermore, the number of Fourier transform points is 512, and based on the frequency domain signal... amplitude The signal features are calculated and output, including 257 frequency points; and / or,

[0012] The one-dimensional conventional convolutional layer, the first depthwise separable convolutional layer, and the second depthwise separable convolutional layer have convolutional kernels of 5, 3, and 5, respectively, and corresponding strides of 2, 1, and 2, respectively; and / or,

[0013] The three groups of one-dimensional transposed convolutional layers have convolutional kernels of 5, 3, and 5, and corresponding strides of 2, 1, and 2, respectively. The output of each group of one-dimensional regular convolutional layers is connected to a batch regularization layer and a ReLU layer; and / or,

[0014] The encoding model and the decoding model exchange information using skip connections.

[0015] Furthermore, the encoding model further includes, in sequence:

[0016] The regularization layer and the ReLU layer are connected to the output of the one-dimensional regular convolutional layer and the input of the first depth-separable convolutional layer; and / or,

[0017] Both the first depthwise separable convolutional layer and the second depthwise separable convolutional layer include a one-dimensional Depthwise convolutional layer and a one-dimensional Pointwise convolutional layer, as well as a batch regularization layer and a ReLU layer connected to the output of the one-dimensional Depthwise convolutional layer and connected to the output of the one-dimensional Pointwise convolutional layer.

[0018] Furthermore, the process of dividing the signal features output by the coding model into sub-frequency bands includes:

[0019] Select consecutive frequencies before and after the current frequency point. Each frequency point constitutes a length of [number]. Sub-bands;

[0020] If the number of frequency points before and after the current frequency point is insufficient At any given time, a circular cyclic method is used to select frequency points to form sub-bands;

[0021] The sub-bands corresponding to all frequency points are used to form a frequency domain sub-band frequency point matrix.

[0022] Furthermore, both the frequency GRU model and the time GRU model include a set of GRU layers and a set of one-dimensional conventional convolutional layers connected to the output of the GRU layers.

[0023] Furthermore, the output of the decoding model is optimized and calculated to output a speech time-domain signal. The process includes:

[0024] Based on the signal characteristics output by the decoding model, a corresponding complex ideal ratio mask is obtained;

[0025] The complex ideal ratio mask real part and the virtual part Acting on the frequency domain signal respectively real part and the virtual part The optimized pure speech domain signal estimate was calculated. real part = - and the virtual part = + ;

[0026] Estimated value of the pure speech audio domain signal Perform inverse Fourier transform to output time-domain signal .

[0027] Furthermore, it also includes training the neural network during the speech denoising process, using a loss function of... ,in, , ; and / or,

[0028] An Adam optimizer with a learning rate of 0.001 is used, which is adjusted based on the preset number of optimization rounds and / or the optimization verification results.

[0029] Secondly, a single-channel speech noise reduction device is proposed, comprising:

[0030] The first module is used to process the acquired speech time-domain signal. Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal ;

[0031] The second module is used to input the signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer.

[0032] The third module is used to divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model, and then concatenate the signal features divided into sub-bands with the signal features output by the frequency GRU model and input them into the sub-band processing model. The sub-band processing model includes, in sequence, one GRU layer, one linear layer and one one-dimensional conventional convolutional layer.

[0033] The fourth module is used to input the signal features output by the sub-band processing model into the decoding model after passing through the time GRU model. The decoding model includes three sets of one-dimensional transposed convolutional layers.

[0034] The fifth module is used to optimize and calculate the signal features output by the decoding model and then output the speech time-domain signal. .

[0035] Thirdly, a computer device is proposed, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the single-channel speech noise reduction method described in the first aspect is performed.

[0036] Fourthly, a computer-readable storage medium is proposed, on which a computer program is stored, which, when executed by a processor, performs the single-channel speech noise reduction method described in the first aspect.

[0037] This instruction manual can achieve at least the following technical effects:

[0038] The present invention is based on a U-shaped network structure of an encoder-decoder framework, which makes full use of the correlation between frequency points, the advantages of recurrent neural networks in time-series processing and convolutional neural networks in feature extraction. It also combines sub-band processing technology to achieve speech noise reduction effect with small model size, few parameters, low performance consumption and real-time operation on local devices. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is one of the schematic diagrams of a single-channel speech noise reduction method provided in the embodiments of this specification.

[0041] Figure 2 This is the second schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0042] Figure 3 This is the third schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0043] Figure 4 This is the fourth schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0044] Figure 5 This is the fifth schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0045] Figure 6 This is the sixth schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0046] Figure 7 This is the seventh schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0047] Figure 8 This is the eighth schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0048] Figure 9 This is diagram nine of the single-channel speech noise reduction methods provided in the embodiments of this specification.

[0049] Figure 10 This is the tenth schematic diagram of the single-channel speech noise reduction method provided in the embodiments of this specification.

[0050] Figure 11 This is one of the schematic diagrams of a single-channel speech noise reduction device provided in the embodiments of this specification.

[0051] Figure 12 This is a schematic diagram of the structure of an electronic device provided as an embodiment of this specification. Detailed Implementation

[0052] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0053] The following section provides a detailed description of a single-channel speech noise reduction scheme described in this manual, using specific examples.

[0054] Example 1

[0055] The purpose of this invention is to overcome the shortcomings and deficiencies of existing deep neural network-based speech denoising methods, such as complexity, large model size, and lack of consideration for frequency correlation, and to provide a low-computational-consumption, real-time, and low-parameter speech denoising scheme. In remote audio and video communication, various types of noise are prevalent in the environment, severely impacting speech communication quality. Suppressing noise while ensuring unaffected speech quality is crucial for remote conferencing systems. Traditional signal processing denoising algorithms are typically based on the assumption of steady-state noise, making them unsuitable for handling non-steady-state noise. Currently disclosed deep neural network-based denoising algorithms generally suffer from large model size and are unsuitable for local real-time processing under low computational resource conditions. Therefore, achieving a low-computational-consumption, real-time, low-parameter speech denoising algorithm that performs well in complex acoustic environments is of great significance. Currently, speech denoising algorithms mainly include traditional signal processing-based algorithms and the increasingly developed deep neural network-based methods. Traditional signal processing algorithms suffer from difficulties in parameter tuning and are unsuitable for handling non-steady-state noise. Deep neural network-based speech denoising algorithms have recently experienced rapid development and achieved better results than traditional signal processing. Deep neural network-based speech denoising methods have been applied to speech enhancement tasks and have achieved good processing results. However, due to the complexity and large scale of the models, they are still difficult to apply to remote audio and video conferencing systems with limited computing resources.

[0056] In real-time audio and video communication systems, the signal collected by the microphone Including pure voice signals and possible environmental noise , can be represented as The goal of speech denoising algorithms is to obtain the purest speech signal possible. Estimate Therefore, the technical approach of this invention is to fully utilize the advantages of recurrent neural networks in temporal processing and convolutional neural networks in feature extraction within an encoding-decoding framework, while also incorporating sub-band processing technology to achieve better speech noise reduction results.

[0057] Reference Figure 1 The diagram shown illustrates the principle of a single-channel speech denoising scheme according to an embodiment of the present invention. The single-channel speech denoising method disclosed in this invention is implemented within an encoder-decoder framework. First, the speech time-domain signal is frequency-domain converted and signal features are extracted. Second, the features are sequentially processed through five parts: an encoder framework, a frequency GRU model, a sub-band processing model, a time GRU model, and a decoder framework. Finally, the output of the decoder framework is converted back into a speech time-domain signal. This method fully combines the advantages of recurrent neural networks and convolutional neural networks, utilizing the correlation between frequency points in the spectral features, and also incorporates sub-band processing technology. It should be noted that the time-domain signals in the embodiments of this specification are represented by lowercase letters, and the corresponding frequency-domain signals are represented by uppercase letters. (Refer to...) Figure 2 The diagram shown is a schematic representation of a single-channel speech noise reduction method according to an embodiment of the present invention. The method includes:

[0058] S1: For the acquired speech time-domain signal Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal .

[0059] Optionally, the number of Fourier transform points is 512 and based on the frequency domain signal. The amplitude X is used to calculate and output the signal features, which include 257 frequency points. Specifically, for deep learning models, appropriate input features are crucial for achieving good noise reduction results. In this invention, amplitude spectrum features are used. The amplitude spectrum of the signal is obtained by performing a Fourier transform on the signal collected by the microphone. The Fourier transform points used in this scheme are 512, which can be expressed as the formula... That is, the input features of the neural network model are obtained from the amplitude spectrum. Considering the conjugate symmetry after Fourier transform, the dimension of the input features is 257.

[0060] S2: Input the signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer.

[0061] Optionally, the convolution kernels of the one-dimensional conventional convolutional layer, the first depth-separable convolutional layer, and the second depth-separable convolutional layer are 5, 3, and 5, respectively, and the corresponding strides are 2, 1, and 2, respectively.

[0062] Optionally, the encoding model further includes, in sequence, the batch regularization layer and the ReLU layer connected to the output of the one-dimensional regular convolutional layer and the input of the one-dimensional depthwise convolutional layer.

[0063] Optionally, both the first depthwise separable convolutional layer and the second depthwise separable convolutional layer include a one-dimensional Depthwise convolutional layer and a one-dimensional Pointwise convolutional layer, as well as a batch regularization layer and a ReLU layer connected to the output of the one-dimensional Depthwise convolutional layer and connected to the output of the one-dimensional Pointwise convolutional layer.

[0064] Specifically, such as Figure 3 As shown, this is a schematic diagram of the structure of the encoding model in one implementation of this embodiment:

[0065] The first group consists of a regular one-dimensional convolutional layer and a batch regularization layer and a ReLU layer connected to the output of this regular one-dimensional convolutional layer. The regular one-dimensional convolutional layer is represented as follows: Where 1 represents the number of input channels, 32 represents the number of output channels, 5 represents the number of convolution kernels, and 2 represents the stride;

[0066] The second group consists of the first depthwise separable convolutional layer. The first depthwise separable convolutional layer is represented as follows: Where 32 represents the number of input channels, 64 represents the number of output channels, 3 represents the number of convolution kernels, and 1 represents the stride. For example... Figure 4 The diagram shows the structure of the first depthwise separable convolutional layer, which includes a one-dimensional depthwise convolutional layer. and one-dimensional pointwise convolutional layers And, the batch regularization layer and the ReLU layer connected to the output of the one-dimensional Depthwise convolutional layer and connected to the output of the one-dimensional Pointwise convolutional layer;

[0067] The third group consists of the second depthwise separable convolutional layer. The second depthwise separable convolutional layer is represented as follows: Where 64 represents the number of input channels, 64 represents the number of output channels, 5 represents the number of convolution kernels, and 2 represents the stride; for example Figure 5 The diagram shows the structure of the second depthwise separable convolutional layer, which includes a one-dimensional depthwise convolutional layer. and one-dimensional pointwise convolutional layers , and the batch regularization layer and the ReLU layer connected to the output of the one-dimensional Depthwise convolutional layer and the output of the one-dimensional Pointwise convolutional layer.

[0068] S3: Divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model. Then, concatenate the signal features divided into sub-bands with the signal features output by the frequency GRU model and input them into the sub-band processing model. The sub-band processing model includes one GRU layer, one linear layer and one one-dimensional conventional convolutional layer in sequence.

[0069] Optionally, such as Figure 6 As shown, the process of dividing the signal features output by the coding model into sub-frequency bands includes:

[0070] S311: Select consecutive frequencies before and after the current frequency point. Each frequency point constitutes a length of [number]. Sub-bands;

[0071] S312: If the number of frequency points before and after the current frequency point is insufficient At any given time, a circular cyclic method is used to select frequency points to form sub-bands;

[0072] S313: The sub-bands corresponding to all frequency points are used to form a frequency domain sub-band frequency point matrix.

[0073] Specifically, after processing by the coding model, 64 frequency points are obtained. If the value is 3, for the first frequency point, the 62nd, 63rd and 64th frequency points are selected together with the 3 frequency points after the first frequency point to form the first sub-band. The division of other sub-bands is carried out according to this principle.

[0074] Optionally, the frequency-based GRU model includes a set of GRU layers and a set of one-dimensional conventional convolutional layers connected to the output of the GRU layers. Specifically, as shown... Figure 7 The image shows one implementation of the frequency GRU model in this embodiment, where the GRU layer is represented as... That is, the gated recurrent neural network has 64 input channels and 64 hidden channels; a conventional one-dimensional convolutional layer is represented as... , where 64 represents the number of input channels, 64 represents the number of output channels, 1 represents the number of convolution kernels, and 1 represents the stride.

[0075] Furthermore, such as Figure 8 The diagram illustrates one implementation of the sub-band processing model in this embodiment, where the GRU layer is represented as follows: That is, the gated recurrent neural network has 64 input channels and 64 hidden channels; the linear layer is represented as... That is, the linear layer has 512 input channels and 64 output channels; a conventional one-dimensional convolutional layer is represented as follows: , where 64 represents the number of input channels, 64 represents the number of output channels, 1 represents the number of convolution kernels, and 1 represents the stride.

[0076] S4: The signal features output by the sub-band processing model are processed by a time GRU model and then input into the decoding model. The decoding model includes three sets of one-dimensional transposed convolutional layers.

[0077] Optionally, the temporal GRU model includes a set of GRU layers and a set of one-dimensional regular convolutional layers connected to the output of the GRU layers. Specifically, as shown... Figure 9 The diagram illustrates one implementation of the time-based GRU model in this embodiment, where the GRU layer is represented as... That is, the gated recurrent neural network has 64 input channels and 64 hidden channels; a conventional one-dimensional convolutional layer is represented as... , where 64 represents the number of input channels, 64 represents the number of output channels, 1 represents the number of convolution kernels, and 1 represents the stride.

[0078] Optionally, the three groups of one-dimensional transposed convolutional layers are arranged in order from input to output, with corresponding convolutional kernels of 5, 3, and 5, and corresponding strides of 2, 1, and 2, respectively. The output of each group of one-dimensional regular convolutional layers is connected to a batch regularization layer and a ReLU layer, which will not be elaborated further here. Optionally, skip connections are used for information exchange between the encoding model and the decoding model.

[0079] S5: After optimizing the signal features output by the decoding model, output the speech time-domain signal. .

[0080] Optionally, such as Figure 10 As shown, the output of the decoding model is optimized and then the speech time-domain signal is output. The process includes:

[0081] S511: Obtain the corresponding complex ideal ratio mask based on the signal characteristics output by the decoding model;

[0082] S512: Apply the complex ideal ratio mask real part and the virtual part Acting on the frequency domain signal respectively real part and the virtual part The optimized pure speech domain signal estimate was calculated. real part = - and the virtual part = + ;

[0083] S513: Estimated value of the optimized pure speech domain signal Perform inverse Fourier transform to output time-domain signal .

[0084] Optionally, training the neural network during the speech denoising process further includes: using a loss function of... ,in, , ; and / or,

[0085] The optimizer used is the Adam optimizer with a learning rate of 0.001, which is adjusted based on the preset number of optimization rounds and / or the optimization verification results. Specifically, if the optimization verification results do not improve after four consecutive optimization rounds, the learning rate can be reduced to half of the original learning rate, and then one or more rounds of optimization can be performed again.

[0086] Example 2

[0087] Figure 11 This is a schematic diagram of a single-channel speech noise reduction device 1100 provided as an embodiment of this specification. Please refer to... Figure 11 In one embodiment, a single-channel speech noise reduction device 1100 includes:

[0088] The first module 1101 is used to process the acquired speech time-domain signal. Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal .

[0089] The second module 1102 is used to input the signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer.

[0090] The third module 1103 is used to divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model, and then concatenate the signal features divided into sub-bands with the signal features output by the frequency GRU model and input them into the sub-band processing model. The sub-band processing model includes, in sequence, one group of GRU layers, one group of linear layers and one group of one-dimensional conventional convolutional layers.

[0091] The fourth module 1104 is used to input the signal features output by the sub-band processing model into the decoding model after passing through the time GRU model. The decoding model includes three sets of one-dimensional transposed convolutional layers.

[0092] The fifth module 1105 is used to optimize and calculate the signal features output by the decoding model and then output the speech time-domain signal. .

[0093] It should be understood that the single-channel speech noise reduction device according to the embodiments of this specification can also perform... Figures 1 to 10 A method for implementing a single-channel speech noise reduction device (or equipment), and for realizing the single-channel speech noise reduction device (or equipment) in... Figures 1 to 10 The functionality of the example shown will not be elaborated upon here.

[0094] Example 3

[0095] Figure 12 This is a schematic diagram of the structure of an electronic device according to one embodiment of this specification. Please refer to it. Figure 12 At the hardware level, the electronic device includes a processor, and optionally also an internal bus, network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0096] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0097] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0098] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a shared resource access control mechanism at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0099] S1: For the acquired speech time-domain signal Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal ;

[0100] S2: Input the signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer;

[0101] S3: Divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model. Then, concatenate the signal features divided into sub-bands with the signal features output by the frequency GRU model and input them into the sub-band processing model. The sub-band processing model includes one GRU layer, one linear layer and one one-dimensional conventional convolutional layer in sequence.

[0102] S4: The signal features output by the sub-band processing model are processed by a time GRU model and then input into the decoding model. The decoding model includes three sets of one-dimensional transposed convolutional layers.

[0103] S5: After optimizing the signal features output by the decoding model, output the speech time-domain signal. .

[0104] The above is as described in this instruction manual. Figures 1 to 10The single-channel speech noise reduction method disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this specification can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0105] Of course, in addition to the software implementation, the electronic devices in the embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0106] Example 4

[0107] This specification also provides an embodiment of a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figures 1 to 10 The method of the illustrated embodiment is specifically used to perform the following method:

[0108] S1: For the acquired speech time-domain signal Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal ;

[0109] S2: Input the signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer;

[0110] S3: Divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model. Then, concatenate the signal features divided into sub-bands with the signal features output by the frequency GRU model and input them into the sub-band processing model. The sub-band processing model includes one GRU layer, one linear layer and one one-dimensional conventional convolutional layer in sequence.

[0111] S4: The signal features output by the sub-band processing model are processed by a time GRU model and then input into the decoding model. The decoding model includes three sets of one-dimensional transposed convolutional layers.

[0112] S5: After optimizing the signal features output by the decoding model, output the speech time-domain signal. .

[0113] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0114] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an electronic data carrier device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0115] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0116] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A single-channel speech noise reduction method, characterized in that, include: For the collected speech time domain signal Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Including pure speech time domain signal and noisy speech time domain signal ; The signal features are input into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer. The signal features output by the coding model are divided into sub-bands and input into the frequency GRU model. The signal features divided into sub-bands are then concatenated with the signal features output by the frequency GRU model and input into the sub-band processing model. The sub-band processing model includes one GRU layer, one linear layer and one one-dimensional conventional convolutional layer. The signal features output by the sub-band processing model are processed by a time GRU model and then input into the decoding model. The decoding model includes three sets of one-dimensional transposed convolutional layers. After optimizing the signal features output by the decoding model, the speech time-domain signal is output. .

2. The single-channel speech noise reduction method according to claim 1, characterized in that, The Fourier transform has 512 points, and the output is the signal features obtained from the Fourier transform and extraction of the speech time-domain signal based on the amplitude of the frequency domain signal. The signal features obtained from the Fourier transform and extraction of the speech time-domain signal include 257 frequency points; and / or, The kernels of the one-dimensional conventional convolutional layer, the first depthwise separable convolutional layer, and the second depthwise separable convolutional layer are 5, 3, and 5, respectively, with corresponding strides of 2, 1, and 2; and / or, The three sets of one-dimensional transposed convolutional layers have kernels of 5, 3, and 5, with corresponding strides of 2, 1, and 2, respectively, and a batch regularization layer and a ReLU layer connected to the output of each set of one-dimensional regular convolutional layers; and / or, The encoding model and the decoding model exchange information using skip connections.

3. The single-channel speech noise reduction method according to claim 2, characterized in that, The coding model further includes, in turn: A batch regularization layer and a ReLU layer connected to the output of a one-dimensional regular convolutional layer; a batch regularization layer and a ReLU layer connected to the input of a first depth separable convolutional layer; and / or, Both the first depthwise separable convolutional layer and the second depthwise separable convolutional layer include a one-dimensional Depthwise convolutional layer and a one-dimensional Pointwise convolutional layer, as well as a batch regularization layer and a ReLU layer connected to the output of the one-dimensional Depthwise convolutional layer, and a batch regularization layer and a ReLU layer connected to the output of the one-dimensional Pointwise convolutional layer.

4. The single-channel speech noise reduction method according to claim 3, characterized in that, The process of dividing the signal features output by the coding model into sub-bands includes: Select consecutive frequencies before and after the current frequency point. Each frequency point constitutes a length of [number]. Sub-bands; If the number of frequency points before and after the current frequency point is insufficient At any given time, a circular cyclic method is used to select frequency points to form sub-bands; The sub-bands corresponding to all frequency points are used to form a frequency domain sub-band frequency point matrix.

5. The single-channel speech noise reduction method according to claim 4, characterized in that, Both the frequency GRU model and the time GRU model include a set of GRU layers and a set of one-dimensional conventional convolutional layers connected to the output of the GRU layers.

6. The single-channel speech noise reduction method according to claim 5, characterized in that, After optimizing the output of the decoding model, the speech time-domain signal is output. The process includes: Based on the signal characteristics output by the decoding model, a corresponding complex ideal ratio mask is obtained; The complex ideal ratio mask real part and the virtual part Acting on the frequency domain signal respectively real part and the virtual part The optimized pure speech domain signal estimate was calculated. real part = - and the virtual part = + ; Estimated value of the pure speech audio domain signal Perform inverse Fourier transform to output time-domain signal .

7. The single-channel speech noise reduction method according to claim 6, characterized in that, This also includes training the neural network during the speech denoising process, using a loss function of... ,in, , ; and / or, An Adam optimizer with a learning rate of 0.001 is used, which is adjusted based on the preset number of optimization rounds and / or the optimization verification results.

8. A single-channel speech noise reduction device, characterized in that, include: The first module is used to process the acquired speech time-domain signal. Perform Fourier transform to obtain the frequency domain signal And extract the corresponding signal features, the speech time-domain signal Pure speech time domain signal and noisy speech time domain signal ; The second module is used to input signal features into the coding model, which sequentially includes a one-dimensional conventional convolutional layer, a first depthwise separable convolutional layer, and a second depthwise separable convolutional layer. The third module is used to divide the signal features output by the coding model into sub-bands and input them into the frequency GRU model. The signal features divided into sub-bands are then concatenated with the signal features output by the frequency GRU model and input into the sub-band processing model. The sub-band processing model includes, in sequence, one GRU layer, one linear layer, and one one-dimensional conventional convolutional layer. The fourth module is used to input the signal features output by the sub-band processing model into the decoding model after passing through the time GRU model. The decoding model includes three sets of one-dimensional transposed convolutional layers. The fifth module is used to optimize and calculate the signal features output by the decoding model and then output the speech time-domain signal. .

9. An electronic device, characterized in that, include: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the single-channel speech noise reduction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform the single-channel speech noise reduction method according to any one of claims 1 to 7.

Images