Audio noise reduction method and device, medium and product

Abstract

The invention provides an audio noise reduction method and device, a medium and a product, and relates to the technical field of audio noise reduction. The audio noise reduction method comprises the steps that a time-frequency graph corresponding to an original audio signal is acquired, the time-frequency graph is input into a preset model, a first noise recognition result is obtained, and the preset model is obtained through training by using a first convolutional neural network based on time-frequency graphs of different types of noise; performing noise reduction processing according to the first noise identification result to obtain a first audio signal corresponding to the original audio signal; and performing noise reduction processing on the first audio signal by using an adaptive filter and a second convolutional neural network to obtain a target audio signal. According to the embodiment of the invention, the accuracy of noise signal separation is effectively improved through a mode of combining dual noise reduction processing and a traditional filtering method with artificial intelligence, the number and influence of noise in the audio signal are reduced, the noise reduction effect is greatly improved, and the use experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio noise reduction, and more specifically, to an audio noise reduction method, device, medium and product. Background Art

[0002] The audio signals collected by the terminal equipment usually contain noise. When further processing the audio signals, it is usually necessary to perform voice noise reduction. Voice noise reduction is an effective processing technology for noise problems. Its purpose is to eliminate the influence of noise, improve voice clarity and improve voice quality.

[0003] For example, during a voice call, the audio signal collected by the terminal may contain environmental noise, other people's voices, etc. Before encoding and transmitting the audio signal, the terminal usually needs to perform voice noise reduction processing on the audio signal to obtain a pure voice signal. A pure voice signal refers to a voice signal obtained after eliminating the noise signal in the audio signal. A pure voice signal does not contain noise, or the noise contained is within the allowable error range.

[0004] In the prior art, a noise reduction algorithm (e.g., the least mean square algorithm) is usually used to perform a single noise reduction process on a speech signal. However, the existing noise reduction method cannot accurately separate the noise signal, and the audio signal obtained after the noise reduction process by the existing noise reduction method still has a lot of noise, and the noise reduction effect is not good. Summary of the invention

[0005] The embodiments of the present application provide an audio noise reduction method, device, medium and product, which can solve the problem that there is still a lot of noise after the existing audio signal noise reduction processing and the noise reduction effect is poor. To achieve this purpose, the embodiments of the present application provide the following solutions.

[0006] According to one aspect of an embodiment of the present application, there is provided an audio noise reduction method, comprising:

[0007] Obtain a time-frequency graph corresponding to the original audio signal, and input the time-frequency graph into a preset model to obtain a first noise recognition result, wherein the preset model is obtained by training a first convolutional neural network based on time-frequency graphs of different types of noise;

[0008] Performing noise reduction processing according to the first noise recognition result to obtain a first audio signal corresponding to the original audio signal;

[0009] The first audio signal is subjected to noise reduction processing using an adaptive filter and a second convolutional neural network to obtain a target audio signal.

[0010] In a possible implementation, the obtaining of a time-frequency graph corresponding to the original audio signal includes:

[0011] Performing enhancement processing on the original audio signal to obtain a second audio signal, wherein the enhancement processing includes one or more of delayed echo, reverberation, harmonic flanger, pitch up / down adjustment, and similar noise mixing;

[0012] Preprocessing the second audio signal to obtain an audio signal to be analyzed, wherein the preprocessing includes at least one of resampling, pre-emphasis, framing, and windowing;

[0013] Obtain a time-frequency diagram of the audio signal to be analyzed.

[0014] In a possible implementation, the preprocessing includes resampling and pre-emphasis, and the preprocessing of the second audio signal includes:

[0015] resampling the enhanced second audio signal based on a preset sampling rate;

[0016] A preset number of first-order high-pass filters are used to perform pre-emphasis processing on the resampled second audio signal.

[0017] In a possible implementation, the preprocessing includes framing and windowing, and the preprocessing of the second audio signal includes:

[0018] Determining a segmentation unit and a step offset according to an audio change feature of the second audio signal;

[0019] processing the second audio signal based on the segmentation unit and the step offset;

[0020] A window function corresponding to the processed second audio signal is obtained, and window processing is performed on the processed second audio signal based on the window function.

[0021] In a possible implementation, the obtaining of the preset model includes:

[0022] Obtain time-frequency plots of different types of noise;

[0023] The time-frequency graph is determined as a training sample, and model training is performed based on the training sample and a first convolutional neural network to obtain the preset model, wherein the first convolutional neural network includes a ResNet50 residual convolutional neural network, and the convolutional layer of the ResNet50 residual convolutional neural network includes a convolutional block of the type of variable convolution.

[0024] In a possible implementation, the denoising process of the first audio signal by using an adaptive filter and a second convolutional neural network includes:

[0025] Processing the first audio signal through an adaptive filter, and updating coefficients of the adaptive filter according to an error between an actual output of the adaptive filter and an expected output;

[0026] The signal sequence output by the adaptive filter is input into the denoising model corresponding to the second convolutional neural network to obtain the denoising result output by the denoising model.

[0027] In a possible implementation, the processing the first audio signal by using an adaptive filter includes:

[0028] Acquire an echo output signal and a corresponding echo signal, wherein the echo output signal is collected when the audio is playing;

[0029] Inputting the echo output signal and the echo signal into a preset prediction model to obtain an echo prediction signal, wherein the preset prediction model includes a Kriging interpolation model;

[0030] The echo prediction signal is input into the adaptive filter for echo noise reduction processing.

[0031] According to one aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0032] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are implemented.

[0033] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0034] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0035] The audio denoising method provided in the present application obtains a time-frequency graph corresponding to an original audio signal, inputs the time-frequency graph into a preset model obtained by training a first convolutional neural network based on time-frequency graphs of different types of noise, and obtains a first noise recognition result; performs denoising processing according to the first noise recognition result to obtain a first audio signal corresponding to the original audio signal; uses an adaptive filter and a second convolutional neural network to perform denoising processing on the first audio signal to obtain a target audio signal. The embodiment of the present application can convert an audio signal into an image signal through a time-frequency graph, perform preliminary denoising processing using the image signal, and combine the adaptive filter with the convolutional neural network to further remove noise from the audio signal. The double denoising processing method effectively improves the accuracy of noise signal separation, reduces the amount and influence of noise in the audio signal, greatly improves the denoising effect, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for use in describing the embodiments of the present application.

[0037] Figure 1 A flowchart of an audio noise reduction method provided in an embodiment of the present application;

[0038] Figure 2 A specific workflow diagram of the audio noise reduction method provided in an embodiment of the present application;

[0039] Figure 3 This is a structural diagram of the electronic device of this application. DETAILED DESCRIPTION

[0040] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0041] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates that it is implemented as "A", or is implemented as "A", or is implemented as "A and B".

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0043] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0044] The audio noise reduction method, device, medium and product provided in this application are intended to solve at least one technical problem existing in the prior art.

[0045] The present application provides an audio noise reduction method, such as Figure 1 , Figure 2 As shown, the audio noise reduction method can be performed by mobile phones, computers, servers, cameras and other terminals that can load convolutional neural networks and use convolutional neural networks to perform noise reduction processing.

[0046] Optionally, the audio noise reduction method includes:

[0047] S101: Obtain a time-frequency graph corresponding to an original audio signal, input the time-frequency graph into a preset model, and obtain a first noise recognition result.

[0048] Optionally, the preset model is obtained by training the first convolutional neural network based on the time-frequency diagrams of different types of noise. The preset model is used to preliminarily perform noise detection.

[0049] Optionally, obtaining the time-frequency diagram corresponding to the original audio signal includes: enhancing the original audio signal to obtain a second audio signal, wherein the enhancement processing includes one or more of delayed echo, reverberation, harmonic flanger, pitch increase or decrease, and similar noise mixing; preprocessing the second audio signal to obtain an audio signal to be analyzed, wherein the preprocessing includes at least one of resampling, pre-emphasis, framing, and windowing; and obtaining the time-frequency diagram of the audio signal to be analyzed.

[0050] Optionally, the original audio signal may be transmitted to the terminal by other devices, or may be obtained by the terminal from the Internet, or may be read from a local storage device or an external storage device.

[0051] Optionally, the original audio signal may be a digital signal or an analog signal. When the original audio signal is an analog signal, it is first converted into a digital signal, and then the audio noise reduction method of the present application is executed.

[0052] Optionally, when the original audio signal is enhanced, a parameter may be set or an instruction may be received to select an enhanced processing method for the original audio signal. Alternatively, a corresponding enhanced processing method may be selected according to a sound effect requirement corresponding to the original audio signal.

[0053] Optionally, delayed echo processing can produce repeated sounds to increase the layering and depth of the audio. Delayed echo is a natural manifestation of delay, which increases the stereoscopic and spatial sense of the audio when the sound waves encounter a reflective surface and return to the listener's ears. Reverberation processing can simulate sound reflections in different spaces, adding a natural or imaginary sense of space to the audio. Harmonic flanging creates rich harmonic effects by superimposing a modulated delayed signal on the original audio signal. Unique sound landscapes can be created by turning the audio up or down, such as simulating robot sounds or making sound effects similar to cartoon characters. The mixing of similar noises can mix noises that are similar to the original audio signal (such as frequency or waveform) with the original audio, which can increase the density and atmosphere of the audio. Among them, the enhancement processing can also include other processing methods that can adjust the sound effects corresponding to the audio signal.

[0054] In one embodiment, the original audio signal is enhanced by delaying echo, reverberation, harmonic flanging, pitch up and down modulation, and similar noise mixing.

[0055] Optionally, after performing enhancement processing to obtain the second audio signal, the second audio signal is preprocessed. Wherein, when the processing includes resampling and pre-emphasis, the preprocessing of the second audio signal includes: resampling the second audio signal after enhancement processing based on a preset sampling rate; and performing pre-emphasis processing on the resampled second audio signal using a preset number of first-order high-pass filters.

[0056] Optionally, the preset sampling rate can be set according to the audio noise reduction requirements, and the preset sampling rate should not affect the audio processing results obtained by the audio noise reduction. The preset sampling rate can be determined according to the noise reduction process (such as obtaining the first noise recognition result, obtaining the target audio signal by noise reduction processing of the first audio signal, etc.).

[0057] Optionally, the drop in high-end spectrum caused by the glottal pulse is offset by pre-emphasis processing, so that the signal spectrum of the second audio signal becomes flat and the resonance amplitudes of audio signals of different frequencies in the second audio signal are similar.

[0058] Optionally, the preset number is 1, and the preset number may also be 3, 5 or other numbers. The number of first-order high-pass filters may be set according to the high-end spectrum drop caused by the gate pulse so that the pre-emphasis operation can offset the high-end spectrum drop caused by the glottal pulse.

[0059] In one embodiment, the resampled audio sampling frequency is 16KHZ. After the resampling process, a first-order high-pass filter is used to perform pre-emphasis processing on the resampled audio signal.

[0060] Optionally, when preprocessing includes framing and windowing, preprocessing the second audio signal includes: determining a segmentation unit and a step offset based on audio change characteristics of the second audio signal; processing the second audio signal based on the segmentation unit and the step offset; obtaining a window function corresponding to the processed second audio signal, and windowing the processed second audio signal based on the window function.

[0061] Optionally, the audio change feature includes at least one of a time domain feature, a frequency domain feature and a time-frequency domain feature. The second audio signal may be segmented according to the audio change feature to ensure that the audio signal obtained after segmentation can reflect the audio change feature.

[0062] In one embodiment, a continuous audio signal is divided into signals in frames with 30 ms as the division unit, and the step offset between adjacent frames is 10 ms. After the audio is framed, in order to ensure the smoothness and continuity between frames, a window function is used to reduce the impact of truncation. The window function is used to reduce leakage and sidelobe effects, thereby improving the accuracy and reliability of spectrum analysis.

[0063] Optionally, after preprocessing the second audio signal to obtain the audio signal to be analyzed, a time-frequency diagram of the audio signal to be analyzed is generated. The process of generating the time-frequency diagram is as follows: transforming the audio signal to be analyzed from the time domain to the frequency domain, combining the audio spectra within a preset time period according to time for the audio signal to be analyzed transformed into the frequency domain, and generating the time-frequency diagram.

[0064] Optionally, after the time-frequency graph is generated, the time-frequency graph is scaled to a fixed size corresponding to a preset model.

[0065] In one embodiment, the preset time period is 3 seconds, and the audio signal to be analyzed is transformed from the time domain to the frequency domain by discrete Fourier transform. Then, based on the 3s time standard, each audio frame spectrum of the audio signal to be analyzed within 3s is combined by time to generate a time-frequency diagram and scale it to a fixed size of 224×224.

[0066] Optionally, obtaining a preset model includes: obtaining time-frequency diagrams of different types of noise; determining the time-frequency diagrams as training samples, performing model training based on the training samples and a first convolutional neural network to obtain a preset model, wherein the first convolutional neural network includes a ResNet50 residual convolutional neural network, and the convolution layer of the ResNet50 residual convolutional neural network includes a convolution block of the type of variable convolution.

[0067] Optionally, the time-frequency diagram of different types of noise can be generated based on the noise signals of different types of noise collected or stored. Various types of noise signals can be analyzed by time-frequency diagram analysis to generate time-frequency diagrams corresponding to the noise signals. The time-frequency diagram includes the frequency components of the noise signal, the change information of these frequency components over time, etc., and the time-frequency diagram provides support for further processing and analysis of the noise.

[0068] Optionally, the preset model corresponds to the input frequency spectrum image size requirements including: a minimum size of 224×224 and a maximum image size of 896×896.

[0069] In one embodiment, the first convolutional neural network uses an improved ResNet50 residual convolutional neural network—ResNet50_dcn, whose feature enhancement network fuses the feature maps output by Conv_3, Conv_4, and Conv_5 in the backbone network, and outputs a feature map FP_y (y=3, 4, 5) to enhance the discriminability of the feature. The classification network of the first convolutional neural network fuses the feature map FP_y (y=3, 4, 5), and the number of neurons in the output layer of the classification network corresponds to the number of noise categories (e.g., when the number of noise categories is 3, the output layer includes 3 neurons), so that the noise classification is detected by the first convolutional neural network. In addition, the 3×3 ordinary convolutions in the last three convolutional blocks in the convolutional layer are replaced with deformable convolutions, which is more helpful in distinguishing the features of the time-frequency maps of different categories of audio noise.

[0070] Optionally, the preset model obtained by training samples and the first convolutional neural network includes a network structure of multiple convolutional layers and pooling layers to extract local features in the time-frequency graph. In the first convolutional layer of the preset model, multiple small-sized filters are used for convolution and multiple feature maps are generated. Starting from the second layer, a similar number of filters continue to be used and a nonlinear activation function is also used. In addition, the preset model performs batch normalization operations between the convolutional layer and the activation layer, which can increase the training speed and speed up the convergence process. In order to more effectively capture the time and spectrum features related to sound events, time attention and spectrum attention mechanisms are also introduced, which can help the model focus more on discriminative features, thereby improving the accuracy of classification.

[0071] Optionally, the first noise recognition result includes information of various detected noises (such as frequency, amplitude, and location information of the noise, etc.), and the noise is preliminarily removed through the recognition result.

[0072] S102: Perform noise reduction processing according to the first noise recognition result to obtain a first audio signal corresponding to the original audio signal.

[0073] Optionally, after obtaining the first noise recognition result, the noise may be removed based on the first noise recognition result by convolution, mean filtering, etc. The noise may also be removed by edge extraction.

[0074] S103: Use an adaptive filter and a second convolutional neural network to perform noise reduction processing on the first audio signal to obtain a target audio signal.

[0075] Optionally, the first audio signal is denoised using an adaptive filter and a second convolutional neural network, including: processing the first audio signal through an adaptive filter, and updating the coefficients of the adaptive filter based on the error between the actual output of the adaptive filter and the expected output; inputting the signal sequence output by the adaptive filter into a denoising model corresponding to the second convolutional neural network, and obtaining a denoising result output by the denoising model.

[0076] Optionally, after the noise is initially removed to obtain the first audio signal, the first audio signal is processed by the adaptive filter and the noise reduction model corresponding to the second convolutional neural network to reduce the noise again. And when reducing the noise, the weighted sum of each item is calculated and updated according to the prediction error and the length of the adaptive filter, so that the adaptive filter converges to the state of minimum mean square error. The prediction error refers to the difference between the actual output signal of the adaptive filter and the expected output signal, and its size reflects the processing effect of the filter on the input signal. Among them, the coefficients of the adaptive filter can be updated by the minimum mean square algorithm to achieve the update of the weighted sum.

[0077] In one embodiment, the first audio signal can be represented as an input signal sequence x(n) with a length of N, which is processed by an adaptive filter h(n) to obtain an output signal y(n). The output signal y(n) can be represented as Where M is the length of the adaptive filter, h(k) is the kth coefficient of the adaptive filter, and the coefficient of the adaptive filter is updated by the least mean square algorithm. The update rule of the coefficient h(k) of the adaptive filter can be:

[0078] h(k+1)=h(k)+μe(n)x(nk), where e(n) is the prediction error, μ is the learning rate parameter, and e(n) is the difference between the actual output signal and the expected output signal, that is, e(n)=d(n)-y'(n), d(n) is the actual output signal, and y'(n) is the difference between the expected output signals.

[0079] Optionally, the second convolutional neural network can also be a convolutional neural network, specifically, it can be LSTM, and the denoising model constructed based on LSTM includes an input gate, a forget gate, and an output gate. The signal sequence input to the denoising model can be y(n), h'(n) is the hidden layer state sequence, c(n) is the cell state sequence, then the input gate i(n) is i(n)=σ[W xi y(n)+W hi h'(n-1)+W ci c(n-1)+b i ], where σ is the sigmoid function, W xi , W hi , W ci is the weight matrix of the input gate, b i is the bias vector. The value of the forget gate f(n) can be obtained by the formula f(n) = σ[w xf y(n)+w hf h'(n-1)+w cf c(n-1)+b f ] calculation, in which w xf 、w hf 、w cf are the weight matrices of the forget gate, b f is the bias vector. The cell state sequence c(n) can be obtained by the formula c(n) = f(n)⊙c(n-1)+i(n)⊙tanh[w xc y(n)+w hc h'(n-1)+b c ] calculation, the formula for updating the cell state sequence is: ⊙ represents element-by-element multiplication, tanh(·) is the hyperbolic tangent function, and w xc 、w hc is the weight matrix of the cell state, bc is the bias vector.

[0080] The value of the output gate o(n) is o(n) = σ[w xo y(n)+w ho h'(n-1)=w co c(n-1)+b o ], where w xo 、w ho 、w co is the weight matrix, b o is the bias vector. The value of the hidden state h'(n) can be calculated by the formula h'(n)=o(n)⊙tanh[c(n)]. The above method can be used to obtain the noise reduction result sequence (ie, the target audio signal) after the noise reduction process.

[0081] Optionally, since echoes are generated when playing audio, noise reduction processing can also be performed when the first audio signal is processed by an adaptive filter and a noise reduction model. The first audio signal is processed by an adaptive filter, including: obtaining an echo output signal and a corresponding echo signal, the echo output signal is collected when the audio is playing; inputting the echo output signal and the echo signal into a preset prediction model to obtain an echo prediction signal, the preset prediction model includes a Kriging interpolation model; inputting the echo prediction signal into an adaptive filter for echo noise reduction processing.

[0082] Optionally, the echo output signal can be obtained by collecting the sound of an audio player such as a speaker using a sound collection device such as a microphone. The corresponding echo signal can be an audio signal in the first audio signal corresponding to the echo in the echo output signal, or can be an audio signal in the target audio signal corresponding to the echo in the echo output signal. The audio signal corresponding to each echo output signal can be determined based on the difference between the collection time of the echo output signal and the acquisition time of the corresponding audio signal.

[0083] Optionally, the echo noise reduction process can be performed after the target audio signal is acquired, or when the first audio signal is transmitted and an adaptive filter is given. Wherein, before the audio signal is acquired and the target audio signal is subjected to noise reduction process, the target audio signal can be enhanced, and the process of the enhanced process can be consistent with the enhanced process operation of the original audio signal.

[0084] Optionally, the preset prediction model can be a Kriging interpolation model. The echo signal currently included in the first audio signal of the Kriging interpolation model is used to transmit the predicted echo signal to the adaptive filter. The adaptive filter performs noise reduction processing on the first audio signal or the target audio signal according to the echo signal, and transmits the audio signal after noise reduction processing to the noise reduction model.

[0085] The audio noise reduction method of the present application has the following advantages:

[0086] 1. Combine traditional filtering methods with artificial intelligence technology to build an overall framework for audio noise reduction, perform more advanced processing and feature learning on audio signals, and further improve the effect of audio noise reduction;

[0087] 2. Use the time-frequency graph analysis method to analyze various types of noise, and then use the AI ​​convolutional neural network structure to train and learn the time-frequency graphs of different types of noise, summarize the time-frequency characteristics of the same type of noise audio, and convert the audio signal into an image signal for noise detection.

[0088] 3. The echo signal is predicted through the Kriging interpolation model, and then the echo noise reduction is performed to reduce the interference caused by the echo during audio playback.

[0089] In an optional embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0090] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0091] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0092] Memory 4003 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disc, optical disk, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium, other magnetic storage devices or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0093] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0094] Among them, the electronic device can be any electronic product that can interact with an object, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0095] The electronic device may also include a network device and / or an object device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers for cloud computing.

[0096] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), etc.

[0097] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0098] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0099] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0100] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0101] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. An audio noise reduction method, characterized in that: include: Obtain a time-frequency graph corresponding to the original audio signal, and input the time-frequency graph into a preset model to obtain a first noise recognition result, wherein the preset model is obtained by training a first convolutional neural network based on time-frequency graphs of different types of noise; Performing noise reduction processing according to the first noise recognition result to obtain a first audio signal corresponding to the original audio signal; The first audio signal is subjected to noise reduction processing using an adaptive filter and a second convolutional neural network to obtain a target audio signal.

2. The audio noise reduction method according to claim 1, characterized in that: The step of obtaining a time-frequency diagram corresponding to the original audio signal includes: Performing enhancement processing on the original audio signal to obtain a second audio signal, wherein the enhancement processing includes one or more of delayed echo, reverberation, harmonic flanger, pitch up / down adjustment, and similar noise mixing; Preprocessing the second audio signal to obtain an audio signal to be analyzed, wherein the preprocessing includes at least one of resampling, pre-emphasis, framing, and windowing; Obtain a time-frequency diagram of the audio signal to be analyzed.

3. The audio noise reduction method according to claim 2, characterized in that: The preprocessing includes resampling and pre-emphasis, and the preprocessing of the second audio signal includes: resampling the enhanced second audio signal based on a preset sampling rate; A preset number of first-order high-pass filters are used to perform pre-emphasis processing on the resampled second audio signal.

4. The audio noise reduction method according to claim 2, characterized in that: The preprocessing includes framing and windowing, and the preprocessing of the second audio signal includes: Determining a segmentation unit and a step offset according to an audio change feature of the second audio signal; processing the second audio signal based on the segmentation unit and the step offset; A window function corresponding to the processed second audio signal is obtained, and window processing is performed on the processed second audio signal based on the window function.

5. The audio noise reduction method according to claim 1, characterized in that: The acquisition of the preset model includes: Obtain time-frequency plots of different types of noise; The time-frequency graph is determined as a training sample, and model training is performed based on the training sample and a first convolutional neural network to obtain the preset model, wherein the first convolutional neural network includes a ResNet50 residual convolutional neural network, and the convolutional layer of the ResNet50 residual convolutional neural network includes a convolutional block of the type of variable convolution.

6. The audio noise reduction method according to claim 1, characterized in that: The denoising process of the first audio signal by using an adaptive filter and a second convolutional neural network includes: Processing the first audio signal through an adaptive filter, and updating coefficients of the adaptive filter according to an error between an actual output of the adaptive filter and an expected output; The signal sequence output by the adaptive filter is input into the denoising model corresponding to the second convolutional neural network to obtain the denoising result output by the denoising model.

7. The audio noise reduction method according to claim 6, characterized in that: The step of processing the first audio signal by using an adaptive filter comprises: Acquire an echo output signal and a corresponding echo signal, wherein the echo output signal is collected when the audio is playing; Inputting the echo output signal and the echo signal into a preset prediction model to obtain an echo prediction signal, wherein the preset prediction model includes a Kriging interpolation model; The echo prediction signal is input into the adaptive filter for echo noise reduction processing.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images