Audio noise reduction methods, media, apparatuses, and computing devices

Abstract

The embodiment of the present disclosure provides an audio noise reduction method, medium, device and computing device, relating to the technical field of artificial intelligence, and the method comprises the following steps: acquiring a to-be-reduced-noise audio signal, performing time-frequency conversion processing on the to-be-reduced-noise audio signal to obtain a to-be-reduced-noise frequency domain signal; extracting acoustic features of a preset dimension of the to-be-reduced-noise frequency domain signal, wherein the acoustic features comprise at least one of signal Bark domain amplitude features, signal pitch features, first-order and second-order pitch difference features and signal flatness features; determining a first probability that the to-be-reduced-noise audio signal contains a music signal according to the acoustic features; and performing noise reduction processing on the to-be-reduced-noise frequency domain signal based on the first probability and an amplitude spectrum of the to-be-reduced-noise frequency domain signal to obtain a noise-reduced audio signal corresponding to the to-be-reduced-noise audio signal. When the present disclosure performs noise reduction processing on a noisy music signal, the music quality can be protected while suppressing noise, and the user can have an extreme music quality experience with high fidelity.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of artificial intelligence technology, and more specifically, the embodiments of this disclosure relate to an audio noise reduction method, medium, apparatus, and computing device. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Audio Noise Suppression (ANS) is an audio signal enhancement technique used to extract useful audio signals (i.e., the noise-reduced signal) from noisy audio signals when the audio signal is interfered with or even submerged by various background noises, thus suppressing or reducing noise interference. In speech processing, such as in Real-Time Communication (RTC) scenarios, ANS plays a crucial role, effectively suppressing noise in the acquired speech signal and safeguarding the audio quality of the call. However, when using ANS to denoise noisy music signals (containing both music and noise), it can severely damage the music signal, resulting in a poor audio quality experience for the user.

[0004] Therefore, how to provide users with a high-fidelity, ultimate music sound quality experience while suppressing noise is a challenge encountered when applying audio noise reduction technology to noisy music signals. Summary of the Invention

[0005] This disclosure provides an audio noise reduction method, medium, apparatus, and computing device to solve the problem that when using ANS to perform noise reduction processing on noisy music signals, the music signal is severely damaged, resulting in a poor music sound quality experience for users.

[0006] In a first aspect of this disclosure, an audio noise reduction method is provided, comprising:

[0007] Acquire the noise frequency signal to be reduced;

[0008] The frequency signal to be denoised is subjected to time-frequency conversion processing to obtain the frequency domain signal to be denoised;

[0009] The acoustic features of the frequency domain signal to be denoised are extracted in a preset dimension. The acoustic features include at least one of the following: signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features.

[0010] Based on acoustic characteristics, determine the first probability that the noise frequency signal to be reduced contains a music signal;

[0011] Based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, the frequency domain signal to be denoised is denoised to obtain the denoised frequency signal corresponding to the frequency domain signal to be denoised.

[0012] In one possible implementation, the noise reduction process is performed on the frequency domain signal to be denoised based on a first probability and the amplitude spectrum of the frequency domain signal to be denoised, to obtain a noise-reduced frequency signal corresponding to the frequency domain signal to be denoised. This includes: performing noise estimation on the noise signal in the amplitude spectrum to obtain a first noise spectrum; performing constraint processing on the first noise spectrum based on the first probability to obtain a constraint-processed noise spectrum; and performing noise reduction processing on the frequency domain signal to be denoised based on the first probability and the constraint-processed noise spectrum to obtain a noise-reduced frequency signal corresponding to the frequency domain signal to be denoised.

[0013] In one possible implementation, the first noise spectrum is constrained based on a first probability to obtain a constrained noise spectrum, including: obtaining the constrained noise spectrum according to the following formula:

[0014] D modify (k,l)=D estimated (k,l)*(1-MPP(l))+D estimated (k,l-1)*

[0015] MPP(l)

[0016] Among them, D modify (k,l) represents the noise spectrum after constraint processing; D estimated (k,l) represents the noise spectrum of the l-th frame at the k-th frequency point, i.e., the first noise spectrum; D estimated (k,l-1) represents the noise spectrum of the (l-1)th frame at the k-th frequency point; MPP(l) represents the first probability.

[0017] In one possible implementation, based on a first probability and a constrained noise spectrum, a noise reduction process is performed on the frequency domain signal to be denoised to obtain a noise-reduced frequency domain signal corresponding to the noise-reduced frequency signal. This includes: obtaining a second probability that the noise-reduced frequency signal contains a speech signal based on the amplitude spectrum and the constrained noise spectrum; obtaining a signal-to-noise ratio (SNR) based on the amplitude spectrum and the constrained noise spectrum; updating the constrained noise spectrum based on the second probability and the SNR to obtain an updated noise spectrum; constraining the updated noise spectrum based on the first probability to obtain a second noise spectrum; obtaining a Wiener value based on the second noise spectrum, the second probability, and the amplitude spectrum; performing noise reduction on the noise-reduced frequency domain signal based on the Wiener value to obtain a noise-reduced frequency domain signal; and performing time-frequency conversion on the noise-reduced frequency domain signal to obtain the noise-reduced frequency domain signal corresponding to the noise-reduced frequency signal.

[0018] In one possible implementation, determining the first probability that a music signal exists in the noise frequency signal to be reduced, based on acoustic characteristics, includes: inputting the acoustic characteristics into a music detection model to determine the first probability that a music signal exists in the noise frequency signal to be reduced.

[0019] In one possible implementation, the music detection model includes a fully connected layer, a first GRU layer, and a second GRU layer. Inputting acoustic features into the music detection model to determine the first probability that the noise frequency signal to be reduced contains a music signal includes: inputting acoustic features into the fully connected layer to obtain a first feature of a first dimension, where the first dimension is greater than a preset dimension; inputting the first feature into the first GRU layer to obtain a second feature of a second dimension, where the second dimension is greater than the first dimension; inputting the second feature into the second GRU layer to obtain a target feature; and determining the first probability that the noise frequency signal to be reduced contains a music signal based on the target feature and an activation function.

[0020] In one possible implementation, the music detection model is trained by: acquiring training samples, which include positive samples, negative samples, and labels. Positive samples include music signals, negative samples include speech signals and noise signals, and labels are used to distinguish between positive and negative samples; and iteratively training the music detection model based on the training samples until both the false recognition rate and the rejection rate of the music detection model are less than a threshold, thus obtaining a well-trained music detection model.

[0021] Secondly, embodiments of this disclosure provide an audio noise reduction device, comprising:

[0022] The acquisition module is used to acquire the noise frequency signal to be reduced;

[0023] The conversion module is used to perform time-frequency conversion processing on the noise-to-frequency signal to obtain the noise-to-frequency domain signal.

[0024] The extraction module is used to extract acoustic features of a preset dimension from the frequency domain signal to be denoised. The acoustic features include at least one of the following: signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features.

[0025] The determination module is used to determine the first probability that the noise frequency signal to be reduced contains a music signal based on the acoustic characteristics.

[0026] The processing module is used to perform noise reduction processing on the frequency domain signal to be denoised based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, so as to obtain the noise-reduced frequency signal corresponding to the frequency domain signal to be denoised.

[0027] In one possible implementation, the processing module is specifically used to: estimate the noise signal in the amplitude spectrum to obtain a first noise spectrum; perform constraint processing on the first noise spectrum based on a first probability to obtain a constrained noise spectrum; and perform noise reduction processing on the frequency domain signal to be denoised based on the first probability and the constrained noise spectrum to obtain a denoised frequency signal corresponding to the frequency signal to be denoised.

[0028] In one possible implementation, when the processing module performs constraint processing on the first noise spectrum based on a first probability to obtain a constraint-processed noise spectrum, it specifically performs the following steps: Obtaining the constraint-processed noise spectrum according to the following formula:

[0029] D modify (k,l)=D estimated (k,l)*(1-MPP(l))+D estimated (k,l-1)*

[0030] MPP(l)

[0031] Among them, D modify (k,l) represents the noise spectrum after constraint processing; D estimated (k,l) represents the noise spectrum of the l-th frame at the k-th frequency point, i.e., the first noise spectrum; D estimated (k,l-1) represents the noise spectrum of the (l-1)th frame at the k-th frequency point; MPP(l) represents the first probability.

[0032] In one possible implementation, when the processing module performs noise reduction processing on the frequency domain signal to be denoised based on the first probability and the noise spectrum after constraint processing to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal, it specifically performs the following steps: obtaining a second probability that the noise-reduced frequency signal contains a speech signal based on the amplitude spectrum and the noise spectrum after constraint processing; obtaining a signal-to-noise ratio based on the amplitude spectrum and the noise spectrum after constraint processing; updating the noise spectrum after constraint processing based on the second probability and the signal-to-noise ratio to obtain an updated noise spectrum; performing constraint processing on the updated noise spectrum based on the first probability to obtain a second noise spectrum; obtaining a Wiener value based on the second noise spectrum, the second probability, and the amplitude spectrum; performing noise reduction processing on the frequency domain signal to be denoised based on the Wiener value to obtain a noise-reduced frequency domain signal; and performing time-frequency conversion processing on the noise-reduced frequency domain signal to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

[0033] In one possible implementation, the determining module is specifically used to: input acoustic features into a music detection model to determine a first probability that the noise frequency signal to be reduced contains a music signal.

[0034] In one possible implementation, the music detection model includes a fully connected layer, a first GRU layer, and a second GRU layer. The determination module, when inputting acoustic features into the music detection model to determine the first probability that a music signal exists in the noise frequency signal to be reduced, specifically performs the following steps: inputting acoustic features into the fully connected layer to obtain a first feature of a first dimension, where the first dimension is greater than a preset dimension; inputting the first feature into the first GRU layer to obtain a second feature of a second dimension, where the second dimension is greater than the first dimension; inputting the second feature into the second GRU layer to obtain a target feature; and determining the first probability that a music signal exists in the noise frequency signal to be reduced based on the target feature and the activation function.

[0035] In one possible implementation, the audio noise reduction device further includes a training module for training a music detection model by: acquiring training samples, which include positive samples, negative samples, and labels. Positive samples include music signals, negative samples include speech signals and noise signals, and labels are used to distinguish between positive and negative samples. Based on the training samples, the music detection model is iteratively trained until the false recognition rate and false rejection rate of the music detection model are both less than a threshold, thereby obtaining a trained music detection model.

[0036] Thirdly, embodiments of this disclosure provide a computing device, including: a processor, and a memory communicatively connected to the processor;

[0037] The memory stores instructions that the computer executes;

[0038] The processor executes computer execution instructions stored in memory to implement the audio noise reduction method as described in the first aspect of this disclosure.

[0039] Fourthly, embodiments of this disclosure provide a storage medium storing computer program instructions, which, when executed, implement the audio noise reduction method as described in the first aspect of this disclosure.

[0040] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed, implements the audio noise reduction method as described in the first aspect of this disclosure.

[0041] The audio noise reduction method, medium, apparatus, and computing device provided in this disclosure acquire a noise-generating frequency signal, perform time-frequency conversion processing on the noise-generating frequency signal to obtain a noise-generating frequency domain signal; extract acoustic features of a preset dimension of the noise-generating frequency domain signal, the acoustic features including at least one of signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features; determine a first probability that the noise-generating frequency signal contains music signals based on the acoustic features; and perform noise reduction processing on the noise-generating frequency domain signal based on the first probability and the amplitude spectrum of the noise-generating frequency domain signal to obtain a noise-generating frequency signal corresponding to the noise-generating frequency signal. Since this disclosure determines the first probability that the noise-generating frequency signal contains music signals based on the acoustic features extracted from the noise-generating frequency domain signal, and uses the first probability to suppress noise in the noise-generating frequency signal, this disclosure can suppress noise without damaging the music quality when performing noise reduction processing on noisy music signals, achieving lossless processing of music signals and providing users with a high-fidelity, ultimate music quality experience. Attached Figure Description

[0042] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0043] Figure 1 A flowchart of an audio noise reduction method provided for related technologies;

[0044] Figure 2 A schematic diagram illustrating the effect of applying ANS to noisy speech signals, provided for related technologies;

[0045] Figure 3 A schematic diagram illustrating the effect of applying ANS to a noisy music signal, provided for related technologies.

[0046] Figure 4 A schematic diagram illustrating the effect of disabling ANS (Answer Key) on noisy music signals provided for related technologies;

[0047] Figure 5 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0048] Figure 6 A flowchart of an audio noise reduction method provided in an embodiment of this disclosure;

[0049] Figure 7 A flowchart of an audio noise reduction method provided in another embodiment of this disclosure;

[0050] Figure 8 A schematic diagram of a music detection model provided in an embodiment of this disclosure;

[0051] Figure 9 A flowchart of an audio noise reduction method provided in yet another embodiment of this disclosure;

[0052] Figure 10 This is a schematic diagram illustrating the effect of an audio noise reduction method provided in an embodiment of the present disclosure;

[0053] Figure 11 This is a schematic diagram of the structure of an audio noise reduction device provided in an embodiment of the present disclosure;

[0054] Figure 12 A schematic diagram of a storage medium provided in an embodiment of this disclosure;

[0055] Figure 13 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0056] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0057] The principles and spirit of this disclosure will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0058] Those skilled in the art will recognize that the embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. The data involved in this disclosure can be data authorized by the user or fully authorized by all parties, and the embodiments / examples of this disclosure can be combined with each other.

[0059] According to embodiments of this disclosure, an audio noise reduction method, medium, apparatus, and computing device are proposed.

[0060] In this article, it is important to understand the following terms:

[0061] RTC is a network communication system used for real-time forwarding of audio and video information;

[0062] The Short-Time Fourier Transform (STFT) is a time-frequency domain transformation technique used to transform time-domain signals into frequency-domain signals.

[0063] The inverse short-time fourier transform (iSTFT) is a time-frequency domain transformation technique used to transform frequency domain signals into time domain signals.

[0064] A neural network (NN) is a mathematical or computational model that mimics the structure of a biological neural network.

[0065] A gated recurrent unit (GRU) is a type of recurrent neural network.

[0066] Music Presence Probability (MPP) is the probability that a music signal exists in the current signal.

[0067] The False Acceptance Rate (FAR) is the probability of classifying a negative sample as a positive sample.

[0068] The False Rejection Rate (FRR) is the probability of classifying a positive sample as a negative sample.

[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0070] Furthermore, the number of any elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0071] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments. Invention Overview

[0073] The inventors have discovered that audio denoising is an audio signal enhancement technique that enhances the acquired audio while suppressing noise. Audio denoising techniques typically include seven modules: a short-time Fourier transform module, a noise spectrum estimation module, a signal presence probability estimation module, a signal-to-noise ratio calculation module, a noise spectrum update module, a Wiener filter calculation module, and an inverse short-time Fourier transform module.

[0074] For example, Figure 1 A flowchart of an audio noise reduction method provided for related technologies. For example... Figure 1 As shown, the acquired noise-to-reduced frequency signal y(n) is input into the short-time Fourier transform module to convert y(n) into a noise-to-reduced frequency domain signal Y(k,l), and the amplitude spectrum |Y(k,l)| of Y(k,l) is obtained, where n represents the time index, k represents the k-th frequency point, and l represents the l-th frame signal. The noise signal in |Y(k,l)| is estimated by the noise spectrum estimation module to obtain the noise spectrum |D(k,l)|. Based on |D(k,l)| and |Y(k,l)|, the probability VPP of the presence of a speech signal in Y(k,l) is obtained by the signal presence probability estimation module. Simultaneously, based on |Y(k,l)| and |D(k,l)|, the signal-to-noise ratio (SNR) is obtained by the signal-to-noise ratio (SNR) calculation module, which includes both prior and posterior SNR. Combining VPP and the SNR, |D(k,l)| is updated by the noise spectrum update module to obtain the updated noise spectrum. Based on the updated noise spectrum, VPP, and |Y(k,l)|, the Wiener value Gain(k,l) is obtained through the Wiener filtering calculation module. Applying the Wiener value Gain(k,l) to Y(k,l) yields the enhanced frequency domain signal X(k,l), which can also be referred to as the denoised frequency domain signal. The inverse short-time Fourier transform module converts X(k,l) into the enhanced signal x(n) corresponding to the noise-reduced frequency signal y(n), which can also be referred to as the denoised frequency signal x(n).

[0075] based on Figure 1 , Figure 2 The illustration provided for the effect of applying ANS to noisy speech signals is as follows: Figure 2 As shown, 201 represents the acquired noisy speech signal, and 202 represents the audio signal after ANS noise suppression (i.e., the enhanced speech signal). The horizontal axis represents time, and the vertical axis represents the signal amplitude (Amp). 2011 represents the human voice signal, 2012 represents noise, 2021 represents the human voice signal, and 2022 represents noise. Comparing 201 and 202, the amplitude value of 2022 is smaller than that of 2012. Therefore, it can be determined that after audio noise reduction processing, the noise is effectively suppressed, and the human voice signal in the audio is enhanced.

[0076] However, when using ANS to denoise noisy music signals (which include both music and noise), it can severely damage the music signal, resulting in a poor music quality experience for users. Figure 3 A schematic diagram illustrating the effect of applying ANS to a noisy music signal is provided for related technologies, such as... Figure 3As shown, 301 represents the acquired noisy music signal, and 302 represents the music signal after noise suppression using ANS. The horizontal axis represents time, and the vertical axis represents the signal amplitude (Amp). 3011 represents the music signal, and 3012 represents noise. Comparing 301 and 302, the amplitude of 3022 is smaller than that of 3012. However, some amplitude values ​​of 3021 are smaller than those of 3011. Therefore, it can be determined that directly applying ANS to the noisy music signal, while suppressing noise, severely damages the music signal. The reason for this damage is that the noise spectrum estimation and update modules in ANS estimate the diffuse components in the music signal as noise, thus suppressing the music signal. The damaged music signal results in a poor audio quality experience for the user.

[0077] In addition, in related technologies, when processing noisy music signals or in music scenarios, audio noise reduction technology is actively turned off in order to achieve high-fidelity music transmission. Turning off audio noise reduction technology can protect the music signal to the greatest extent and achieve high-fidelity music transmission. Figure 4 A schematic diagram illustrating the effect of disabling noisy music signals by turning off ANS, as provided for related technologies, is shown below. Figure 4 As shown, 401 represents the acquired noisy music signal, and 402 represents the music signal with ANS disabled. The horizontal axis represents time, and the vertical axis represents the signal amplitude (Amp). 4011 represents the music signal, and 4012 and 4021 represent the music signal and noise, respectively. Comparing 401 and 402, since ANS is disabled, 402 remains unchanged from 401. Therefore, it can be determined that disabling audio noise reduction technology makes it difficult to suppress the noise components in the acquired noisy music signal, resulting in a poor user experience.

[0078] To address the aforementioned issues, this disclosure provides an audio noise reduction method, medium, apparatus, and computing device. By converting the noise-reducing frequency signal into a noise-reducing frequency domain signal, analyzing the noise-reducing frequency domain signal, extracting necessary acoustic features, determining whether the noise-reducing frequency signal contains music signals based on the acoustic features, and outputting the probability that the noise-reducing frequency signal contains music signals, the noise in the noise-reducing frequency signal can be constrained by probability. This allows for the suppression of noise while providing users with a high-fidelity, ultimate music sound quality experience.

[0079] Application Scenarios Overview

[0080] First refer to Figure 5 Examples of application scenarios for the solutions provided in this disclosure are given. Figure 5 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure, such as... Figure 5As shown, in this application scenario, a user enters the streamer's live broadcast room via mobile phone 501 to watch the live broadcast. The live video stream is sent to mobile phone 501 via server 502. While the streamer is broadcasting, they play music through speakers. When mobile phone 501 receives the music signal from server 502, it performs audio noise reduction processing on the music signal to play high-fidelity music for the user.

[0081] It should be noted that, Figure 5 This is merely a schematic diagram illustrating one application scenario provided by an embodiment of this disclosure; this embodiment does not necessarily represent an application scenario. Figure 5 The included equipment is not limited, nor is it restricted. Figure 5 The positional relationships between the devices are defined.

[0082] Exemplary methods

[0083] The following is combined Figure 5 Application scenarios, refer to Figure 6 This document describes an audio noise reduction method according to exemplary embodiments of the present disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0084] First, the audio noise reduction method is introduced through specific examples.

[0085] Figure 6 This is a flowchart illustrating an audio noise reduction method according to an embodiment of the present disclosure. The method described in this embodiment can be applied to a computing device, which may be a terminal. Figure 6 As shown, the method in this embodiment includes:

[0086] S601, Obtain the noise frequency signal to be reduced.

[0087] In this embodiment of the disclosure, the noise frequency signal to be reduced may, for example, be input by a user to a computing device performing this method embodiment, or may be sent by another device to the computing device performing this method embodiment. For example, the noise frequency signal to be reduced may be a noise-generated music signal, which includes music signals and noise.

[0088] S602. Perform time-frequency conversion processing on the noise-to-frequency signal to be denoised to obtain the noise-to-frequency domain signal.

[0089] For example, refer to Figure 1 The noise-reducing frequency signal can be input into the short-time Fourier transform module for time-frequency conversion processing to obtain the noise-reducing frequency domain signal.

[0090] S603. Extract acoustic features of the frequency domain signal to be denoised in a preset dimension. The acoustic features include at least one of the following: signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features.

[0091] In this step, the preset dimension is, for example, 60 dimensions. For example, after obtaining the frequency domain signal to be denoised, the frequency domain signal to be denoised can be analyzed to extract 60-dimensional acoustic features of the frequency domain signal to be denoised. The 60-dimensional acoustic features include, for example, signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features of different dimensions. For specific methods of extracting acoustic features, please refer to the relevant current technologies.

[0092] S604. Based on acoustic characteristics, determine the first probability that the noise frequency signal to be reduced contains a music signal.

[0093] In this step, after extracting the acoustic features of the frequency domain signal to be denoised in a preset dimension, the first probability of the presence of a music signal in the frequency domain signal to be denoised can be determined based on the acoustic features. For example, the acoustic features are input into a pre-trained neural network to determine the first probability of the presence of a music signal in the frequency domain signal to be denoised. For details on how to determine the first probability of the presence of a music signal in the frequency domain signal to be denoised based on the acoustic features, please refer to subsequent embodiments; these will not be elaborated upon here.

[0094] S605. Based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, perform denoising processing on the frequency domain signal to be denoised to obtain the denoised frequency signal corresponding to the frequency domain signal to be denoised.

[0095] In this step, after determining the first probability that the noise-reducing frequency signal contains a music signal, noise reduction processing can be performed on the noise-reducing frequency signal based on the first probability and the amplitude spectrum of the noise-reducing frequency signal to be denoised, thereby obtaining the noise-reduced frequency signal corresponding to the noise-reducing frequency signal. For details on how to obtain the noise-reduced frequency signal corresponding to the noise-reducing frequency signal, please refer to subsequent embodiments; details will not be elaborated here.

[0096] The audio noise reduction method provided in this disclosure involves acquiring a noise-generating frequency signal, performing time-frequency conversion on the noise-generating frequency signal to obtain a noise-generating frequency domain signal, extracting acoustic features of a preset dimension from the noise-generating frequency domain signal, the acoustic features including at least one of signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features, determining a first probability that the noise-generating frequency signal contains music based on the acoustic features, and performing noise reduction processing on the noise-generating frequency domain signal based on the first probability and the amplitude spectrum of the noise-generating frequency domain signal to obtain a noise-generated frequency signal corresponding to the noise-generating frequency signal. Since this disclosure determines the first probability that the noise-generating frequency signal contains music based on the acoustic features extracted from the noise-generating frequency domain signal, and uses the first probability to suppress noise in the noise-generating frequency signal, this disclosure can suppress noise without damaging the music quality, achieving lossless processing of the music signal and providing users with a high-fidelity, ultimate music quality experience.

[0097] Figure 7 This is a flowchart illustrating an audio noise reduction method according to another embodiment of this disclosure. Based on the above embodiments, this disclosure further describes the audio noise reduction method. Figure 7 As shown, the method in this embodiment of the disclosure may include:

[0098] S701, Obtain the noise frequency signal to be reduced.

[0099] For a detailed description of this step, please refer to [link / reference]. Figure 6 The relevant description of S601 in the illustrated embodiment will not be repeated here.

[0100] S702. Perform time-frequency conversion processing on the noise-to-frequency signal to be denoised to obtain the noise-to-frequency domain signal.

[0101] For a detailed description of this step, please refer to [link / reference]. Figure 6 The relevant description of S602 in the illustrated embodiment will not be repeated here.

[0102] S703. Extract acoustic features of the frequency domain signal to be denoised in a preset dimension. The acoustic features include at least one of the following: signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features.

[0103] For a detailed description of this step, please refer to [link / reference]. Figure 6 The relevant description of S603 in the illustrated embodiment will not be repeated here.

[0104] In this embodiment of the disclosure, Figure 6 Step S604 may further include the following step S704:

[0105] S704. Input the acoustic features into the music detection model to determine the first probability that the noise frequency signal to be reduced contains a music signal.

[0106] In this step, after extracting the acoustic features of the frequency domain signal to be denoised in a preset dimension, the acoustic features can be input into the music detection model to determine the first probability that the frequency domain signal to be denoised contains a music signal.

[0107] Optionally, the music detection model includes a fully connected layer, a first GRU layer, and a second GRU layer. Inputting acoustic features into the music detection model to determine the first probability that the noise frequency signal to be reduced contains a music signal can include: inputting acoustic features into the fully connected layer to obtain a first feature of a first dimension, where the first dimension is greater than a preset dimension; inputting the first feature into the first GRU layer to obtain a second feature of a second dimension, where the second dimension is greater than the first dimension; inputting the second feature into the second GRU layer to obtain a target feature; and determining the first probability that the noise frequency signal to be reduced contains a music signal based on the target feature and an activation function.

[0108] For example, Figure 8 This is a schematic diagram of a music detection model provided in an embodiment of the present disclosure, as shown below. Figure 8 As shown, assuming a preset dimension of 60, the 60-dimensional acoustic features are input into the fully connected layer of the music detection model to obtain the first feature of the first dimension, which is then a feature of a higher dimension. The first feature is then input into the first GRU layer to obtain the second feature of the second dimension, which is also a feature of a higher dimension. The second feature is then input into the second GRU layer to obtain the target feature. An activation function is applied to the target feature to determine whether the noise frequency signal to be reduced contains a music signal. If it does, the first probability that the noise frequency signal to be reduced contains a music signal is output.

[0109] For details on how to train and obtain the music detection model, please refer to the subsequent implementation examples, which will not be repeated here.

[0110] In this embodiment of the disclosure, Figure 6 Step S605 may further include steps S705 to S713 as follows:

[0111] S705. Perform noise estimation on the noise signal in the amplitude spectrum of the frequency domain signal to be denoised to obtain the first noise spectrum.

[0112] For example, refer to Figure 1 The noise spectrum can be estimated by the noise spectrum estimation module to obtain the first noise spectrum.

[0113] S706. Based on the first probability, constrain the first noise spectrum to obtain the constrained noise spectrum.

[0114] In this step, after obtaining the first probability that the noise frequency signal to be reduced contains a music signal and the first noise spectrum, the first noise spectrum can be constrained based on the first probability to obtain the constrained noise spectrum.

[0115] Optionally, based on the first probability, the first noise spectrum is constrained to obtain a constrained noise spectrum, which may include obtaining the constrained noise spectrum according to the following formula:

[0116] D modify (k,l)=D estimated (k,l)*(1-MPP(l))+D estimated (k,l-1)*

[0117] MPP(l)

[0118] Among them, D modify (k,l) represents the noise spectrum after constraint processing; D estimated (k,l) represents the noise spectrum of the l-th frame at the k-th frequency point, i.e., the first noise spectrum; D estimated (k,l-1) represents the noise spectrum of the (l-1)th frame at the k-th frequency point; MPP(l) represents the first probability.

[0119] It can be understood that the MPP of the l-th frame is used to constrain the noise spectrum estimated at the k-th frequency point in the l-th frame, thereby obtaining the constrained noise spectrum D of the k-th frequency point in the l-th frame. modify (k,l). The physical meaning of the above formula is as follows: when the probability of music signal in the noise frequency signal to be reduced in the l-th frame is high, the constrained noise spectrum tends to use the noise spectrum estimated in the (l-1)-th frame, i.e., the previous frame, rather than the noise spectrum estimated in the current frame, thus constraining the update of the noise spectrum; when the probability of music signal in the noise frequency signal to be reduced in the l-th frame is low, the constrained noise spectrum tends to use the noise spectrum estimated in the l-th frame, i.e., the current frame, thus ensuring the update of the noise spectrum. The audio noise reduction method provided in this embodiment can suppress the noise component of the music signal without damaging the music component when processing noisy music signals, achieving the effect of high-fidelity processing of music signals.

[0120] After obtaining the noise spectrum after constraint processing, the noise reduction signal to be denoised can be processed based on the first probability and the noise spectrum after constraint processing to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal. Specifically, the following step S707 is executed.

[0121] S707. Based on the amplitude spectrum and the noise spectrum after constraint processing, obtain the second probability that the noise frequency signal to be reduced contains a speech signal.

[0122] For example, refer to Figure 1By inputting the amplitude spectrum and the constrained noise spectrum into the signal existence probability estimation module, the second probability that the noise frequency signal to be reduced contains a speech signal can be obtained.

[0123] S708. Obtain the signal-to-noise ratio based on the amplitude spectrum and the noise spectrum after constraint processing.

[0124] For example, refer to Figure 1 By inputting the amplitude spectrum and the constrained noise spectrum into the signal-to-noise ratio (SNR) calculation module, the SNR can be obtained.

[0125] It should be noted that the embodiments disclosed herein do not limit the order in which S707 and S708 are executed.

[0126] S709. Based on the second probability and the signal-to-noise ratio, update the noise spectrum after constraint processing to obtain the updated noise spectrum.

[0127] For example, refer to Figure 1 By combining the second probability and the signal-to-noise ratio, the noise spectrum after constraint processing is updated through the noise spectrum update module to obtain the updated noise spectrum.

[0128] S710. Based on the first probability, constrain the updated noise spectrum to obtain the second noise spectrum.

[0129] For example, according to the above formula, the updated noise spectrum is constrained to obtain a second noise spectrum, wherein the updated noise spectrum corresponds to D. estimated (k,l), the second noise spectrum corresponds to D modify (k,l).

[0130] S711. Obtain the Wiener value based on the second noise spectrum, the second probability, and the amplitude spectrum.

[0131] For example, refer to Figure 1 The Wiener value is obtained through the Wiener filtering calculation module based on the second noise spectrum, the second probability, and the amplitude spectrum.

[0132] S712. Based on the Wiener value, perform noise reduction processing on the frequency domain signal to be denoised to obtain the denoised frequency domain signal.

[0133] For example, refer to Figure 1 Applying the Wiener value to the frequency domain signal to be denoised yields an enhanced frequency domain signal, which is the denoised frequency domain signal.

[0134] S713. Perform time-frequency conversion processing on the noise reduction frequency domain signal to obtain the noise reduction frequency signal corresponding to the noise reduction frequency signal to be noise-reduced.

[0135] For example, refer to Figure 1The noise-reducing frequency domain signal is processed by the inverse short-time Fourier transform module to obtain the noise-reduced frequency signal corresponding to the noise-reducing frequency signal.

[0136] The audio noise reduction method provided in this embodiment acquires the noise frequency signal to be denoised, performs time-frequency conversion processing on the noise frequency signal to be denoised to obtain the noise frequency domain signal to be denoised; extracts acoustic features of a preset dimension of the noise frequency domain signal to be denoised, the acoustic features including at least one of signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features; inputs the acoustic features into a music detection model to determine a first probability that the noise frequency signal to be denoised contains a music signal; performs noise estimation on the noise signal in the amplitude spectrum of the noise frequency domain signal to be denoised to obtain a first noise spectrum; and performs constraint processing on the first noise spectrum based on the first probability to obtain constrained noise. The process involves: First, obtaining the second probability that the noise signal to be denoised exists in the frequency domain signal. Based on the amplitude spectrum and the constrained noise spectrum, the second probability is obtained. Simultaneously, the signal-to-noise ratio (SNR) is obtained based on the amplitude spectrum and the constrained noise spectrum. Then, the constrained noise spectrum is updated based on the second probability and the SNR to obtain the updated noise spectrum. Finally, the updated noise spectrum is constrained based on the first probability to obtain the second noise spectrum. The Wiener value is obtained based on the second noise spectrum, the second probability, and the amplitude spectrum. The Wiener value is then used to denoise the frequency domain signal to be denoised, resulting in the denoised frequency domain signal. Finally, the denoised frequency domain signal undergoes time-frequency conversion to obtain the denoised frequency signal corresponding to the noise-reduced frequency signal. Since this embodiment of the present disclosure determines the first probability that a music signal exists in the noise-generating frequency signal by using a music detection model based on the acoustic features extracted from the noise-generating frequency signal, and performs constraint processing on the first noise spectrum based on the first probability to obtain the constrained noise spectrum, and then performs constraint processing on the updated noise spectrum based on the first probability to obtain the second noise spectrum, and uses the second noise spectrum to suppress the noise in the noise-generating frequency signal, this embodiment of the present disclosure can suppress the noise without damaging the music quality when performing noise reduction processing on the noisy music signal, achieving the effect of lossless processing of the music signal and giving users a high-fidelity ultimate music quality experience.

[0137] Based on the above embodiments, optionally, the music detection model is trained in the following way: obtaining training samples, which include positive samples, negative samples and labels. Positive samples include music signals, negative samples include speech signals and noise signals, and labels are used to distinguish between positive and negative samples; based on the training samples, the music detection model is iteratively trained until the false recognition rate and false rejection rate of the music detection model are both less than the threshold, thus obtaining a trained music detection model.

[0138] For example, three types of signals can be collected: speech signals, music signals, and noise signals, to form training samples. Positive samples include music signals, and negative samples include both speech and noise signals. The labels are a binary distribution [0, 1]. When the signal contains music signals, the label is 1, representing a positive sample; the label is 0, representing a negative sample. The training impairment function is, for example, cross-entropy. Based on the training samples, the music detection model is iteratively trained until both the false recognition rate and the false rejection rate are less than a threshold, resulting in a well-trained music detection model. The threshold is, for example, 2%. For example, the FAR and FRR of the music detection model can be obtained based on the validation set. When both FAR and FRR are less than 2%, the music detection model training converges.

[0139] Based on the above embodiments, Figure 9 A flowchart of an audio noise reduction method provided in another embodiment of this disclosure is shown below. Figure 9 As shown, with Figure 1 Compared to previous versions, a music detection module has been added. This module determines the first probability that a music signal exists in the noise frequency signal to be reduced, and then uses this first probability to constrain the noise spectrum estimation module and the noise spectrum update module. Specifically, the acquired noise frequency signal y(n) to be reduced is input into the short-time Fourier transform module, which converts y(n) into the noise-reducing frequency domain signal Y(k,l), and obtains the amplitude spectrum |Y(k,l)| of Y(k,l). The music detection module extracts acoustic features of Y(k,l) in a preset dimension, and inputs these features into the music detection model to determine the first probability that a music signal exists in the noise frequency signal to be reduced. The noise spectrum estimation module estimates the noise signal in |Y(k,l)| to obtain the noise spectrum |D(k,l)|. Based on the first probability, |D(k,l)| is constrained to obtain the constrained noise spectrum. Based on |Y(k,l)| and the constrained noise spectrum, the signal existence probability estimation module obtains the second probability that a speech signal exists in the noise frequency signal to be reduced. Simultaneously, based on |Y(k,l)| and the constrained noise spectrum, the signal-to-noise ratio (SNR) is obtained through the SNR calculation module, which includes both prior and posterior SNR. Based on the second probability and the SNR, the constrained noise spectrum is updated using the noise spectrum update module to obtain the updated noise spectrum. Based on the first probability, the updated noise spectrum is constrained to obtain the second noise spectrum. Based on the second noise spectrum, the second probability, and the amplitude spectrum, the Wiener value is obtained through the Wiener filtering calculation module. The Wiener value is used to denoise Y(k,l) to obtain the denoised frequency domain signal. The inverse short-time Fourier transform module performs time-frequency conversion on the denoised frequency domain signal to obtain the denoised frequency signal corresponding to the original denoised frequency signal.

[0140] Based on the above embodiments, Figure 10This is a schematic diagram illustrating the effect of an audio noise reduction method provided in an embodiment of the present disclosure, such as... Figure 10 As shown, 1001 represents the acquired noisy music signal, and 1002 represents the music signal after noise suppression using the audio noise reduction method provided in this embodiment. The horizontal axis represents time, and the vertical axis represents the signal amplitude (Amp). 10011 represents the music signal, 10012 represents noise, 10021 represents the music signal, and 10022 represents noise. Comparing 1001 and 1002, the amplitude value of 10022 is smaller than that of 10012, and the amplitude value of 10021 remains essentially unchanged compared to 10011. Therefore, it can be determined that applying the audio noise reduction method provided in this embodiment to the noisy music signal can suppress noise while simultaneously... Figure 3 , Figure 4 The effects of the two related technologies present a stark contrast.

[0141] Exemplary device

[0142] Having introduced the medium of exemplary embodiments of this disclosure, the following references are made to... Figure 11 An audio noise reduction apparatus according to an exemplary embodiment of this disclosure will be described. The apparatus of the exemplary embodiment of this disclosure can implement the various processes in the foregoing audio noise reduction method embodiments and achieve the same function and effect.

[0143] Figure 11 This is a schematic diagram of the structure of an audio noise reduction device provided in an embodiment of the present disclosure, as shown below. Figure 11 As shown, the audio noise reduction device 1100 of this embodiment includes: an acquisition module 1101, a conversion module 1102, an extraction module 1103, a determination module 1104, and a processing module 1105. Wherein:

[0144] The acquisition module 1101 is used to acquire the noise frequency signal to be reduced.

[0145] The conversion module 1102 is used to perform time-frequency conversion processing on the noise-reducing frequency signal to obtain the noise-reducing frequency domain signal.

[0146] The extraction module 1103 is used to extract acoustic features of a preset dimension of the frequency domain signal to be denoised. The acoustic features include at least one of the following: signal Barker domain amplitude features, signal pitch features, first-order and second-order pitch difference features, and signal flatness features.

[0147] The determination module 1104 is used to determine the first probability that the noise frequency signal to be reduced contains a music signal based on acoustic characteristics.

[0148] The processing module 1105 is used to perform noise reduction processing on the frequency domain signal to be denoised based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, so as to obtain the noise-reduced frequency signal corresponding to the frequency domain signal to be denoised.

[0149] In one possible implementation, the processing module 1105 may be specifically used to: perform noise estimation on the noise signal in the amplitude spectrum to obtain a first noise spectrum; perform constraint processing on the first noise spectrum based on a first probability to obtain a constrained noise spectrum; and perform noise reduction processing on the frequency domain signal to be denoised based on the first probability and the constrained noise spectrum to obtain a noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

[0150] In one possible implementation, when processing module 1105 performs constraint processing on the first noise spectrum based on a first probability to obtain a constraint-processed noise spectrum, it may specifically be used to: obtain the constraint-processed noise spectrum according to the following formula:

[0151] D modify (k,l)=D estimated (k,l)*(1-MPP(l))+D estimated (k,l-1)*

[0152] MPP(l)

[0153] Among them, D modify (k,l) represents the noise spectrum after constraint processing; D estimated (k,l) represents the noise spectrum of the l-th frame at the k-th frequency point, i.e., the first noise spectrum; D estimated (k,l-1) represents the noise spectrum of the (l-1)th frame at the k-th frequency point; MPP(l) represents the first probability.

[0154] In one possible implementation, when the processing module 1105 performs noise reduction processing on the frequency domain signal to be denoised based on the first probability and the noise spectrum after constraint processing to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal, it may specifically perform the following steps: obtaining a second probability that the noise-reduced frequency signal contains a speech signal based on the amplitude spectrum and the noise spectrum after constraint processing; obtaining a signal-to-noise ratio based on the amplitude spectrum and the noise spectrum after constraint processing; updating the noise spectrum after constraint processing based on the second probability and the signal-to-noise ratio to obtain an updated noise spectrum; performing constraint processing on the updated noise spectrum based on the first probability to obtain a second noise spectrum; obtaining a Wiener value based on the second noise spectrum, the second probability, and the amplitude spectrum; performing noise reduction processing on the frequency domain signal to be denoised based on the Wiener value to obtain a noise-reduced frequency domain signal; and performing time-frequency conversion processing on the noise-reduced frequency domain signal to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

[0155] In one possible implementation, the determining module 1104 may be specifically used to: input acoustic features into a music detection model to determine a first probability that the noise frequency signal to be reduced contains a music signal.

[0156] In one possible implementation, the music detection model includes a fully connected layer, a first GRU layer, and a second GRU layer. When determining the probability that a music signal exists in the noise frequency signal to be reduced, the determination module 1104 can specifically be used to: input acoustic features into the fully connected layer to obtain a first feature of a first dimension, where the first dimension is greater than a preset dimension; input the first feature into the first GRU layer to obtain a second feature of a second dimension, where the second dimension is greater than the first dimension; input the second feature into the second GRU layer to obtain a target feature; and determine the probability that a music signal exists in the noise frequency signal to be reduced based on the target feature and the activation function.

[0157] In one possible implementation, the audio noise reduction device 1100 may further include a training module 1106 for training a music detection model by: acquiring training samples, which include positive samples, negative samples, and labels. Positive samples include music signals, negative samples include speech signals and noise signals, and labels are used to distinguish between positive and negative samples. Based on the training samples, the music detection model is iteratively trained until the false recognition rate and false rejection rate of the music detection model are both less than a threshold, thereby obtaining a trained music detection model.

[0158] The apparatus of this disclosure can be used to execute the audio noise reduction method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0159] Exemplary media

[0160] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. Figure 12 The storage medium of the exemplary embodiments of this disclosure will be described.

[0161] Figure 12 This is a schematic diagram of a storage medium provided according to an embodiment of the present disclosure. (See reference...) Figure 12 As shown, the storage medium 1200 stores a program product for implementing the above-described method according to embodiments of the present disclosure. This program product may be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0162] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0163] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium.

[0164] Program code for performing the operations disclosed herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0165] Exemplary computing device

[0166] Having described the methods, media, and apparatus of exemplary embodiments of this disclosure, the following references... Figure 13 A computing device according to an exemplary embodiment of the present disclosure will be described.

[0167] Figure 13 The computing device 1300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0168] Figure 13 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure, as shown below. Figure 13As shown, the computing device 1300 is presented in the form of a general-purpose computing device. Components of the computing device 1300 may include, but are not limited to: at least one processing unit 1301, at least one storage unit 1302, and a bus 1303 connecting different system components (including the processing unit 1301 and the storage unit 1302). For example, the processing unit 1301 may specifically be a processor, and the storage unit 1302 stores computer execution instructions. The processing unit 1301 executes the computer execution instructions stored in the storage unit 1302 to implement the aforementioned audio noise reduction method.

[0169] Bus 1303 includes a data bus, a control bus, and an address bus.

[0170] Storage unit 1302 may include readable media in the form of volatile memory, such as random access memory (RAM) 13021 and / or cache memory 13022, and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 13023.

[0171] Storage unit 1302 may also include a program / utility 13025 having a set (at least one) of program modules 13024, such program modules 13024 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0172] The computing device 1300 can also communicate with one or more external devices 1304 (e.g., keyboard, pointing device, etc.). This communication can be performed via the input / output (I / O) interface 1305. Furthermore, the computing device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 1306. Figure 13 As shown, network adapter 1306 communicates with other modules of computing device 1300 via bus 1303. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0173] It should be noted that although several units / modules or sub-units / modules of the audio noise reduction device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0174] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0175] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. An audio noise reduction method, comprising: Acquire the noise frequency signal to be reduced; The noise-to-frequency signal is subjected to time-frequency conversion processing to obtain the noise-to-frequency domain signal. The acoustic features of the frequency domain signal to be denoised are extracted in a preset dimension. The acoustic features include at least one of signal pitch features, first-order and second-order pitch difference features, and signal flatness features. Based on the acoustic characteristics, a first probability is determined that the noise frequency signal to be reduced contains a music signal; Based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, the frequency domain signal to be denoised is denoised to obtain the denoised frequency signal corresponding to the frequency domain signal to be denoised. The step of performing noise reduction processing on the frequency domain signal to be denoised based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, to obtain the noise-reduced frequency signal corresponding to the frequency domain signal to be denoised, includes: Noise estimation is performed on the noise signal in the amplitude spectrum to obtain the first noise spectrum; Based on the first probability, the first noise spectrum is constrained according to the following formula to obtain the constrained noise spectrum: in, This represents the noise spectrum after the constraint processing; Indicates the first The frequency point Frame noise spectrum, i.e., the first noise spectrum; Indicates the first The frequency point Frame noise spectrum; Indicates the first probability; Based on the first probability and the noise spectrum after constraint processing, the noise reduction process is performed on the frequency domain signal to be denoised to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

2. The audio noise reduction method according to claim 1, wherein the step of performing noise reduction processing on the frequency domain signal to be denoised based on the first probability and the noise spectrum after constraint processing to obtain the noise-reduced frequency signal corresponding to the frequency signal to be denoised, includes: Based on the amplitude spectrum and the noise spectrum after constraint processing, obtain the second probability that the noise frequency signal to be reduced contains a speech signal; The signal-to-noise ratio is obtained based on the amplitude spectrum and the constrained noise spectrum; Based on the second probability and the signal-to-noise ratio, the noise spectrum after the constraint processing is updated to obtain the updated noise spectrum; Based on the first probability, the updated noise spectrum is constrained to obtain the second noise spectrum; The Wiener value is obtained based on the second noise spectrum, the second probability, and the amplitude spectrum; The noise reduction process is performed on the frequency domain signal to be denoised according to the Wiener value to obtain the noise-reduced frequency domain signal. The noise-reduced frequency domain signal is subjected to time-frequency conversion processing to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

3. The audio noise reduction method according to claim 1 or 2, wherein determining the first probability that the noise-to-audio signal to be reduced contains a music signal based on the acoustic characteristics includes: The acoustic features are input into the music detection model to determine the first probability that the noise frequency signal to be reduced contains a music signal.

4. The audio noise reduction method according to claim 3, wherein the music detection model comprises a fully connected layer, a first gated recurrent unit (GRU) layer, and a second GRU layer, and the step of inputting the acoustic features into the music detection model to determine the first probability that the noise-reducing audio signal contains a music signal comprises: The acoustic features are input into the fully connected layer to obtain a first feature of a first dimension, wherein the first dimension is greater than the preset dimension. The first feature is input into the first GRU layer to obtain a second feature with a second dimension, where the second dimension is greater than the first dimension. The second feature is input into the second GRU layer to obtain the target feature; Based on the target features and activation function, a first probability is determined that the noise frequency signal to be reduced contains a music signal.

5. The audio noise reduction method according to claim 3, wherein the music detection model is trained in the following manner: Acquire training samples, which include positive samples, negative samples, and labels. The positive samples include music signals, the negative samples include speech signals and noise signals, and the labels are used to distinguish between the positive samples and the negative samples. Based on the training samples, the music detection model is iteratively trained until both the false recognition rate and the rejection rate of the music detection model are less than the threshold, thus obtaining a well-trained music detection model.

6. An audio noise reduction device, comprising: The acquisition module is used to acquire the noise frequency signal to be reduced; The conversion module is used to perform time-frequency conversion processing on the noise-to-frequency signal to obtain the noise-to-frequency domain signal; The extraction module is used to extract acoustic features of a preset dimension from the frequency domain signal to be denoised, wherein the acoustic features include at least one of signal pitch features, first-order and second-order pitch difference features, and signal flatness features. The determining module is used to determine, based on the acoustic characteristics, a first probability that the noise frequency signal to be reduced contains a music signal; The processing module is used to perform noise reduction processing on the frequency domain signal to be denoised based on the first probability and the amplitude spectrum of the frequency domain signal to be denoised, so as to obtain the noise-reduced frequency signal corresponding to the frequency domain signal to be denoised. The processing module is specifically used for: Noise estimation is performed on the noise signal in the amplitude spectrum to obtain the first noise spectrum; Based on the first probability, the first noise spectrum is constrained to obtain the constrained noise spectrum. Based on the first probability and the noise spectrum after constraint processing, the noise reduction processing is performed on the frequency domain signal to be denoised to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal; When the processing module performs constraint processing on the first noise spectrum based on the first probability to obtain the constrained noise spectrum, it is specifically used for: The noise spectrum after constraint processing is obtained according to the following formula: in, This represents the noise spectrum after the constraint processing; Indicates the first The frequency point Frame noise spectrum, i.e., the first noise spectrum; Indicates the first The frequency point Frame noise spectrum; This represents the first probability.

7. The audio noise reduction device according to claim 6, wherein when the processing module performs noise reduction processing on the frequency domain signal to be denoised based on the first probability and the noise spectrum after constraint processing to obtain the noise-reduced frequency signal corresponding to the frequency signal to be denoised, it is specifically used for: Based on the amplitude spectrum and the noise spectrum after constraint processing, obtain the second probability that the noise frequency signal to be reduced contains a speech signal; The signal-to-noise ratio is obtained based on the amplitude spectrum and the constrained noise spectrum; Based on the second probability and the signal-to-noise ratio, the noise spectrum after the constraint processing is updated to obtain the updated noise spectrum; Based on the first probability, the updated noise spectrum is constrained to obtain the second noise spectrum; The Wiener value is obtained based on the second noise spectrum, the second probability, and the amplitude spectrum; The noise reduction process is performed on the frequency domain signal to be denoised according to the Wiener value to obtain the noise-reduced frequency domain signal. The noise-reduced frequency domain signal is subjected to time-frequency conversion processing to obtain the noise-reduced frequency signal corresponding to the noise-reduced frequency signal.

8. The audio noise reduction device according to claim 6 or 7, wherein the determining module is specifically used for: The acoustic features are input into the music detection model to determine the first probability that the noise frequency signal to be reduced contains a music signal.

9. The audio noise reduction device according to claim 8, wherein the music detection model comprises a fully connected layer, a first gated recurrent unit (GRU) layer, and a second GRU layer, and the determining module, when inputting the acoustic features into the music detection model to determine the first probability that the noise-reducing audio signal contains a music signal, is specifically used for: The acoustic features are input into the fully connected layer to obtain a first feature of a first dimension, wherein the first dimension is greater than the preset dimension. The first feature is input into the first GRU layer to obtain a second feature with a second dimension, where the second dimension is greater than the first dimension. The second feature is input into the second GRU layer to obtain the target feature; Based on the target features and activation function, a first probability is determined that the noise frequency signal to be reduced contains a music signal.

10. The audio noise reduction apparatus according to claim 8 further includes a training module for training the music detection model in the following manner: Acquire training samples, which include positive samples, negative samples, and labels. The positive samples include music signals, the negative samples include speech signals and noise signals, and the labels are used to distinguish between the positive samples and the negative samples. Based on the training samples, the music detection model is iteratively trained until both the false recognition rate and the rejection rate of the music detection model are less than the threshold, thus obtaining a well-trained music detection model.

11. A computing device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the audio noise reduction method as described in any one of claims 1 to 5.

12. A storage medium storing computer program instructions, which, when executed, implement the audio noise reduction method as described in any one of claims 1 to 5.

13. A computer program product comprising a computer program that, when executed, implements the audio noise reduction method as described in any one of claims 1 to 5.

Images