Audio noise reduction method, electronic equipment and storage medium

Abstract

The invention discloses an audio noise reduction method, electronic equipment and a storage medium. The audio noise reduction method comprises the steps that target audio features are acquired, the target audio features are obtained by performing feature extraction on target audio, and the target audio is audio to be subjected to noise reduction or reference audio related to the audio to be subjected to noise reduction; taking the target audio features as input of a scene detection model to obtain a scene type corresponding to the target audio; and selecting a noise reduction mode matched with the scene type corresponding to the target audio to process the audio to be subjected to noise reduction to obtain audio data after noise reduction. According to the scheme, the sound quality of the audio data after noise reduction can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio signal processing, and in particular to an audio noise reduction method, an electronic device, and a storage medium. Background Art

[0002] In online voice call scenarios, audio noise reduction (ANS, Audio Noise Suppression) and other processing are essential.

[0003] Traditional audio noise reduction takes noise stationarity as an assumption to remove steady-state noise in the environment. Noise reduction algorithms often set parameters based on the assumption that both the playback sound source and the near-end voice are speaking sounds. With the continuous enrichment of usage scenarios, audio signals include but are not limited to speaking sounds or non-speaking sound sources. For example, in the remote teaching scenario of music classes, the audio signal often includes non-speaking sound sources such as music sources. Among them, the spectral characteristics of music sources are more obviously different from those of speaking sounds, and the noise reduction algorithm is designed using the parameters of the sound source as speaking sounds. However, this kind of noise reduction algorithm designed only based on speaking sound parameters to reduce the noise of audio signals including music sources will cause the noise reduction algorithm to cause great sound quality damage to the music sound, resulting in low sound quality of the denoised audio data.

[0004] In view of the existing technical defects, how to provide a solution to improve the sound quality of the noise-reduced audio data is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The present application at least provides an audio noise reduction method, an electronic device, and a storage medium.

[0006] The present application provides an audio noise reduction method, comprising: obtaining target audio features, where the target audio features are obtained by extracting features of a target audio, and the target audio is the audio to be noise-reduced or a reference audio related to the audio to be noise-reduced; using the target audio features as an input of a scene detection model to obtain a scene type corresponding to the target audio; and selecting a noise reduction method that matches the scene type corresponding to the target audio to process the audio to be noise-reduced, to obtain noise-reduced audio data.

[0007] The present application provides an audio noise reduction device, including: an acquisition module, a scene detection module and a processing module; the acquisition module is used to acquire target audio features, the target audio features are obtained by feature extraction of target audio, and the target audio is the audio to be denoised or a reference audio related to the audio to be denoised; the scene detection module is used to use the target audio features as the input of a scene detection model to obtain a scene type corresponding to the target audio; the processing module is used to select a noise reduction method that matches the scene type corresponding to the target audio to process the audio to be denoised, and obtain denoised audio data.

[0008] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned audio noise reduction method.

[0009] The present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned audio noise reduction method is implemented.

[0010] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0013] Figure 1 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 1 ; Figure 2 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 2 ; Figure 3 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 3 ; Figure 4 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 4 ; Figure 5 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 5 ; Figure 6 It is a structural schematic diagram of an embodiment of the audio noise reduction device of the present application; Figure 7 It is a structural schematic diagram of an embodiment of the electronic device of the present application; Figure 8It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0014] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0015] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0016] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0017] The present application provides some audio noise reduction methods and audio noise reduction devices. The application scenarios of the audio noise reduction method include but are not limited to online conference scenarios, online teaching scenarios or double-talk voice call scenarios. The executor of the audio noise reduction method may be an audio noise reduction device or a conference device. For example, the audio noise reduction device may be arranged in a terminal device or a server or other processing device, wherein the terminal device may be a device for audio noise reduction, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, etc. In some possible implementations, the audio noise reduction method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0018] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 1 Specifically, the audio noise reduction method may include the following steps: Step S11: Acquire target audio features.

[0019] The target audio feature is obtained by extracting the feature of the target audio, where the target audio is the audio to be noise reduced or a reference audio related to the audio to be noise reduced.

[0020] The target audio may be relevant audio data for noise reduction. The audio to be reduced in noise may be audio data collected by an audio acquisition device. Exemplarily, the audio acquisition device may be a microphone of the current device. The current device may be a computer or mobile phone of the current user. The reference audio may be audio data received by the audio acquisition device from the peer device. The peer device may be a computer or mobile phone of the peer user. The audio data collected by the peer device is played by the audio playback device of the current device, so that the current device can collect the voice recorded by the current user to obtain the audio to be reduced in noise. The audio playback device of the current device may be a speaker of the current device. It is understandable that after the audio playback device of the current device plays the relevant audio, when the audio acquisition device of the current device collects the voice recorded by the current user, the noise of the environment in which the current user is located and the sound played by the audio playback device of the current device will also be collected by the audio acquisition device of the current device to obtain the audio to be reduced in noise. The denoised audio data obtained by denoising the audio to be reduced in noise can be sent to the peer device and used as the reference audio received by the peer device.

[0021] After acquiring the target audio, feature extraction is performed on the target audio to obtain target audio features corresponding to the target audio. Specifically, the target audio is input into a preset feature extraction network to obtain target audio features corresponding to the target audio. The functions performed by the preset feature extraction network may be to determine the Mel-frequency cepstral coefficients of the target audio, perform Fourier transform on the target audio, determine the autocorrelation function of the target audio, determine the time-frequency features of the target audio, and further determine audio-related features of the target audio according to the audio noise reduction requirements.

[0022] Step S12: Using the target audio feature as the input of the scene detection model to obtain the scene type corresponding to the target audio.

[0023] The scene detection model can be used to perform scene detection based on the input target audio features to determine the scene type corresponding to the target audio. Exemplarily, the model structure of the scene detection model includes but is not limited to a deep neural network, a convolutional neural network, a recurrent neural network, etc. The scene type corresponding to the target audio can indicate that the sound source corresponding to the target audio is a silent type or a sound type. The sound type can be a conventional type or a preset special type. The scene type includes but is not limited to a silent type, a conventional type, or a preset special type. The conventional type can be that the sound source of the target audio is only the user's voice. In the case of different audio noise reduction requirements, the preset special types are different. Exemplarily, in the application scenario of film and television post-production, environmental noise may be recorded during the shooting process so that the sound source of the target audio may include crowd sounds and traffic sounds. The preset special type can be that the sound source of the target audio includes crowd sounds and traffic sounds in addition to the voice. Therefore, it is necessary to set a professionally designed noise reduction configuration mode to obtain a noise reduction method to process the noise reduction audio. Exemplarily, in an online teaching scenario, the preset special type can be that the sound source of the target audio includes music sounds in addition to the voice. Therefore, it is necessary to set a professionally designed music configuration mode to obtain a noise reduction method to process the noise reduction audio.

[0024] It is understandable that in the case of different audio noise reduction requirements, different special types are preset, and the preset special types include several other types except that the sound source is speech. For example, the preset special types include but are not limited to crowd audio type, traffic audio type or music audio type.

[0025] Step S13: Select a noise reduction method that matches the scene type corresponding to the target audio to process the audio to be noise reduced, and obtain noise-reduced audio data.

[0026] The scene type includes but is not limited to a silent type, a regular type or a preset special type. The noise reduction method may include performing noise reduction processing on the audio to be noise reduced so that the audio data after the noise reduction has less noise interference than the audio to be noise reduced.

[0027] Different scene types have different noise reduction methods matched to the scene types. When the scene type is a silent type, the noise reduction method matched to the scene type may be no noise reduction processing or default noise reduction processing. The default noise reduction processing may be a default noise reduction of the sound source in the acquisition environment in the absence of speaking sounds. When the scene type is a conventional type, the noise reduction method matched to the scene type may be conventional noise reduction of audio data whose sound source only includes speaking sounds. When the scene type is any one of the preset special types, the noise reduction method matched to the scene type may be special noise reduction of crowd sounds, traffic sounds, and music sounds in the sound source. It can be understood that different noise reduction methods are matched to different scene types according to different audio noise reduction requirements.

[0028] The target audio may be a reference audio, an audio to be denoised, or an audio to be denoised after a predetermined process. The predetermined process may be an echo cancellation process on the audio to be denoised. Specifically, feature extraction is performed on the target audio to obtain target audio features corresponding to the target audio. The target audio features are used as inputs of a scene detection model to obtain a scene type corresponding to the target audio. A noise reduction method matching the scene type corresponding to the target audio is selected to process the audio to be denoised, and audio data after denoising is obtained. The following content may be referred to: In some application scenarios, when the target audio is a reference audio, feature extraction is performed on the reference audio to obtain target audio features corresponding to the reference audio. The target audio features corresponding to the reference audio are used as inputs of a scene detection model to obtain a scene type corresponding to the reference audio. A noise reduction method matching the scene type corresponding to the reference audio is selected to process the audio to be denoised, and audio data after denoising is obtained. Specifically, a noise reduction method matching the scene type corresponding to the reference audio is selected to perform preset processing on the audio to be denoised, and audio data after denoising is obtained. In other application scenarios, when the target audio is the audio to be de-noised, feature extraction processing is performed on the audio to be de-noised to obtain the target audio features corresponding to the audio to be de-noised. The target audio features corresponding to the audio to be de-noised are used as the input of the scene detection model to obtain the scene type corresponding to the audio to be de-noised. A noise reduction method matching the scene type corresponding to the audio to be de-noised is selected to perform preset processing on the audio to be de-noised to obtain the de-noised audio data. In some application scenarios, when the target audio is the audio to be de-noised after a predetermined processing, feature extraction processing is performed on the audio to be de-noised after a predetermined processing to obtain the target audio features corresponding to the audio to be de-noised after a predetermined processing. The target audio features corresponding to the audio to be de-noised after a predetermined processing are used as the input of the scene detection model to obtain the scene type corresponding to the audio to be de-noised. A noise reduction method matching the scene type corresponding to the audio to be de-noised is selected to perform preset processing on the audio to be de-noised to obtain the de-noised audio data. It can be understood that the above-mentioned preset processing can be echo cancellation processing and / or noise reduction processing. Among them, the preset processing can refer to only echo cancellation processing, only noise reduction processing, and simultaneous echo cancellation processing and noise reduction processing. Different preset processing methods are set according to the accuracy requirements of audio noise reduction, which will not be described here.

[0029] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0030] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 2 .

[0031] In some embodiments, before the above step S11, the audio noise reduction method may further include the following steps: Step S21: Determine the cepstral coefficients corresponding to the target audio. And use the cepstral coefficients corresponding to the target audio as the initial audio features corresponding to the target audio. Step S22: Fusion process the initial audio features and the historical audio features to obtain the target audio features of the target audio. The historical audio features are used to represent the cepstral coefficients corresponding to at least one historical target audio.

[0032] The cepstral coefficients may be Bark-frequency cepstral coefficients (BFCC). The cepstral coefficient features corresponding to the target audio are determined. The cepstral coefficient features are obtained by determining the cepstral coefficients corresponding to the target audio, and the cepstral coefficient features are used as the initial audio features corresponding to the target audio. In other application scenarios, the cepstral coefficients may also be Mel-scale Frequency Cepstral Coefficients (MFCC).

[0033] In some embodiments, the above step S21 may include the following steps: first, Fourier transform the target audio to obtain a frequency domain signal corresponding to the target audio. Then, frequency domain compression is performed on the frequency domain signal corresponding to the target audio to obtain number domain data corresponding to the target audio, and the number domain data is used to represent the subband coefficients of the target audio in the Bark domain. Subsequently, discrete cosine transform processing is performed on the number domain data corresponding to the target audio to obtain the cepstrum coefficients corresponding to the target audio.

[0034] The Fourier transform can be a short-time Fourier transform, a discrete Fourier transform, or a fast Fourier transform. This application takes the Fourier transform as a short-time Fourier transform as an example to convert the time domain signal of the target audio into a frequency domain signal. A triangular window is used to compress the signal spectrum of the frequency domain signal corresponding to the target audio. The triangular window is simply used as a bandpass filter on the Bark domain to divide each Bark domain subband to obtain the subband coefficient of the target audio in the Bark domain, and the subband coefficient of the target audio in the Bark domain is used as the number domain data corresponding to the target audio. The Bark domain is used to represent the Bark domain. The logarithmic domain data is subjected to a discrete cosine transform to obtain the cepstral coefficients in the log domain after logarithmic transformation.

[0035] The audio data is divided into frames. The target audio is equivalent to the audio data related to the current frame. The historical audio feature can be the cepstral coefficient corresponding to the historical target audio in the historical frame. Specifically, the target audio corresponding to the initial audio feature and the historical target audio corresponding to the historical audio feature are respectively the audio data of the historical frames adjacent to the current frame and the current frame.

[0036] The initial audio features and the historical audio features are fused to obtain the target audio features of the target audio, including: fusing the initial audio features with the historical audio features corresponding to the historical target audio of a preset number of adjacent historical frames.

[0037] Among them, the preset number of frames can be one frame, two frames, three frames, etc., which is not limited here. When the preset number of frames is one frame, the initial audio features and the historical audio features corresponding to the historical target audio of an adjacent historical frame are directly fused to obtain the target audio features of the target audio. Among them, the adjacent historical frame can be any historical frame before the current frame, or it can be a historical frame with the smallest time distance before the current frame. When the preset number of frames is multiple frames, the initial audio features and the historical audio features corresponding to the historical target audio of multiple adjacent historical frames are directly fused to obtain the target audio features of the target audio. Multiple adjacent historical frames can be any historical frames before the current frame, or they can be multiple historical frames that are closest to the current frame in chronological order.

[0038] Exemplarily, when the preset number of frames is two frames, and the adjacent multi-frame historical frames are the first two frames of the current frame, the historical audio features of the adjacent multi-frame historical frames include the first historical audio feature of the first frame of the historical frame and the second historical audio feature of the second frame of the historical frame, and the acquisition time of the target audio corresponding to the second historical audio feature is earlier than the acquisition time of the target audio corresponding to the first historical audio feature. The specific fusion processing step can be to directly fuse the initial audio feature with the first historical audio feature and the second historical audio feature to obtain the target audio feature of the target audio. It can be understood that the target audio feature of the current frame and the historical audio feature of the historical frame can both be the mapping values ​​corresponding to each feature to be calculated in the feature space.

[0039] In some embodiments, the historical audio features include a first historical audio feature and a second historical audio feature, and the acquisition time of the target audio corresponding to the second historical audio feature is earlier than the acquisition time of the target audio corresponding to the first historical audio feature. The above step S22 may include the following steps: First, the first-order differential coefficients are calculated for the initial audio feature and the first historical audio feature to obtain the first-order audio feature corresponding to the target audio. Then, the second-order differential coefficients are calculated for the initial audio feature, the first historical audio feature, and the second historical audio feature to obtain the second-order audio feature corresponding to the target audio. Subsequently, the initial audio feature, the first-order audio feature, and the second-order audio feature are used as the target audio feature of the target audio.

[0040] The first-order difference coefficient is used to represent the difference between the initial audio feature and the first historical audio feature. Specifically, the process of calculating the first-order difference coefficient may be to subtract the first historical audio feature from the initial audio feature to obtain the first-order audio feature corresponding to the target audio.

[0041] The second-order difference coefficient is used to represent the second-order audio feature corresponding to the target audio obtained by using the initial audio feature, the first historical audio feature and the second historical audio feature through a predetermined transformation. Specifically, the process of calculating the second-order difference coefficient can be to subtract twice the first historical audio feature from the second historical audio feature to obtain a difference as a difference feature. Then, the difference feature is summed with the initial audio feature to obtain the second-order audio feature corresponding to the target audio.

[0042] It can be considered that the target audio features constructed by the first-order audio features, the second-order audio features and the initial audio features not only have the initial audio features corresponding to the static feature parameters of a single current frame and the first-order audio features and the second-order audio features that more completely describe the feature relationship between frames, but can dynamically describe the dynamic feature parameters between adjacent frames. The first-order audio features and the second-order audio features are the slopes of the initial audio features corresponding to the target audio of the current frame relative to time, that is, the laws of change relative to time, so that the constructed target audio features have a combination of static feature parameters and dynamic feature parameters, thereby enhancing the description capability of the target audio features.

[0043] The process of extracting features from the reference audio to obtain target audio features corresponding to the reference audio can refer to the following content: determine the cepstral coefficients corresponding to the reference audio, and use the cepstral coefficients corresponding to the reference audio as the initial audio features corresponding to the reference audio. Specifically, perform Fourier transform on the reference audio to obtain a frequency domain signal corresponding to the reference audio; perform frequency domain compression on the frequency domain signal corresponding to the reference audio to obtain number domain data corresponding to the reference audio, and the number domain data is used to represent the subband coefficients of the reference audio in the Bark domain; perform discrete cosine transform processing on the number domain data corresponding to the reference audio to obtain the cepstral coefficients corresponding to the reference audio. In some application scenarios, the initial audio features corresponding to the reference audio and the historical audio features of the historical reference audio are fused to obtain the target audio features of the reference audio, and the historical audio features are used to represent the cepstral coefficients corresponding to at least one historical reference audio. Specifically, the historical audio features of the historical reference audio include a first historical audio feature and a second historical audio feature, the collection time of the reference audio corresponding to the second historical audio feature is earlier than the collection time of the reference audio corresponding to the first historical audio feature, and the initial audio features and the historical audio features are fused to obtain the target audio features of the reference audio, including: calculating the first-order difference coefficients of the initial audio features and the first historical audio features to obtain the first-order audio features corresponding to the reference audio; calculating the second-order difference coefficients of the initial audio features, the first historical audio features and the second historical audio features to obtain the second-order audio features corresponding to the reference audio; and using the initial audio features, the first-order audio features and the second-order audio features as the target audio features of the reference audio.

[0044] Exemplarily, the signal collected by the microphone and the remote signal are obtained: the audio to be reduced in noise collected by the microphone can be represented as d(n), and the reference audio played through the speaker at the remote end can be represented as x(n). The near-end signal and the remote signal are respectively converted from the time domain to the frequency domain. Specifically, the frame overlapping segmentation method is adopted, in which the overlap between frames is the frame shift, and the value of the frame shift here is half of the frame length. The reference audio is Fourier transformed to obtain the frequency domain signal corresponding to the reference audio: the short-time Fourier transform method is used to convert the time domain signals of the audio to be reduced in noise and the reference audio into frequency domain signals, and the frequency domain microphone collection signal of the current frame is obtained, that is, the frequency domain signal corresponding to the audio to be reduced in noise is recorded as , the current frame frequency domain remote signal, that is, the frequency domain signal corresponding to the reference audio is recorded as The frequency domain signal corresponding to the reference audio is compressed in the frequency domain to obtain the digital domain data corresponding to the reference audio. The method can be: using the frequency domain block adaptive filtering method, caching The frequency domain signal of the block reference audio is recorded as: Where T is used to represent transposition, N is used to represent the number of cached frames, and k is used to represent the frequency sequence number. Each block of reference audio is equivalent to each frame of reference audio.

[0045] The structure of the target object's ear happens to form a set of auditory filters. For a specific frequency band, there will be an increase in amplitude, but for other frequency bands, the amplitude will be suppressed by the filter, so that only the frequency band of a specific critical point can be significantly perceived by the target object's ear. That is to say, the critical frequency band is the bandwidth of the auditory filter generated by the target object's ear structure. Only when the signal is in this bandwidth can the target object's ear be significantly perceived. Since the target object's ear's frequency perception range is between 20Hz and 20KHz, and considering that the target object's ear auditory perception has a logarithmic characteristic, the critical bandwidth will increase with the increase of the center frequency. At present, 24 Bark domains have been scientifically divided. Considering that the audio sampling frequency of mainstream processing is 16KHz and the maximum useful frequency of speech is 8KHz, according to the division rule, the number of Bark subbands can be obtained as 22. Use a triangular window to compress the signal spectrum, and use the triangular window as a simple bandpass filter on the Bark domain to divide each Bark domain subband.

[0046] In some application scenarios, based on the theoretical weighted values ​​of all sub-bands at each frequency point and the frequency domain signal corresponding to the reference audio of the current frame, the sub-band coefficients of the reference audio of the current frame in the Bark domain are determined, and the sub-band coefficients of the reference audio of the current frame in the Bark domain are used as the digital domain data corresponding to the reference audio of the current frame. Exemplarily, the process of performing frequency domain compression on the frequency domain signal corresponding to the reference audio to obtain the digital domain data corresponding to the reference audio can refer to formula (1): Formula (1); In the above formula, Used to indicate the The subband is The theoretical weighted value of frequency points, and satisfy . It can be used to represent the frequency domain signal corresponding to the reference audio of the current frame, and specifically can represent the frequency domain coefficient of the reference audio of the current frame. It can be used to represent the digital domain data corresponding to the reference audio, and can represent the The energy or power of each sub-band.

[0047] In other application scenarios, discrete cosine transform (DCT) and Bark-frequency cepstral coefficients (BFCC) based on the Bark-domain spectrum in the log domain can be used as input features of the scene detection model to improve the discrimination of the input features. For example, the process of performing discrete cosine transform processing on the number domain data corresponding to the reference audio to obtain the cepstral coefficients corresponding to the reference audio can refer to formula (2): Formula (2); in, It can be used to represent the n-th dimension feature component of BFCC, that is, the cepstral coefficient corresponding to the reference audio of the current frame. It can be used to represent the extracted 16-dimensional BFCC features. M can represent the total number of dimensions of the Buck domain. m can represent the mth dimension of the Buck domain. N can represent the total dimension of the BFCC domain. n can represent the nth dimension of the BFCC domain. Can be used to represent the digital domain data corresponding to the reference audio.

[0048] It can be considered that in order to be able to perform scene detection on each frame of target audio in real time, a large number of features cannot be used for direct training, otherwise it will lead to excessive calculation, resulting in a long model inference time and failure to meet real-time requirements. The present application can further process the signal spectrum features to extract key information that is useful for determining whether it belongs to the above-mentioned preset special type, such as music classification, thereby reducing the amount of calculation of the scene detection model on the input features.

[0049] The process of calculating the first-order difference coefficients between the initial audio features and the first historical audio features to obtain the first-order audio features corresponding to the reference audio can refer to formula (3). The process of calculating the second-order difference coefficients between the initial audio features and the first historical audio features and the second historical audio features to obtain the second-order audio features corresponding to the reference audio can refer to formula (4): Formula (3); Formula (4); in, It can be used to represent the cepstral coefficients corresponding to the reference audio of the current frame. It can be used to represent the above-mentioned first historical audio feature. It can be used to represent the above-mentioned second historical audio feature. It can be used to represent the first-order audio features corresponding to the reference audio of the current frame. It can be used to represent the second-order audio features corresponding to the reference audio of the current frame. For example, the present application uses 16-dimensional BFCC features, which are the cepstral coefficients corresponding to the reference audio of the current frame, to calculate 8-dimensional BFCC first-order differences, which are the first-order audio features corresponding to the reference audio of the current frame, and 8-dimensional BFCC second-order differences, which are the second-order audio features corresponding to the reference audio of the current frame.

[0050] It is understandable that the target audio features corresponding to the target audio are obtained by feature extraction of the target audio, wherein the target audio may be reference audio, may be the audio to be noise reduced, or may be the audio to be noise reduced after a predetermined processing. It is understandable that the present application takes the process of extracting features from the reference audio to obtain the target audio features corresponding to the reference audio as an example, and the steps of extracting features from the audio to be noise reduced to obtain the target audio features corresponding to the audio to be noise reduced, and the steps of extracting features from the audio to be noise reduced after a predetermined processing to obtain the target audio features corresponding to the audio to be noise reduced after a predetermined processing may refer to the process of extracting features from the reference audio to obtain the target audio features corresponding to the reference audio, which will not be described in detail here.

[0051] In some embodiments, the above step S12 may include the following steps: first, input the target audio feature into the first linear layer for mapping processing to obtain the initial feature corresponding to the target audio. Then, the initial feature and the historical fusion feature are fused to obtain the fusion feature corresponding to the target audio. The historical fusion feature includes fusion features corresponding to several historical target audios. Subsequently, the fusion feature corresponding to the target audio is classified to obtain the scene type corresponding to the target audio.

[0052] The first linear layer is used to map the target audio features input by the scene detection model to obtain the initial features corresponding to the target audio. In other application scenarios, when the target audio features include multiple features, the first linear layer can also perform weighted fusion on the target audio features including the initial audio features, the first-order audio features, and the second-order audio features to obtain the initial features corresponding to the target audio.

[0053] The scene detection model also includes a feature fusion module. The feature fusion module is used to fuse the initial features and the historical fusion features to obtain the fusion features corresponding to the target audio output by the feature fusion module. The historical fusion features include the fusion features corresponding to the historical target audio of several historical frames. The historical target audio is used to represent the target audio of the historical frame. The fusion features corresponding to the historical target audio of each historical frame can be the fusion features corresponding to the historical target audio output by the feature fusion module after the target audio features corresponding to the historical target audio of each historical frame are processed in sequence by the first linear layer and the feature fusion module. It can be understood that the fusion features corresponding to the target audio output by the feature fusion module each time will be retained in the feature fusion module so that when the initial features corresponding to the next frame of target audio are input into the feature fusion module, the fusion features corresponding to the target audio of the current frame are used as historical fusion features and fused with the initial features corresponding to the target audio of the next frame to obtain the fusion features corresponding to the target audio of the next frame.

[0054] The several historical frames may be at least part of all historical frames before the current frame, or may be a preset number of adjacent historical frames. Exemplarily, 10 consecutive frames of audio data are cached. The target audio of the current frame may be the 10th frame. There may be 9 frames in all cached historical frames, and the preset number of adjacent historical frames may be any one or more frames of all cached historical frames, or may be any one or more consecutive frames that are closest to the 10th frame in time.

[0055] The classification process may be to directly input the fusion features corresponding to the target audio into a preset classification network to obtain the scene type corresponding to the target audio. The preset classification network is used to perform scene type classification processing on the fusion features corresponding to the target audio. Among them, the preset classification network may be a convolutional neural network, a fully connected layer, a classifier, etc., which can perform scene type classification processing on the fusion features. The scene types include but are not limited to silent type, regular type or preset special type.

[0056] It can be considered that the fusion features of the target audio constructed at this time can capture the temporal dependency between the initial features of the current frame and the historical fusion features of several historical frames, making the fusion features of the target audio richer and more effective, thereby directly using the fusion features of the target audio for classification processing, thereby improving the accuracy of the scene type corresponding to the target audio obtained after classification processing.

[0057] In some embodiments, the above-mentioned step of classifying the fusion features corresponding to the target audio to obtain the scene type corresponding to the target audio may include the following steps: First, the fusion features corresponding to the target audio are input into the second linear layer and the third linear layer for linear transformation processing respectively, and the feature vector output by the second linear layer and the feature vector output by the third linear layer are obtained. Then, the feature vector output by the second linear layer and the feature vector output by the third linear layer are normalized respectively to obtain a first mapping value and a second mapping value. Subsequently, the first mapping value is compared with the first threshold to determine a first classification result, and the first classification result is used to indicate that the target audio is a silent signal or a sound signal. And, the second mapping value is compared with the second threshold to determine a second classification result, and the second classification result is used to indicate that the target audio is a music signal or a non-music signal. Then, based on the first classification result and the second classification result, the scene type corresponding to the target audio is determined, wherein the scene type corresponding to the target audio includes a silent scene, a music scene or a non-music scene.

[0058] The second linear layer performs a linear transformation on the fusion features of the target audio to obtain a feature vector output by the second linear layer. The third linear layer performs a linear transformation on the fusion features of the target audio to obtain a feature vector output by the third linear layer. The second linear layer and the third linear layer have different classification tasks. The classification task corresponding to the second linear layer is a binary classification task of the target audio being a sound signal and a silent signal. The classification task corresponding to the third linear layer is a binary classification task of the target audio being a music signal and a non-music signal. A predetermined activation function is applied to normalize the feature vector output by the second linear layer and the feature vector output by the third linear layer, respectively, to obtain a first mapping value and a second mapping value. The predetermined activation function may be a Sigmoid function or a Softmax function, and the setting method of the predetermined activation function is not limited here.

[0059] The first threshold and the second threshold are both preset threshold values. Different first thresholds and second thresholds are set according to different audio noise reduction requirements. When the first mapping value is less than the first threshold, the first classification result is that the target audio is a silent signal. When the first mapping value is greater than or equal to the first threshold, the first classification result is that the target audio is a sound signal. In some application scenarios, inputting the target audio into the second linear layer and the third linear layer can be performed in parallel and synchronously, or in series. When the target audio is input into the second linear layer and the third linear layer in series, the order in which the target audio is input into the second linear layer and the third linear layer is not limited. If the target audio is first input into the second linear layer and then into the third linear layer, if the first classification result is that the target audio is a silent signal, it is not necessary to input the target audio into the third linear layer to obtain the second classification result, and the scene type of the target audio output by the scene detection model is directly determined as a silent scene. When the second mapping value is less than the second threshold, the second classification result is a non-music signal. When the second mapping value is greater than or equal to the second threshold, the second classification result is a music signal. The scene type corresponding to the target audio is determined by combining the first classification result and the second classification result, wherein the scene type corresponding to the target audio includes a silent scene, a music scene or a non-music scene.

[0060] Combining the first classification result and the second classification result can be: in some application scenarios, when the first classification result is that the target audio is a silent signal, regardless of whether the second classification result is that the target audio is a music signal or a non-music signal, the scene type corresponding to the target audio is a silent scene. Among them, the silent scene can be the above-mentioned silent type. In other application scenarios, when the first classification result is that the target audio is a sound signal, if the second classification result is that the target audio is a non-music signal, the scene type corresponding to the target audio is determined to be a non-music scene. Among them, the non-music scene can be represented as the above-mentioned conventional type. In other application scenarios, when the first classification result is that the target audio is a sound signal, if the second classification result is that the target audio is a music signal, the scene type corresponding to the target audio is determined to be a music scene. Among them, the non-music scene can be represented as a music type in the above-mentioned preset special type.

[0061] Exemplarily, in the case where the target audio is the reference audio, the target audio features corresponding to the reference audio are used as the input of the scene detection model, and the step of obtaining the scene type corresponding to the reference audio may be: inputting the reference audio features corresponding to the reference audio into the first linear layer for mapping processing to obtain the initial features corresponding to the reference audio; fusing the initial features corresponding to the reference audio and the historical fusion features to obtain the fusion features corresponding to the reference audio, the historical fusion features include the fusion features corresponding to several historical reference audios; classifying the fusion features corresponding to the reference audio to obtain the scene type corresponding to the reference audio. Specifically, based on the first classification result corresponding to the reference audio and the second classification result corresponding to the reference audio, the scene type corresponding to the reference audio is determined, wherein the first classification result corresponding to the reference audio is that the reference audio is a silent signal or a sound signal. The second classification result corresponding to the reference audio is that the reference audio is a music signal or a non-music signal. The scene types corresponding to the reference audio include the reference audio being a silent scene, a music scene, or a non-music scene.

[0062] Exemplarily, the scene detection model structure may be: the first linear layer may be a Dense Tanh linear layer. The feature fusion module may be a GRU layer in the scene detection model. The preset classification network may include a second linear layer and a third linear layer. The second linear layer and the third linear layer may be represented as Dense sig linear layers, respectively.

[0063] It is understandable that the target audio features of the target audio are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, wherein the target audio can be the reference audio, can be the audio to be denoised, or can be the audio to be denoised after a predetermined processing. It is understandable that the present application takes the process of using the target audio features of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio as an example, and the step of using the target audio features of the audio to be denoised as the input of the scene detection model to obtain the scene type corresponding to the audio to be denoised, and the step of using the target audio features of the audio to be denoised after a predetermined processing as the input of the scene detection model to obtain the scene type corresponding to the audio to be denoised can refer to the above-mentioned process of using the target audio features of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio, which will not be repeated here.

[0064] See also Figure 3 , Figure 3 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 3 .

[0065] In some embodiments, the reference audio includes audio data received by an audio collection device from a peer device, the audio to be noise-reduced is collected by the audio collection device, and the audio data after noise reduction includes audio data after echo cancellation.

[0066] The above step S12 may include the following steps: Step S31: taking the target audio feature of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio. The noise reduction method matching the scene type corresponding to the reference audio includes an echo cancellation method.

[0067] The echo cancellation mode is used to perform echo cancellation processing on the audio to be de-noised so as to eliminate the echo and reverberation in the audio to be de-noised, thereby improving the quality of voice and audio transmission. Different echo cancellation modes are corresponding to different scene types.

[0068] The target audio features of the reference audio are used as inputs of the scene detection model to obtain the scene type corresponding to the reference audio. The scene type corresponding to the current frame reference audio is one of a silent type, a regular type, or a preset special type. Exemplarily, the scene type corresponding to the current frame reference audio is one of a silent scene, a music scene, or a non-music scene. After obtaining the scene type corresponding to the reference audio, an echo cancellation method matching the scene type corresponding to the reference audio is obtained.

[0069] In some embodiments, the above step S13 may include the following steps: Step S32: Perform echo cancellation processing on the audio to be noise-reduced according to the echo cancellation method to obtain audio data after echo cancellation.

[0070] According to the echo cancellation method matching the scene type corresponding to the reference audio, the audio to be reduced in noise is processed by echo cancellation to obtain audio data after echo cancellation. It can be understood that the audio to be reduced in noise after the predetermined processing can be the audio data after echo cancellation.

[0071] Exemplarily, the scene type corresponding to the reference audio may be one of a silent scene, a music scene, or a non-music scene. The echo cancellation method that matches the scene type corresponding to the reference audio may be A1, B1, or C1. When the scene type corresponding to the reference audio is a silent scene, the echo cancellation method that matches the scene type corresponding to the reference audio is A1. When the scene type corresponding to the reference audio is a non-music scene, the echo cancellation method that matches the scene type corresponding to the reference audio is B1. When the scene type corresponding to the reference audio is a music scene, the echo cancellation method that matches the scene type corresponding to the reference audio is C1. Exemplarily, A1 may represent Processing, the current speaker does not play sound, and the echo cancellation algorithm does not process the microphone acquisition signal. When the echo cancellation mode is A1, this echo cancellation mode can be called PASS MODE. For example, B1 can represent The speaker plays a non-music signal. This mode uses the general configuration of echo cancellation. When the echo cancellation mode is B1, this echo cancellation mode can be called Normal MODE. For example, C1 can represent Processing, the speaker plays music signals. If the echo cancellation method of Normal MODE is used in this mode, the echo signal will leak. This mode needs to adopt a professionally designed music configuration mode for echo cancellation. When the echo cancellation method is C1, this echo cancellation method can be called MusicMODE.

[0072] In some application scenarios, the audio data after echo cancellation is directly used as the noise reduction data corresponding to the audio to be reduced. In other application scenarios, the audio data after echo cancellation is subjected to noise reduction processing to obtain target noise reduction data. The target noise reduction data is used as the noise reduction data corresponding to the audio to be reduced.

[0073] See also Figure 4 , Figure 4 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 4 .

[0074] In some embodiments, the audio data after echo elimination is used as the audio to be processed, and the audio data after noise reduction also includes the target noise reduction data obtained by performing noise reduction processing on the audio to be processed. After the above step S32, the audio noise reduction method may also include the following steps: Step S41: Extract features of the audio to be processed to obtain the target audio features of the audio to be processed. Step S42: Use the target audio features of the audio to be processed as the input of the scene detection model to obtain the scene type corresponding to the audio to be processed. Step S43: Select a noise reduction method that matches the scene type corresponding to the audio to be processed to perform noise reduction processing on the audio to be processed to obtain the target noise reduction data.

[0075] For step S41, reference can be made to the above-mentioned process of extracting features from the reference audio to obtain the target audio features of the reference audio. In some application scenarios, the cepstral coefficients corresponding to the audio to be processed are determined, and the cepstral coefficients corresponding to the audio to be processed are used as the initial audio features corresponding to the audio to be processed. Specifically, the audio to be processed is subjected to Fourier transform to obtain the frequency domain signal corresponding to the audio to be processed; the frequency domain signal corresponding to the audio to be processed is subjected to frequency domain compression to obtain the number domain data corresponding to the audio to be processed, and the number domain data is used to represent the subband coefficients of the audio to be processed in the Bark domain; the number domain data corresponding to the audio to be processed is subjected to discrete cosine transform processing to obtain the cepstral coefficients corresponding to the audio to be processed. In some application scenarios, the initial audio features corresponding to the audio to be processed and the historical audio features of the historical audio to be processed are fused to obtain the target audio features of the audio to be processed, and the historical audio features are used to represent the cepstral coefficients corresponding to at least one historical audio to be processed. Specifically, the historical audio features of the historical audio to be processed include a first historical audio feature and a second historical audio feature, the collection time of the audio to be processed corresponding to the second historical audio feature is earlier than the collection time of the audio to be processed corresponding to the first historical audio feature, the initial audio feature and the historical audio feature are fused to obtain the target audio features of the audio to be processed, including: calculating the first-order difference coefficients of the initial audio feature and the first historical audio feature to obtain the first-order audio features corresponding to the audio to be processed; calculating the second-order difference coefficients of the initial audio feature, the first historical audio feature and the second historical audio feature to obtain the second-order audio features corresponding to the audio to be processed; and using the initial audio feature, the first-order audio feature and the second-order audio feature as the target audio features of the audio to be processed.

[0076] The above step S42 can refer to the above step of using the target audio features of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio. Exemplarily, in the case where the target audio is the audio to be processed, the target audio features corresponding to the audio to be processed are used as the input of the scene detection model, and the step of obtaining the scene type corresponding to the audio to be processed can be: inputting the audio features corresponding to the audio to be processed into the first linear layer for mapping processing to obtain the initial features corresponding to the audio to be processed; fusing the initial features corresponding to the audio to be processed and the historical fusion features to obtain the fusion features corresponding to the audio to be processed, and the historical fusion features include the fusion features corresponding to several historical audio to be processed; classifying the fusion features corresponding to the audio to be processed to obtain the scene type corresponding to the audio to be processed. Specifically, based on the first classification result corresponding to the audio to be processed and the second classification result corresponding to the audio to be processed, the scene type corresponding to the audio to be processed is determined, wherein the first classification result corresponding to the audio to be processed is that the audio to be processed is a silent signal or a sound signal. The second classification result corresponding to the audio to be processed is that the audio to be processed is a music signal or a non-music signal. The scene types corresponding to the audio to be processed include the audio to be processed being a silent scene, a music scene or a non-music scene. It is understandable that the scene type corresponding to the audio to be processed can be used to represent the scene type corresponding to the audio to be reduced in noise. The scene type corresponding to the audio to be processed is directly used as the scene type corresponding to the audio to be reduced in noise.

[0077] Exemplarily, the scene type corresponding to the audio to be processed may be one of a silent scene, a music scene or a non-music scene. The noise reduction method that matches the scene type corresponding to the audio to be processed may be A2, B2, or C2. When the scene type corresponding to the audio to be processed is a silent scene, the noise reduction method that matches the scene type corresponding to the audio to be processed is A2. When the scene type corresponding to the audio to be processed is a non-music scene, the noise reduction method that matches the scene type corresponding to the audio to be processed is B2. When the scene type corresponding to the audio to be processed is a music scene, the noise reduction method that matches the scene type corresponding to the audio to be processed is C2. Exemplarily, A2 may indicate that no noise reduction processing is performed on the audio to be processed or that default noise reduction processing is performed. Exemplarily, B2 may indicate that This noise reduction method is a general noise reduction mode, which tends to keep the voice signal as much as possible and remove the steady-state noise in the environment. For example, C2 can represent This noise reduction method is the music noise reduction mode, which tends to ensure the continuity of sounds such as music and voice as much as possible.

[0078] It can be considered that the target noise reduction data obtained by performing noise reduction processing on the audio to be processed in a noise reduction method matching the scene type corresponding to the audio to be processed is subjected to echo cancellation processing and noise reduction processing adapted to the scene type, so that the target noise reduction data has higher sound quality.

[0079] In some embodiments, the audio noise reduction method also includes a training step for a scene detection model, and the training step includes: first, obtaining a sample target audio and a sample scene type corresponding to the sample target audio, and the sample scene type is obtained by pre-marking. Then, the sample target audio feature is used as the input of the scene detection model to obtain the predicted scene type corresponding to the sample target audio. The predicted scene type includes a first predicted classification result and a second predicted classification result. The sample target audio feature is obtained by feature extraction of the sample target audio. The first predicted classification result is used to indicate that the sample target audio is a silent signal or a sound signal, and the second predicted classification result is used to indicate that the sample target audio is a music signal or a non-music signal. Subsequently, based on the difference between the first predicted classification result and the second predicted classification result and the sample scene type, a target loss is obtained. Then, using the target loss, the parameters in the scene detection model are adjusted to obtain a trained scene detection model.

[0080] The sample target audio may be training data corresponding to the relevant audio data for noise reduction. The sample target audio is the sample audio to be noise reduced or the sample reference audio related to the sample audio to be noise reduced. The sample scene type corresponding to the sample target audio is the scene type obtained by pre-marking. The sample scene type corresponding to the sample target audio includes but is not limited to a silent type, a regular type or a preset special type. Specifically, the sample scene type corresponding to the sample target audio includes but is not limited to a silent scene, a non-music scene or a music scene.

[0081] The sample target audio may be the sample audio to be noise reduced and / or the sample reference audio related to the sample audio to be noise reduced. The step of extracting features from the sample target audio to obtain the sample target audio features may refer to the above-mentioned process of extracting features from the reference audio to obtain the target audio features of the reference audio, which will not be repeated here. The step of using the sample target audio features as the input of the scene detection model to obtain the predicted scene type corresponding to the sample target audio may refer to the above-mentioned step of using the target audio features of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio, which will not be repeated here.

[0082] Among them, the target loss is determined based on the first loss and the second loss. Specifically, the first loss is determined based on the difference between the first predicted classification result and the first sample classification result. The second loss is determined based on the difference between the second predicted classification result and the second sample classification result. The first loss and the second loss are weighted summed to obtain the target loss. Exemplarily, the cross entropy function is used as the loss function in the process of determining the first loss and determining the second loss, respectively. The process of determining the first loss can refer to formula (5), the process of determining the second loss can refer to formula (6), and the process of weighted summing the first loss and the second loss to obtain the target loss can refer to formula (7): Formula (5); Formula (6); Formula (7); in, and The first loss and the second loss can be represented respectively. is the number of samples, that is, the number of sample target audios. There can be several sample target audios. and Respectively The true label of the sample scene type of the sample target audio, where the true label can be (0 or 1). It can be used to represent the true label related to the first sample classification result, used to represent a sound signal or a silent signal. It can be used to represent the true label associated with the second sample classification result, used to represent a music signal or a non-music signal. and Can be expressed separately The first sample mapping value corresponding to the first predicted classification result of the sample target audio and the second sample mapping value corresponding to the second predicted classification result. and The weighted coefficients of the first loss and the second loss can be respectively represented. It can be considered that, when the sample target audio is sufficient data, the parameters in the scene detection model are trained using the determined target loss until the scene detection model converges to obtain a trained scene detection model. The above step S12 is performed using the trained scene detection model, and the target audio features are input into the trained scene detection model to obtain the scene type of the target audio.

[0083] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of the audio noise reduction method of the present application. Figure 5 .

[0084] Specifically, the audio noise reduction method may include the following steps: Execute step S51, step S52, step S53, step S54 and step S55 in sequence. Step S51: Use the target audio features of the reference audio as the input of the scene detection model to obtain the scene type corresponding to the reference audio. The noise reduction method that matches the scene type corresponding to the reference audio includes an echo cancellation method. Step S52: According to the echo cancellation method, perform echo cancellation processing on the audio to be noise-reduced to obtain audio data after echo cancellation. Step S53: Perform feature extraction on the audio to be processed to obtain the target audio features of the audio to be processed. Among them, the audio to be processed is the audio data after the above-mentioned echo cancellation. Step S54: Use the target audio features of the audio to be processed as the input of the scene detection model to obtain the scene type corresponding to the audio to be processed. Step S55: Select a noise reduction method that matches the scene type corresponding to the audio to be processed to perform noise reduction processing on the audio to be processed to obtain target noise reduction data.

[0085] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0086] See also Figure 6 , Figure 6 It is a structural diagram of an embodiment of an audio noise reduction device of the present application. The audio noise reduction device 60 includes an acquisition module 61, a scene detection module 62 and a processing module 63; the acquisition module 61 is used to acquire target audio features, the target audio features are obtained by feature extraction of the target audio, the target audio is the audio to be noise-reduced or a reference audio related to the audio to be noise-reduced; the scene detection module 62 is used to use the target audio features as the input of the scene detection model to obtain the scene type corresponding to the target audio; the processing module 63 is used to select a noise reduction method that matches the scene type corresponding to the target audio to process the audio to be noise-reduced, and obtain the denoised audio data.

[0087] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0088] For the functions performed by each module, please refer to the audio noise reduction method, which will not be repeated here.

[0089] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 70 includes a memory 71 and a processor 72, and the processor 72 is used to execute program instructions stored in the memory 71 to implement the steps in the above-mentioned audio noise reduction method embodiment. In a specific implementation scenario, the electronic device 70 may include but is not limited to: a multi-camera device, a microcomputer, and a server. In addition, the electronic device 70 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0090] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in the above-mentioned audio noise reduction method embodiment. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by an integrated circuit chip.

[0091] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0092] See also Figure 8 , Figure 8 The computer-readable storage medium 80 stores program instructions 801, which implement the steps in any of the above-mentioned audio noise reduction method embodiments when executed by a processor.

[0093] The above scheme obtains the target audio features. The target audio features are obtained by feature extraction of the target audio. The target audio is the audio to be denoised or a reference audio related to the audio to be denoised. The target audio features obtained by feature extraction are used as the input of the scene detection model to obtain the scene type corresponding to the target audio, so that the determined scene type of the target audio is more accurate, and the noise reduction method matched by the scene type of the target audio is a noise reduction method that is more suitable for the audio to be denoised. Then, the noise-reduced audio data obtained by processing the audio to be denoised by a noise reduction method that is more suitable for the audio to be denoised can improve the sound quality of the noise-reduced audio data.

[0094] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0095] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0096] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0097] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An audio noise reduction method, characterized in that: The method comprises: Acquire a target audio feature, where the target audio feature is obtained by extracting a feature of the target audio, where the target audio is the audio to be de-noised or a reference audio related to the audio to be de-noised; Using the target audio feature as an input of a scene detection model to obtain a scene type corresponding to the target audio; A noise reduction method matching the scene type corresponding to the target audio is selected to process the audio to be noise reduced, so as to obtain audio data after noise reduction.

2. The method according to claim 1, characterized in that The step of using the target audio feature as an input of a scene detection model to obtain a scene type corresponding to the target audio includes: Inputting the target audio feature into the first linear layer for mapping processing to obtain the initial feature corresponding to the target audio; The initial features and the historical fusion features are fused to obtain fusion features corresponding to the target audio, wherein the historical fusion features include fusion features corresponding to a plurality of historical target audios; Classify the fusion features corresponding to the target audio to obtain the scene type corresponding to the target audio.

3. The method according to claim 2, characterized in that The classifying process of the fusion features corresponding to the target audio to obtain the scene type corresponding to the target audio includes: Inputting the fusion features corresponding to the target audio into the second linear layer and the third linear layer for linear transformation processing respectively, to obtain a feature vector output by the second linear layer and a feature vector output by the third linear layer; Normalizing the feature vector output by the second linear layer and the feature vector output by the third linear layer respectively to obtain a first mapping value and a second mapping value; Comparing the first mapping value with a first threshold value to determine a first classification result, wherein the first classification result is used to indicate that the target audio is a silent signal or a sound signal; and comparing the second mapping value with a second threshold value to determine a second classification result, wherein the second classification result is used to indicate that the target audio is a music signal or a non-music signal; Based on the first classification result and the second classification result, a scene type corresponding to the target audio is determined, wherein the scene type corresponding to the target audio includes a silent scene, a music scene, or a non-music scene.

4. The method according to any one of claims 1 to 3, characterized in that Before acquiring the target audio feature, the method further includes: Determine the cepstral coefficients corresponding to the target audio, and use the cepstral coefficients corresponding to the target audio as initial audio features corresponding to the target audio; The initial audio features and the historical audio features are fused to obtain target audio features of the target audio, where the historical audio features are used to represent cepstral coefficients corresponding to at least one historical target audio.

5. The method according to claim 4, characterized in that The determining of the cepstral coefficients corresponding to the target audio includes: Performing Fourier transform on the target audio to obtain a frequency domain signal corresponding to the target audio; Performing frequency domain compression on a frequency domain signal corresponding to the target audio to obtain digital domain data corresponding to the target audio, wherein the digital domain data is used to represent a subband coefficient of the target audio in a Bark domain; Discrete cosine transform processing is performed on the number domain data corresponding to the target audio to obtain the cepstral coefficients corresponding to the target audio.

6. The method according to claim 4, characterized in that The historical audio feature includes a first historical audio feature and a second historical audio feature, and a target audio corresponding to the second historical audio feature is collected earlier than a target audio corresponding to the first historical audio feature. The fusing the initial audio features and the historical audio features to obtain the target audio features of the target audio includes: Calculating a first-order difference coefficient between the initial audio feature and the first historical audio feature to obtain a first-order audio feature corresponding to the target audio; Calculating a second-order difference coefficient for the initial audio feature, the first historical audio feature, and the second historical audio feature to obtain a second-order audio feature corresponding to the target audio; The initial audio feature, the first-order audio feature, and the second-order audio feature are used as target audio features of the target audio.

7. The method according to any one of claims 1 to 3, characterized in that The reference audio includes audio data received by an audio acquisition device from a peer device, the audio to be denoised is collected by the audio acquisition device, and the denoised audio data includes audio data after echo cancellation. The step of using the target audio feature as an input of a scene detection model to obtain a scene type corresponding to the target audio includes: Using the target audio feature of the reference audio as an input of the scene detection model to obtain a scene type corresponding to the reference audio, wherein the noise reduction method matching the scene type corresponding to the reference audio includes an echo cancellation method; The selecting a noise reduction method that matches the scene type corresponding to the target audio to process the audio to be noise reduced to obtain the audio data after noise reduction includes: According to the echo cancellation method, the audio to be reduced in noise is subjected to echo cancellation processing to obtain the audio data after echo cancellation.

8. The method according to claim 7, characterized in that The audio data after echo cancellation is used as the audio to be processed, and the audio data after noise reduction also includes target noise reduction data obtained by performing noise reduction processing on the audio to be processed. After performing echo cancellation processing on the audio to be noise reduced according to the echo cancellation method to obtain the audio data after echo cancellation, the method further includes: Extracting features of the audio to be processed to obtain target audio features of the audio to be processed; Using the target audio feature of the audio to be processed as the input of the scene detection model to obtain the scene type corresponding to the audio to be processed; A noise reduction method matching the scene type corresponding to the audio to be processed is selected to perform noise reduction processing on the audio to be processed, so as to obtain the target noise reduction data.

9. The method according to any one of claims 1 to 3, characterized in that: The method further comprises a step of training the scene detection model, wherein the training step comprises: Acquire sample target audio and a sample scene type corresponding to the sample target audio, wherein the sample scene type is obtained by pre-marking; Using the sample target audio feature as the input of the scene detection model, obtaining a predicted scene type corresponding to the sample target audio, the predicted scene type including a first predicted classification result and a second predicted classification result, the sample target audio feature is obtained by extracting features from the sample target audio, the first predicted classification result is used to indicate that the sample target audio is a silent signal or a sound signal, and the second predicted classification result is used to indicate that the sample target audio is a music signal or a non-music signal; Obtaining a target loss based on the first prediction classification result and a difference between the second prediction classification result and the sample scene type; The target loss is used to adjust the parameters in the scene detection model to obtain a trained scene detection model.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 9.

11. A computer-readable storage medium having program instructions stored thereon, characterized in that: include: A program file is stored, and when the program file is executed by a processor, it is used to implement the method according to any one of claims 1 to 9.

Images