A noise cancellation method and apparatus in an audio signal, a device, and a medium

Abstract

The application provides a noise cancellation method and device in an audio signal, equipment and a medium, which are used for effectively canceling residual noise in the audio signal, reducing the calculation amount and avoiding distortion of voice information. In the embodiment of the application, a buffer vector is arranged, and the value corresponding to the audio frame in the buffer vector is updated according to whether the audio frame is a voice frame, so that the vector mean of the updated buffer vector is obtained, the Mask vector is adjusted by using the vector mean, and the voice vector corresponding to the enhanced audio frame is obtained according to the adjusted Mask vector. Because the process of identifying the audio frame does not need to increase the model, the calculation amount is reduced, and the voice signal is not damaged additionally when the noise is cancelled, so that the accuracy of the processed audio signal is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio signal processing, and in particular to a noise elimination method and device in an audio signal, an equipment and a medium. BACKGROUND

[0002] Noise elimination and echo cancellation are important technical points in the field of audio signal processing, and the purpose is to eliminate noise and echo in an audio signal containing speech. The technical solutions adopted by the prior art in noise elimination include:

[0003] 1. Using a noise elimination algorithm to process each audio frame without determining whether each audio frame is a speech frame, and directly taking the processed audio frame as the final result. This scheme will cause residual noise in the processed audio frame because it does not determine the speech frame.

[0004] 2. Using a noise elimination algorithm to process an audio frame to obtain a mask (Mask) vector of the audio frame, determining whether the audio frame is a speech frame according to a Mask average value of the Mask vector and a preset Mask threshold, and setting the non-speech frame to zero if it is determined that the audio frame is a non-speech frame. This scheme can eliminate residual noise, but because the non-speech frame is set to zero, some speech frames also have the risk of being set to zero, resulting in distortion of speech information.

[0005] 3. Using a noise elimination algorithm to process an audio frame to obtain a Mask vector of the audio frame, and then inputting the Mask vector into a neural network model to determine whether the audio frame is a speech frame, and setting the non-speech frame to zero if it is determined that the audio frame is a non-speech frame. This scheme uses a neural network model to determine whether the audio frame is a speech frame, which not only increases the amount of calculation, but also causes distortion of speech information because the non-speech frame is set to zero. SUMMARY

[0006] The present application provides a noise elimination method and device in an audio signal, an equipment and a medium, which solves the problems of residual noise, large amount of calculation and distortion of speech information caused by processing of the prior art on the audio signal.

[0007] In a first aspect, the present application provides a noise elimination method in an audio signal, which comprises:

[0008] For each audio frame in the audio signal, a Mask vector corresponding to the audio frame is determined based on a Mask noise reduction algorithm; whether the audio frame is a speech frame is determined according to each value of the Mask vector and a preset threshold value; a value corresponding to the audio frame in a current buffer vector is updated according to whether the audio frame is a speech frame; a vector mean of the buffer vector is determined according to each value in the updated buffer vector; an adjusted Mask vector is determined according to the vector mean and the Mask vector; and a speech vector corresponding to the enhanced audio frame is determined according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame.

[0009] In a second aspect, the present application provides a noise elimination device in an audio signal, the device comprising:

[0010] A determination module is configured to determine, for each audio frame in the audio signal, a Mask vector corresponding to the audio frame based on a Mask noise reduction algorithm.

[0011] A judgment module is configured to determine whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value.

[0012] The determination module is further configured to update a value corresponding to the audio frame in a current buffer vector according to whether the audio frame is a speech frame, to determine a vector mean of the buffer vector according to each value in the updated buffer vector, to determine an adjusted Mask vector according to the vector mean and the Mask vector, and to determine a speech vector corresponding to the enhanced audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame.

[0013] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the processor is configured to implement the steps of the noise elimination method in an audio signal according to any one of the above aspects when executing a computer program stored in the memory.

[0014] In a fourth aspect, the present application provides a computer storage medium storing a computer program executable by an electronic device, wherein the program, when executed on the electronic device, causes the electronic device to perform the steps of the noise elimination method in an audio signal according to any one of the above aspects.

[0015] The application provides a noise elimination method, device and equipment in an audio signal and a medium. The method comprises the following steps: determining a Mask vector corresponding to each audio frame in the audio signal based on a Mask noise reduction algorithm; determining whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value; updating a value corresponding to the audio frame in a current buffer vector according to whether the audio frame is a speech frame; determining a vector mean of the buffer vector according to each value in the updated buffer vector; determining an adjusted Mask vector according to the vector mean and the Mask vector; and determining an enhanced speech vector corresponding to the audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame. In the embodiment of the application, a buffer vector is provided, and the value corresponding to the audio frame in the buffer vector is updated according to whether the audio frame is a speech frame, so that the vector mean of the updated buffer vector can be obtained, the Mask vector is adjusted by using the vector mean, and the enhanced speech vector corresponding to the audio frame can be obtained according to the adjusted Mask vector. Because the process of identifying the audio frame does not need to increase a model, the calculation amount is reduced, and the audio signal is not damaged additionally when the noise is eliminated, so that the accuracy of the processed audio signal is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the application or the implementation manners in the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art according to these drawings.

[0017] Figure 1 A noise elimination process schematic diagram in an audio signal provided by some embodiments of the application;

[0018] Figure 2 A noise elimination process schematic diagram in an audio signal based on a Mask average value provided by some embodiments of the application;

[0019] Figure 3 A noise elimination process schematic diagram in an audio signal based on an exponential weighted average value provided by some embodiments of the application;

[0020] Figure 4 A comparison schematic diagram of audio signals processed by using the prior art and the method provided by some embodiments of the application;

[0021] Figure 5 A comparison schematic diagram of audio signals processed by using another prior art and the method provided by some embodiments of the application;

[0022] Figure 6 Some embodiments of this application are provided based on Figure 5 The diagram shown is a comparison of audio signals processed using two existing technologies and the method provided in the embodiments of this application.

[0023] Figure 7 Schematic diagrams of audio signal processing provided for some embodiments of this application;

[0024] Figure 8 A schematic diagram comparing noise in an audio signal processed by another prior art and the method provided in the embodiments of this application is provided for some embodiments of this application;

[0025] Figure 9 A comparative diagram showing the noise signals contained in audio signals processed by the noise reduction algorithm and the method provided in the embodiments of this application, provided for some embodiments of this application;

[0026] Figure 10 Some embodiments of this application are provided based on Figure 8 The diagram shown is a comparison of noise in audio signals processed by two existing technologies and the methods provided in the embodiments of this application.

[0027] Figure 11 A schematic diagram of the structure of a noise cancellation device in an audio signal is provided for some embodiments of this application;

[0028] Figure 12 A schematic diagram of an electronic device provided for some embodiments of this application. Detailed Implementation

[0029] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0030] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0031] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0032] The terms "comprises", "comprising", "includes", "including", "has", "having", "contains" and "containing" are intended to be open-ended terms that do not preclude additional components, steps, or elements.

[0033] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software codes that can perform the function related to the element.

[0034] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, but not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to part or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0035] For the convenience of explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. Various modifications and variations can be derived according to the above teachings. The selection and description of the above embodiments are for better explanation of the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

[0036] The present application provides a noise cancellation method, device and equipment in an audio signal and a medium. The method comprises: for each audio frame in the audio signal, determining a Mask vector corresponding to the audio frame based on a Mask noise reduction algorithm; determining whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value; updating a value corresponding to the audio frame in a current buffer vector according to whether the audio frame is a speech frame; determining a vector mean of the buffer vector according to each value in the updated buffer vector; determining an adjusted Mask vector according to the vector mean and the Mask vector; and determining an enhanced speech vector corresponding to the audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame.

[0037] In order to effectively eliminate residual noise in an audio signal, reduce the amount of calculation, and avoid distortion of speech information, the present application provides a noise cancellation method, device and equipment in an audio signal and a medium.

[0038] Embodiment 1:

[0039] Figure 1A noise elimination process in an audio signal is provided for some embodiments of the present application, and the process comprises the following steps:

[0040] S101: For each audio frame in the audio signal, a Mask vector corresponding to the audio frame is determined based on a Mask noise reduction algorithm.

[0041] The audio information processing method provided by the embodiments of the present application is applied to an electronic device, which can be a device containing a microphone, such as a mobile phone, a smart television, a smart speaker, etc.

[0042] The audio signal collected by the electronic device is generally an audio signal with residual noise. In the embodiments of the present application, in order to eliminate the residual noise in the audio signal, the audio signal is first divided into multiple audio frames, and noise elimination processing is performed on each audio frame.

[0043] The field used by the noise elimination method of the present application includes but is not limited to noise elimination and echo cancellation. If used for noise elimination, the residual noise after processing by a noise reduction algorithm can be eliminated, and if used for echo cancellation, the residual echo after processing by an echo cancellation algorithm can be eliminated.

[0044] For each audio frame in the audio signal, a Mask vector corresponding to the audio frame is determined based on a Mask noise reduction algorithm. In the embodiments of the present application, the Mask noise reduction algorithm can be any Mask-based noise reduction or echo cancellation algorithm. Each dimension of the Mask vector is usually a value between 0 and 1, and the value tends to 0, i.e. the signal corresponding to the audio frame is suppressed, which realizes noise reduction.

[0045] S102: According to each value of the Mask vector and a preset threshold, it is determined whether the audio frame is a speech frame.

[0046] In the embodiments of the present application, after determining the Mask vector of each audio frame, for the Mask vector of each audio frame, the Mask vector contains f dimensions, i.e. there are f values, and each value is between 0 and 1. According to each value contained in the Mask vector, the value with the highest occurrence frequency in the Mask vector can be identified, and according to the value and a preset threshold, it can be determined whether the audio frame is a speech frame. Specifically, if the value is not less than the preset threshold, it is determined that the audio frame is a speech frame, otherwise, it is determined that the audio frame is a non-speech frame.

[0047] S103: updating the value corresponding to the audio frame in the current buffer vector according to whether the audio frame is a speech frame, determining a vector mean of the buffer vector according to each value in the updated buffer vector, determining an adjusted Mask vector according to the vector mean and the Mask vector, and determining an enhanced speech vector corresponding to the audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame.

[0048] In order to avoid the audio signal being suddenly truncated at the speech cutoff, affect the hearing, make the change of the audio frame from the speech frame to the non-speech frame more natural, and avoid the loss of speech information caused by the truncation of the non-speech frame, a buffer vector is provided in the embodiment of the present application, the buffer vector is a vector with a dimension of n, the size of n can determine the slow degree from the speech frame to the non-speech frame, and the length of the buffer vector can be flexibly set according to the needs.

[0049] In the embodiment of the present application, each value of the buffer vector is initialized to an initial value before the buffer vector is used to process the Mask vector of the audio frame, the initial value is a value between 0 and 1, for example, the initial value of each value of the buffer vector can be set to 1.

[0050] In order to update the buffer vector, after it is determined whether the audio frame is a speech frame or a non-speech frame, if the audio frame is a speech frame, the value corresponding to the audio frame can be the same as the initial value of each value in the buffer vector, if the audio frame is a non-speech frame, the value corresponding to the audio frame can be 0. According to the value corresponding to the audio frame, the value corresponding to the audio frame in the buffer vector is updated. Specifically, the first position of the buffer vector can be the position of the value corresponding to each audio frame, and the value corresponding to the audio frame is updated in the buffer vector using the value corresponding to the audio frame.

[0051] After the buffer vector is updated, according to each value in the updated buffer vector, according to the cumulative sum of each value and the length n of the buffer vector, the vector mean of the buffer vector can be determined, and the Mask vector is adjusted according to the vector mean and the Mask vector, that is, each value in the adjusted Mask vector is determined using the vector mean and each value in the Mask vector, and the adjusted Mask vector is obtained.

[0052] After the adjusted Mask vector is obtained, a speech vector corresponding to the enhanced audio frame can be determined according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame, where the frequency domain vector corresponding to the audio frame is a frequency domain vector obtained by performing a short-time Fourier transform on the audio frame, the length of the frequency domain vector is consistent with the length of the Mask vector, and the frequency domain vector is also f-dimensional, where f can be 128, 256, or the like. Specifically, the frequency domain can be a complex spectrum, an amplitude spectrum, a power spectrum, or the like.

[0053] If the first bit of the buffer vector can be a position of a value corresponding to each audio frame, that is, the first bit is a to-be-updated bit, after the value corresponding to the to-be-updated bit in the buffer vector is updated, the values corresponding to each bit in the buffer vector can be moved one bit backward, the value at the last bit is removed, and the value at the first bit of the buffer vector is an initial value, to facilitate updating the value corresponding to the to-be-updated bit of the subsequent audio frame.

[0054] In the embodiment of the present application, a buffer vector is provided, and the value corresponding to the audio frame in the buffer vector can be updated according to whether the audio frame is a speech frame, so that the vector mean of the updated buffer vector can be obtained, the Mask vector is adjusted using the vector mean, so that the speech vector corresponding to the enhanced audio frame can be obtained according to the adjusted Mask vector. Because the audio frame recognition process does not need to increase the model, the calculation amount is reduced, and at the same time, the noise is eliminated without causing additional damage to the speech signal, the accuracy of the processed audio signal is ensured.

[0055] Embodiment 2

[0056] To accurately determine whether the audio frame is a speech frame, on the basis of the above-mentioned embodiments, in the embodiment of the present application, the determining whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold comprises:

[0057] calculating a Mask average value corresponding to the audio frame according to each value of the Mask vector;

[0058] determining whether the Mask average value is not less than a preset first threshold;

[0059] If yes, it is determined that the audio frame is a speech frame, otherwise, it is determined that the audio frame is a non-speech frame.

[0060] The Mask vector corresponding to the audio frame is determined through the Mask noise reduction algorithm. Whether the audio frame is a speech frame can be determined according to the values in the Mask vector. Specifically, the lower the value in the Mask vector, the more likely the audio frame does not contain speech signals that need to be preserved, that is, the audio frame can be a noise frame that can be completely set to zero, that is, the audio frame is likely to be a non-speech frame. Correspondingly, the higher the value in the Mask vector, the more likely the audio frame is a speech frame containing speech signals that need to be preserved, that is, the audio frame is likely to be a speech frame.

[0061] Therefore, in the embodiments of the present application, in order to accurately determine whether each audio frame is a speech frame, the cumulative sum of the values in the Mask vector corresponding to the audio frame can be determined according to the values in the Mask vector corresponding to the audio frame. The Mask average of the Mask vector can be determined according to the cumulative sum and the length of the Mask vector. Specifically, the Mask average can be calculated according to the following formula:

[0062]

[0063] wherein mean(Mask (t,f) ) is the Mask average of the Mask vector corresponding to the audio frame t, Mask(t, k) is the kth value in the Mask vector corresponding to the audio frame t, wherein k is an integer from 1 to f, and f is the length of the Mask vector. The audio frame t is the tth audio frame.

[0064] After the Mask average corresponding to the audio frame is determined, it is judged whether the Mask average is not less than a preset first threshold. If the Mask average corresponding to the audio frame is not less than the preset first threshold, the audio frame is determined to be a speech frame. Otherwise, the audio frame is determined to be a non-speech frame.

[0065] Figure 2 The process diagram for removing noise in an audio signal based on the Mask average provided by some embodiments of the present application is shown in FIG. 1, which includes the following steps: Figure 2

[0066] S201: For each audio frame in the audio signal, the Mask vector corresponding to the audio frame is determined based on the Mask noise reduction algorithm.

[0067] S202: The Mask average corresponding to the audio frame is calculated according to each value in the Mask vector.

[0068] S203: It is judged whether the Mask average is not less than a preset first threshold. If yes, the audio frame is determined to be a speech frame. If no, the audio frame is determined to be a non-speech frame. ​

[0069] S204: updating the value corresponding to the audio frame in the current buffer vector according to whether the audio frame is a speech frame.

[0070] Embodiment 3:

[0071] In order to effectively eliminate the residual noise in the audio signal, reduce the amount of calculation, and avoid distortion of speech information, on the basis of the above embodiment, in the embodiment of the present application, the step of determining whether the audio frame is a speech frame according to each value of the Mask vector and the preset threshold value comprises:

[0072] calculating a Mask average value corresponding to the audio frame according to each value of the Mask vector;

[0073] determining an exponentially weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and the Mask average values corresponding to other audio frames before the audio frame;

[0074] determining whether the exponentially weighted average value is not less than a preset second threshold value;

[0075] if yes, determining that the audio frame is a speech frame, otherwise, determining that the audio frame is a non-speech frame.

[0076] In the embodiment of the present application, in order to accurately determine whether each audio frame is a speech frame, the determination of whether the audio frame is a speech frame can be made according to the exponentially weighted average value corresponding to the audio frame.

[0077] Specifically, the determination process of the Mask average value corresponding to the audio frame has been described in the above embodiment, which will not be repeated here. The exponentially weighted average value corresponding to the audio frame is determined according to the Mask average value corresponding to the audio frame and the Mask average values corresponding to other audio frames before the audio frame. For example, the exponentially weighted average value corresponding to the audio frame is determined according to the Mask average value corresponding to the audio frame and the Mask average values corresponding to other audio frames before the audio frame and adjacent to the audio frame. For example, the average of the Mask average value corresponding to the audio frame and the Mask average values corresponding to other audio frames can be calculated, and the average is taken as the exponentially weighted average value corresponding to the audio frame.

[0078] After obtaining the exponentially weighted average value corresponding to the audio frame, it is determined whether the exponentially weighted average value corresponding to the audio frame is not less than a preset second threshold value. If yes, it is determined that the audio frame is a speech frame, otherwise, it is determined that the audio frame is a non-speech frame.

[0079] Of course, in order to further ensure the accuracy of speech frame recognition, in the embodiment of the present application, the determination of whether the audio frame is a speech frame can also be made comprehensively according to the Mask average value corresponding to the audio frame and the exponentially weighted average value corresponding to the audio frame.

[0080] Specifically, after determining the Mask average value and the exponentially weighted average value corresponding to the audio frame, it is determined whether the Mask average value is not less than a preset first threshold value and the exponentially weighted average value corresponding to the audio frame is not less than a preset second threshold value. If both are no, it is determined that the audio frame is a non-speech frame, otherwise, it is determined that the audio frame is a speech frame. That is, as long as any one of the Mask average value and the exponentially weighted average value corresponding to the audio frame is not less than the corresponding preset threshold value, it is determined that the audio frame is a speech frame.

[0081] Figure 3 The process diagram of noise elimination in an audio signal based on an exponentially weighted average value provided for some embodiments of the present application is shown in FIG. 1, which includes the following steps: Figure 3

[0082] S301: For each audio frame in the audio signal, a Mask vector corresponding to the audio frame is determined based on a Mask noise reduction algorithm.

[0083] S302: A Mask average value corresponding to the audio frame is calculated according to each value of the Mask vector.

[0084] S303: An exponentially weighted average value corresponding to the audio frame is determined according to the Mask average value corresponding to the audio frame and the Mask average values corresponding to other audio frames before the audio frame.

[0085] S304: It is determined whether the exponentially weighted average value is not less than a preset second threshold value. If yes, it is determined that the audio frame is a speech frame, if no, it is determined that the audio frame is a non-speech frame.

[0086] S305: The value corresponding to the audio frame in the current buffer vector is updated according to whether the audio frame is a speech frame.

[0087] The determination of whether the audio frame is a speech frame can be realized based on the following manner:

[0088] First, the initial value of the value of each bit in the buffer vector is 1. For each audio frame t, if the Mask average value mean_t of the Mask vector corresponding to the audio frame is less than a preset first threshold value th_1 and the exponentially weighted average value moving_mean_t corresponding to the audio frame is less than a preset second threshold value th_2, it is determined that the audio frame is a non-speech frame, the value corresponding to the audio frame is 0, otherwise, it is determined that the audio frame is a speech frame, the value corresponding to the audio frame is 1; the value corresponding to the audio frame in the buffer vector is updated according to the value corresponding to the audio frame, and the adjusted Mask vector is determined according to the product of the Mask vector mask_t corresponding to the audio frame and the vector mean value np.mean(win) of the updated buffer vector.​

[0089] Embodiment 4:

[0090] To further accurately determine whether each audio frame is a speech frame, on the basis of the above embodiments, in the embodiment of the present application, the determining the exponentially weighted average value corresponding to the audio frame according to the average value of the Mask corresponding to the audio frame and the average values of the Masks corresponding to other audio frames before the audio frame comprises:

[0091] determining the exponentially weighted average value corresponding to the audio frame according to the average value of the Mask corresponding to the audio frame and a preset first weight value, and the average value of the Mask corresponding to the previous audio frame before the audio frame and a preset second weight value; or

[0092] determining an average value according to the average value of the Mask corresponding to the audio frame and the average values of the Masks corresponding to a preset number of other audio frames before the audio frame, and taking the average value as the exponentially weighted average value corresponding to the audio frame.

[0093] The exponentially weighted average value corresponding to each audio frame is determined according to the average value of the Mask corresponding to the audio frame and the average values of the Masks corresponding to other audio frames before the audio frame. In the embodiment of the present application, when determining the exponentially weighted average value corresponding to the audio frame, the exponentially weighted average value corresponding to the audio frame can be determined according to the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to the previous audio frame before the audio frame. Specifically, a first weight corresponding to the average value of the Mask corresponding to the audio frame can be set, and a second weight corresponding to the average value of the Mask corresponding to the previous audio frame before the audio frame can be set. According to the two average values and the respective weights, the exponentially weighted average value corresponding to the audio frame can be determined, wherein the sum of the first weight and the second weight is 1.

[0094] In the embodiment of the present application, when the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to the previous audio frame before the audio frame are obtained, the exponentially weighted average value corresponding to the audio frame can be determined according to the following formula:

[0095] moving_mean_t = ((s-1) x mean(Mask (t-1,f) (t-1)) + mean(Mask (t,f) )) / s,

[0096] wherein moving_mean_t is the exponentially weighted average value corresponding to the audio frame t, mean(Mask (t-1,f) (t-1)) is the average value of the Mask corresponding to the previous audio frame (t-1) before the audio frame t, and mean(Mask (t,f)) is the average value of the Mask corresponding to the audio frame t, wherein s is a set value, for example, the s can be 20, 30, 35, etc.

[0097] In the above embodiment, the exponential weighted average value corresponding to the audio frame is determined according to the audio frame and the average value of the Mask corresponding to the previous audio frame before the audio frame, and whether the audio frame is a speech frame is determined according to the exponential weighted average value, which can effectively prevent frame loss of speech information.

[0098] In addition, in the embodiment of the present application, when determining the exponential weighted average value corresponding to the audio frame, the exponential weighted average value corresponding to the audio frame can also be determined according to the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to a set number of other audio frames before the audio frame. The specific set number can be set according to requirements. Specifically, when the exponential weighted average value corresponding to the audio frame is determined according to the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to a set number of other audio frames before the audio frame, the average value of the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to a set number of other audio frames before the audio frame can be taken as the exponential weighted average value corresponding to the audio frame. The specific calculation process can be realized by the following formula:

[0099]

[0100] wherein moving_mean_t is the exponential weighted average value corresponding to the audio frame t, mean(Mask (k,f) ) is the average value of the Mask corresponding to the audio frame k, wherein s is a set value, for example, the s can be 20, 30, 35, etc.

[0101] In the above embodiment, the exponential weighted average value corresponding to the audio frame is determined according to the average value of the Mask corresponding to the audio frame and the average value of the Mask corresponding to a set number of other audio frames before the audio frame, which can effectively prevent frame loss of speech information.

[0102] Embodiment 5:

[0103] In order to effectively eliminate residual noise in the audio signal, reduce the amount of calculation, and avoid distortion of speech information, on the basis of the above embodiment, in the embodiment of the present application, the updating of the value corresponding to the audio frame in the current buffer vector according to whether the audio frame is a speech frame comprises:

[0104] determining the value corresponding to the audio frame according to whether the audio frame is a speech frame;

[0105] The value corresponding to the to-be-updated bit in the buffer vector is updated by using the value, and the value of each bit in the updated buffer vector is moved by one bit in a set direction, wherein the initial value of each bit in the buffer vector is a preset value, and the preset value is a value greater than 0 and not more than 1.

[0106] In the embodiments of the present application, in order to avoid the audio signal being suddenly truncated at the voice cutoff time and affecting the hearing, make the change of the audio frame from the voice frame to the non-voice frame more natural, and avoid the loss of voice information caused by the truncation of the non-voice frame, a buffer vector is provided. After it is determined whether the audio frame is a voice frame or a non-voice frame, if the audio frame is a voice frame, the value corresponding to the audio frame can be the initial value of the value in the buffer vector, if the initial value is 1, if the audio frame is a voice frame, the value corresponding to the audio frame is 1, and if the audio frame is a non-voice frame, the value corresponding to the audio frame is 0.

[0107] After the value corresponding to the to-be-updated bit in the buffer vector is updated, the vector mean of the buffer vector can be determined according to the value of each bit in the updated buffer vector, the Mask vector is adjusted according to the vector mean and the Mask vector corresponding to the audio frame, so that the audio frame is denoised based on the adjusted Mask vector.

[0108] The buffer vector is used to adjust the Mask vector, which can avoid the audio signal being suddenly truncated at the voice cutoff time and affecting the hearing, so that the change of the voice frame to the non-voice frame is more natural.

[0109] Any bit of the buffer vector can be a to-be-updated bit, but in order to facilitate updating, the first bit of the buffer vector can be a to-be-updated bit, or the last bit of the buffer vector can be a to-be-updated bit. After the value corresponding to the audio frame is determined according to whether the audio frame is a voice frame, the to-be-updated bit in the buffer vector is updated by using the value, if the first bit is a to-be-updated bit, the value of the first bit in the buffer vector is updated by using the value, and if the last bit is a to-be-updated bit, the value of the last bit in the buffer vector is updated by using the value.

[0110] In addition, in order to facilitate updating the value of the to-be-updated bit in the buffer vector according to the value corresponding to the next audio frame, the value of each bit in the updated buffer vector is moved by one bit in a set direction, when the bit is moved by one bit, the value corresponding to some bits in the buffer vector will be moved out, and some bits will be moved into new values, and the moved-in new values are initial values.

[0111] Specifically, when the bit to be updated in the buffer vector is the first bit, the setting direction is backward, shifting the value of each bit in the buffer vector one bit to the right. The value of the last bit will be shifted out, and the first bit of the buffer vector will be the bit to be updated. The value of the bit to be updated is newly shifted in, and the value corresponding to the bit to be updated is the initial value.

[0112] When the bit to be updated in the buffer vector is the last bit, the setting direction is forward, and the value of each bit in the buffer vector is moved forward by one bit. The value of the first bit will be moved out, and the last bit of the buffer vector is the bit to be updated. The value of the bit to be updated is newly moved in, and the value corresponding to the bit to be updated is the initial value.

[0113] If the initial value of each bit in the buffer vector is 1, when the audio frame is a speech frame, the value corresponding to that audio frame in the buffer vector is updated to 1; when the audio frame is not a speech frame, the value corresponding to that audio frame in the buffer vector is updated to 0. Therefore, as the audio signal changes from a speech signal to no speech signal, the number of 1s in the buffer vector decreases, and the number of 0s increases, causing the average value of the buffer vector to decrease. This gradually zeros out the audio signal, preventing the speech signal from being suddenly truncated and affecting the listening experience. When the audio signal consists entirely of non-speech frames, the value in the buffer vector is always 0, thus completely eliminating residual noise.

[0114] Figure 4 This application provides schematic diagrams comparing audio signals processed using existing technology and methods provided in the embodiments of this application, for the purpose of illustrating some embodiments of the application. Figure 5 This application provides schematic diagrams comparing audio signals processed using another prior art technique and the method provided in the embodiments of this application, which are shown in some of the embodiments of this application. Figure 6 Some embodiments of this application are provided based on Figure 5 The diagram shown is a comparison of audio signals processed using two existing technologies and the methods provided in the embodiments of this application. Figure 4 , Figure 5 and Figure 6 As shown, the first row contains the original audio signals. Figure 4 The second line and Figure 6 The third line in the text uses the audio signal processed by existing technology 1. In existing technology 1, the determination of whether an audio frame is a speech frame is based on the average value of the Mask. However, some speech frames may still be set to zero. Figure 5 and Figure 6 The second line in the text is an audio signal processed by another existing technology, which does not determine whether the audio frame is a speech frame or does not use a buffer vector for smoothing. Figure 4 The second row, located within box 1, still shows frame drops in the first half of the processed audio signal, such as...Figure 4 In the second row, the blank area in the box 1 shows that, Figure 6 In the third row, the same problem also exists. Figure 5 In the second row, Figure 6 In the second row, noise exists relative to the third row, that is, non-speech frames are identified as speech frames. Because the two methods in the prior art do not use a buffer vector, Figure 4 In the second row, the stop position of the latter half of the speech of the processed audio signal in the box 2 is zeroed faster, such as Figure 4 In the second row, the blank area in the box 2 shows that, Figure 6 In the last part of the third row, therefore, another prior art does not judge whether the audio frame is a speech frame, and therefore Figure 5 In the second row, Figure 6 In the second row, noise exists in the latter half, that is, non-speech frames are identified as speech frames. Figure 4 In the third row, Figure 5 In the third row, and Figure 6 In the last row, the audio signal processed by the method provided in the embodiments of the present application, because in the embodiments of the present application, whether the audio frame is a speech frame is judged according to the Mask average value corresponding to the audio frame and the first threshold preset, and the exponential weighted average value corresponding to the audio frame and the second threshold preset, therefore Figure 4 In the third row, in the box 1, and Figure 5 In the third row, and Figure 6 In the last row, the former half of the processed audio signal has no frame loss, so that frame loss can be effectively prevented, and in addition, the buffer vector is used to process the audio frame in the embodiments of the present application, so that the stop position of the speech can be slowly zeroed, such as Figure 4 In the third row, in the box 2, Figure 5 In the latter half of the third row, and Figure 6 In the latter half of the last row, so that the speech is prevented from being suddenly truncated to make the listening experience worse.

[0115] Figure 7 The process of processing the audio signal provided in some embodiments of the present application is shown in the schematic diagram, such as Figure 5 As shown, the process includes the following steps:

[0116] S701: For each audio frame in the audio signal, based on the Mask noise reduction algorithm, determine the Mask vector corresponding to the audio frame.

[0117] S702: According to each value of the Mask vector, calculate the Mask average value corresponding to the audio frame.

[0118] S703: determining an exponential weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and Mask average values corresponding to other audio frames before the audio frame.

[0119] S704: determining whether the Mask average value is not less than a preset first threshold value and the exponential weighted average value is not less than a preset second threshold value, if both are yes, determining that the audio frame is a speech frame, if both are no, determining that the audio frame is a non-speech frame.

[0120] S705: updating a value corresponding to the audio frame in the current buffer vector according to whether the audio frame is a speech frame.

[0121] Figure 8 a comparison diagram of noises in audio signals processed by another prior art and the method provided by some embodiments of the present application, Figure 9 a comparison diagram of noises in audio signals processed by a noise reduction algorithm and the method provided by some embodiments of the present application, Figure 10 a comparison diagram of noises in audio signals processed by the method provided by some embodiments of the present application based on Figure 8 the noise signal shown in the figure, a comparison diagram of noises in audio signals processed by two prior arts and the method provided by some embodiments of the present application, as shown in Figure 8 , Figure 9 and Figure 10 , the first row is the noise signal contained in the original audio signal, Figure 8 and Figure 10 the second row is the noise signal contained in the audio signal processed by another prior art, the method does not determine whether the audio frame is a speech frame and does not use a buffer vector for smoothing, therefore, the existence of non-speech frames causes a large amount of noise signals to be left, as shown by the black lines in the second row in Figure 8 ; Figure 9 the second row in Figure 10 is the noise signal in the audio signal processed by a noise reduction algorithm, but the method does not determine whether the audio frame is a speech frame and does not use a buffer vector for smoothing, therefore, the existence of non-speech frames also causes a large amount of noise signals to be left; Figure 8 the third row in Figure 9 and Figure 10The last line in the table is a noise signal contained in the audio signal processed by the method provided in the embodiments of the present application. The noise elimination method in the audio signal provided in the embodiments of the present application is used to process the audio signal. The residual noise in the non-speech frame is only a small part, and the noise elimination method in the audio signal provided in the embodiments of the present application can effectively set the residual noise in most non-speech frames to zero, Figure 8 、 Figure 9 and Figure 10 It is indicated that the noise elimination method in the audio signal provided in the embodiments of the present application is used to determine whether the audio frame is a speech frame according to the Mask average value, the exponential weighted average value and the corresponding threshold value. The purpose is to completely set the non-speech frame to zero, reduce the residual noise in the non-speech frame, and use the buffer vector in the present application to process the audio frame. The purpose is to prevent frame loss of speech and slow zero setting of the cut-off position of speech, and prevent the speech from being suddenly truncated to make the listening experience worse.

[0122] Embodiment 6:

[0123] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide a noise elimination device in an audio signal, Figure 11 A structure diagram of a noise elimination device in an audio signal provided for some embodiments of the present application is shown in the figure. The device comprises:

[0124] A determination module 1101 is configured to determine, for each audio frame in an audio signal, a Mask vector corresponding to the audio frame based on a mask noise reduction algorithm.

[0125] A judgment module 1102 is configured to determine whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value.

[0126] The determination module 1101 is further configured to update a value corresponding to the audio frame in a current buffer vector according to whether the audio frame is a speech frame, determine a vector average of the buffer vector according to each value in the updated buffer vector, determine an adjusted Mask vector according to the vector average and the Mask vector, and determine an enhanced speech vector corresponding to the audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame.

[0127] Further, the judgment module 1102 is specifically configured to calculate a Mask average value corresponding to the audio frame according to each value of the Mask vector, determine whether the Mask average value is not less than a preset first threshold value, determine that the audio frame is a speech frame if yes, and otherwise, determine that the audio frame is a non-speech frame.

[0128] Further, the judgment module 1102 is specifically used to calculate the average Mask value corresponding to the audio frame based on each value of the Mask vector; determine the exponentially weighted average value corresponding to the audio frame based on the average Mask value corresponding to the audio frame and the average Mask values ​​corresponding to other audio frames before the audio frame; determine whether the exponentially weighted average value is not less than a preset second threshold; if so, determine that the audio frame is a speech frame; otherwise, determine that the audio frame is a non-speech frame.

[0129] Further, the judgment module 1102 is specifically used to determine the exponentially weighted average value corresponding to the audio frame based on the average Mask value corresponding to the audio frame and a preset first weight value, and the average Mask value corresponding to the previous audio frame before the audio frame and a preset second weight value; or to determine the average value based on the average Mask value corresponding to the audio frame and the average Mask values ​​corresponding to a set number of other audio frames before the audio frame, and use the average value as the exponentially weighted average value corresponding to the audio frame.

[0130] Further, the determining module 1101 is specifically used to determine the value corresponding to the audio frame based on whether the audio frame is a speech frame; to update the value corresponding to the bit to be updated in the current buffer vector using the value, and to move the value of each bit in the updated buffer vector one bit in a set direction, wherein the initial value of each bit in the buffer vector is a preset value, wherein the preset value is a value greater than 0 and not exceeding 1.

[0131] In this embodiment, a buffer vector is provided, and the value corresponding to the audio frame in the buffer vector can be updated according to whether the audio frame is a speech frame. The average value of the updated buffer vector can be obtained. The Mask vector is adjusted using the average value of the updated buffer vector, so that the enhanced speech vector corresponding to the audio frame can be obtained from the adjusted Mask vector. Since the audio frame recognition process does not require adding a model, the amount of computation is reduced. At the same time, it does not cause additional damage to the speech signal when eliminating noise, thus ensuring the accuracy of the processed audio signal.

[0132] Example 7:

[0133] Based on the above embodiments, this application also provides an electronic device. Figure 12 A schematic diagram of an electronic device is provided for some embodiments of this application, such as... Figure 12 As shown, the device includes a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204, wherein the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0134] The memory 1203 stores a computer program which, when executed by the processor 1201, causes the processor 1201 to perform the noise removal method in the audio signal of any of the above embodiments.

[0135] The communication bus mentioned by the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0136] The communication interface 1202 is used for communication between the above electronic device and other devices.

[0137] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the above processor.

[0138] The above processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; can also be a Digital Signal Processing (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0139] Embodiment 8:

[0140] On the basis of the above embodiments, the embodiments of the present application further provide a computer storage medium, which stores a computer program executable by an electronic device, and when the program runs on the electronic device, causes the electronic device to perform the noise removal method in the audio signal of any of the above embodiments.

[0141] In the embodiment of the present application, a buffer vector is set, and the value corresponding to the audio frame in the buffer vector can be updated according to whether the audio frame is a speech frame, so that the vector mean of the updated buffer vector is obtained, the Mask vector is adjusted by using the vector mean, and the enhanced speech vector corresponding to the audio frame can be obtained according to the adjusted Mask vector. Because the process of identifying the audio frame does not need to increase a model, the calculation amount is reduced, meanwhile, the speech signal is not damaged additionally when the noise is eliminated, and the accuracy of the processed audio signal is ensured.

[0142] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0143] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the flow Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the function specified in the flow or flows and / or block or blocks.

[0144] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the flow Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the function specified in the flow or flows and / or block or blocks.

[0145] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the flow Figure 1 one or more flows and / or blocks Figure 1The functions of the various elements shown in the figures, including any functional blocks labeled as "processors", "controllers", "logic", etc., can be provided through the use of dedicated hardware as well as hardware capable of executing software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared or distributed. Moreover, explicit use of the terms "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. Other hardware, conventional and / or custom, can also

Claims

1. A method of noise cancellation in an audio signal, characterized by, The method comprises: For each audio frame in the audio signal, determining a Mask vector corresponding to the audio frame based on a mask noise reduction algorithm; determining whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value; updating a value corresponding to the audio frame in a current buffer vector according to whether the audio frame is a speech frame; determining a vector mean of the buffer vector according to each value in the updated buffer vector; determining an adjusted Mask vector according to the vector mean and the Mask vector; and determining an enhanced speech vector corresponding to the audio frame according to the adjusted Mask vector and a frequency domain vector corresponding to the audio frame. The updating of the value corresponding to the audio frame in the current buffer vector according to whether the audio frame is a speech frame comprises: determining a value corresponding to the audio frame according to whether the audio frame is a speech frame; updating a value corresponding to a to-be-updated bit in the current buffer vector using the value, and moving each bit value in the updated buffer vector to a set direction by one bit, wherein an initial value of each bit in the buffer vector is a preset value, and the preset value is a value greater than 0 and not more than 1; and the to-be-updated bit is any bit in the buffer vector.

2. The method of claim 1, wherein, The determination of whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value comprises: calculating a Mask average value corresponding to the audio frame according to each value of the Mask vector; determining whether the Mask average value is not less than a preset first threshold value; if yes, determining that the audio frame is a speech frame, and otherwise, determining that the audio frame is a non-speech frame.

3. The method of claim 1 or 2, wherein, The determination of whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value comprises: calculating a Mask average value corresponding to the audio frame according to each value of the Mask vector; determining an exponentially weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and Mask average values corresponding to other audio frames before the audio frame; determining whether the exponentially weighted average value is not less than a preset second threshold value; if yes, determining that the audio frame is a speech frame, and otherwise, determining that the audio frame is a non-speech frame.

4. The method of claim 3, wherein, The determination of the exponentially weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and Mask average values corresponding to other audio frames before the audio frame comprises: determining the exponentially weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and a preset first weight value, and a Mask average value corresponding to a previous audio frame before the audio frame and a preset second weight value; or determining an average value according to the Mask average value corresponding to the audio frame and Mask average values corresponding to a preset number of other audio frames before the audio frame, and taking the average value as the exponentially weighted average value corresponding to the audio frame.

5. A noise canceling apparatus in an audio signal, characterized by, The device comprises: a determination module configured to determine, for each audio frame in an audio signal, a Mask vector corresponding to the audio frame based on a mask noise reduction algorithm; determining whether the audio frame is a speech frame according to each value of the Mask vector and a preset threshold value; The determining module is specifically configured to: determine a value corresponding to the audio frame according to whether the audio frame is a speech frame; update a value corresponding to a to-be-updated bit in the current buffer vector using the value; and move each value in the updated buffer vector by one bit in a set direction, where an initial value of each bit in the buffer vector is a preset value, and the preset value is a value greater than 0 and not more than 1; and the to-be-updated bit is any bit in the buffer vector. The determining module is specifically configured to: calculate a Mask average value corresponding to the audio frame according to each value of the Mask vector; determine whether the Mask average value is not less than a preset first threshold value; if yes, determine that the audio frame is a speech frame; otherwise, determine that the audio frame is a non-speech frame.

6. The apparatus of claim 5, wherein, The determining module is specifically configured to: calculate a Mask average value corresponding to the audio frame according to each value of the Mask vector; determine an exponentially weighted average value corresponding to the audio frame according to the Mask average value corresponding to the audio frame and Mask average values corresponding to other audio frames before the audio frame; determine whether the exponentially weighted average value is not less than a preset second threshold value; if yes, determine that the audio frame is a speech frame; otherwise, determine that the audio frame is a non-speech frame.

7. The apparatus of claim 5 or 6, wherein, The electronic device at least includes a processor and a memory, and the processor is configured to implement the steps of the noise elimination method in an audio signal according to any one of claims 1-4 when executing a computer program stored in the memory.

8. An electronic device, comprising: The electronic device at least includes a processor and a memory, and the processor is configured to implement the steps of the noise elimination method in an audio signal according to any one of claims 1-4 when executing a computer program stored in the memory.

9. A computer storage medium, characterized in that ​

Images