Voice activity detection method and headphone

Abstract

The invention provides a voice activity detection method and a headset, and relates to the field of noise reduction. The voice activity detection method comprises the steps that multiple microphone voice signals corresponding to the same frame of voice are acquired, one frame of the multiple microphone voice signals comprises a forward voice signal and a backward voice signal corresponding to the same frame of voice, the forward voice signal is a voice signal in a preset user voice transmission direction, and the backward voice signal is a voice signal in a preset user voice transmission direction; the backward voice signal is a voice signal in a direction opposite to a preset user voice transmission direction; calculating the correlation of the forward voice signal and the backward voice signal and a first energy ratio; and determining the type of the multi-path microphone voice signal based on the correlation and the first energy ratio, wherein the type of the multi-path microphone voice signal is a voice frame or a noise frame. The method can effectively improve the detection accuracy of voice activity detection.

Description

Technical Field

[0001] The present application relates to the field of noise reduction, and in particular, to a voice activity detection method, a voice activity detection method, and a headset. Background Art

[0002] With the popularity of remote conferencing, users have an increasing demand for making calls and holding meetings in noisy open environments, which places higher and higher requirements on the noise reduction capabilities of headphones.

[0003] Currently, when the noise reduction headset is working, it is necessary to perform voice activity detection on the voice signal frame collected by the microphone, that is, to determine whether the voice signal frame is a noise frame or a voice frame, and then suppress the noise frame.

[0004] The existing different voice activity detection methods have the problem of low detection accuracy when distinguishing between noise frames and voice frames. For example, some detection methods cannot track the rapid changes in the noise spectrum, and the detection accuracy is low in noisy environments; some detection methods cannot distinguish between the user's own voice and the voice of others, and the accuracy is low when the ambient noise is large. Summary of the invention

[0005] In view of this, the present application aims to provide a voice activity detection method and a headphone to improve the detection accuracy of voice activity and thereby improve the noise reduction effect of the headphone.

[0006] First, an embodiment of the present application provides a voice activity detection method, which is applied to a processing unit of a headset, and the voice activity detection method includes: obtaining multi-channel microphone voice signals corresponding to the same frame of sound, one frame of the multi-channel microphone voice signals includes a forward voice signal and a backward voice signal corresponding to the same frame of sound, the forward voice signal is a voice signal in a preset user voice propagation direction, and the backward voice signal is a voice signal in the opposite direction of the preset user voice propagation direction; calculating the correlation and a first energy ratio of the forward voice signal and the backward voice signal; determining the type of the multi-channel microphone voice signal based on the correlation and the first energy ratio, and the type of the multi-channel microphone voice signal is a voice frame or a noise frame.

[0007] In the embodiment of the present application, voice activity detection is performed on each frame of sound, so that the rapid changes of the noise spectrum can be tracked, so that a high detection accuracy can be achieved even in a noisy environment. The forward voice signal is a voice signal in the preset user voice propagation direction, which can effectively suppress the characteristics of noise and reflect the characteristics of the voice emitted by the user. Conversely, the reverse voice signal is a signal in the opposite direction of the propagation direction, which can better suppress the characteristics of the user's voice and better reflect the characteristics of noise. The forward voice signal and the backward voice signal are used to judge the noise frame and the voice frame, which can effectively improve the detection accuracy. Accordingly, when judging whether it is a voice frame and a noise frame, the excessive or too small first energy ratio of the forward voice signal and the backward voice signal can effectively characterize whether the two are both noise frames or voice frames. The forward voice signal and the backward voice signal have different suppressed directions and different prominent voice features. Therefore, when the first energy ratio of the two cannot distinguish between the noise frame and the voice frame, a more accurate detection result can be obtained by combining the correlation for judgment.

[0008] In one embodiment, the earphone is a headset, and the headset includes a microphone rod, on which a first microphone and a second microphone are arranged; the obtaining of multi-channel microphone voice signals corresponding to the same frame of sound includes: obtaining a first voice signal collected by the first microphone and a second voice signal collected by the second microphone on the microphone rod when collecting the same frame of sound; performing forward beamforming processing on the first voice signal and the second voice signal to obtain the forward voice signal; performing backward beamforming processing on the first voice signal and the second voice signal to obtain the backward voice signal.

[0009] In the embodiment of the present application, compared with the microphone arranged in the earphone cavity, the microphone arranged on the microphone rod can be closer to the sound source, that is, the user's mouth, and the voice signal collected by the microphone on the microphone rod has a higher signal-to-noise ratio. Voice activity detection is performed using the first voice signal and the second voice signal collected by the microphone on the microphone rod, which can effectively improve the accuracy of voice activity detection.

[0010] In one embodiment, on the microphone pole, the first microphone and the second microphone are spaced apart by a preset distance.

[0011] In the embodiment of the present application, beamforming processing requires the use of differential signals. If the distance between the first microphone and the second microphone is too close, the difference between the voice signals collected by the two may be small and cannot be used for beamforming. Therefore, the first microphone and the second microphone are separated by a certain distance. The voice signals collected by the two have a more obvious difference, which helps to improve the effect of beamforming, and further improve the accuracy of voice activity detection based on the forward voice signal and the backward voice signal of the beamforming.

[0012] In one embodiment, the preset distance ranges from 2 cm to 4 cm.

[0013] In this embodiment, for the headphones, the preset distance is within the range of 2 cm to 4 cm, which can make the first voice signal and the second voice signal have obvious differences, while avoiding the situation where the distance is too large, resulting in the first voice signal and the second voice signal being too different to accurately determine the common forward or backward features through beamforming.

[0014] In one embodiment, the voice activity detection result is determined based on the correlation and the first energy ratio, including: if the first energy ratio is less than or equal to the first preset energy ratio threshold, determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is the noise frame; if the first energy ratio is greater than or equal to the second preset energy ratio threshold, determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is the voice frame; the second preset energy ratio threshold is greater than the first preset energy ratio threshold; if the first energy ratio is between the first preset energy ratio threshold and the second preset energy ratio threshold, judging: if the correlation is less than the preset correlation threshold, determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is the noise frame; if the correlation is greater than the preset correlation threshold, determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is the voice frame.

[0015] In the embodiment of the present application, the first energy ratio is the energy ratio of the forward voice signal and the backward voice signal. The forward voice signal and the backward voice signal have different characteristics, which are the forward and backward sound characteristics, respectively. If the first energy ratio is small, it indicates that the backward voice signal is large, which means that the noise signal is large. Therefore, it can be directly determined as a noise frame when the first energy ratio is greater than the second preset energy ratio threshold. On the contrary, if the first energy ratio is large, greater than the first preset energy ratio threshold or even close to 1, it means that the backward voice signal is similar to the forward voice signal, and it is highly likely to be a voice frame. When the first energy ratio is between the first preset energy ratio threshold and the second preset energy ratio threshold, there is interference, but the size of the interference is uncertain, and it is impossible to directly determine the voice frame or the noise frame. Therefore, in this case, it can be judged in combination with the correlation. When the first energy ratio is between the first preset energy ratio threshold and the second preset energy ratio threshold, if the correlation between the two is small, it indicates that the noise is large, and it can be judged as a noise frame. On the contrary, if the correlation between the two is large, it indicates that the noise is small, and it can be judged as a voice frame. In the above manner, voice activities under different circumstances can be accurately judged, and more accurate detection results can be obtained.

[0016] In one embodiment, the headset includes a left earphone, a right earphone, a third microphone and a fourth microphone; the third microphone is arranged on the left earphone, and the fourth microphone is arranged on the right earphone; the multi-channel microphone voice signal also includes a third voice signal collected by the third microphone and a fourth voice signal collected by the fourth microphone; the voice activity detection method also includes: when the first energy ratio is greater than or equal to the first preset energy ratio threshold, obtaining and determining a second energy ratio between the third voice signal and the fourth voice signal; if the second energy ratio is greater than a fifth energy threshold, or the second energy ratio is less than a sixth energy threshold, re-determining the type of the multi-channel microphone voice signal as the noise frame; the fifth energy threshold is greater than the sixth energy threshold.

[0017] In the embodiment of the present application, when the first energy ratio is greater than or equal to the first preset energy ratio threshold, it may be because the noise signals on both sides of the user are larger, and the energy ratios of the noise on both sides in the forward and backward directions are similar. Therefore, in this case, the second energy ratio can be calculated by combining the voice signals collected by the third microphone and the fourth microphone on both sides of the headset. The second energy ratio is used to characterize the sound volume on both sides of the user to assist in determining whether it is a noise frame. If the second energy ratio is too large or too small, it is characterized as a noise frame. Therefore, the noise frame can be further judged by the second energy ratio, which can improve the accuracy of voice activity detection.

[0018] In one embodiment, the calculation of the correlation and the first energy ratio of the forward speech signal and the backward speech signal includes: calculating the correlation and the first energy ratio based on speech signals in a preset frequency range in the forward speech signal and the backward speech signal; the preset frequency range matches the frequency range of human voice.

[0019] In this embodiment, the preset frequency range matches the frequency range of human voice, and the voice signal in the human voice frequency range is intercepted to calculate the correlation and energy ratio, which can more accurately intercept the human voice features in the forward voice signal and the backward voice signal, especially the voice of the headphone user. The correlation and the first energy ratio are calculated based on the voice signal in the preset frequency range, so that the correlation and the first energy ratio are more closely related to the human voice, and thus the noise frame and the voice frame determined based on the correlation and the first energy ratio are more accurate.

[0020] In one embodiment, the processing unit is configured with multiple noise suppression modules with different noise suppression methods; after determining the types of the multi-channel microphone voice signals based on the correlation and the first energy ratio, the voice activity detection method also includes: processing the forward voice signal based on the multiple noise suppression modules with different noise suppression methods respectively to obtain the gain output by each noise suppression module; based on a preset gain selection condition, determining a target gain from the gains output by each noise suppression module; and using the target gain to reduce noise on the forward voice signal.

[0021] In this embodiment, multiple noise suppression modules are set to determine multiple noise reduction gains, thereby, for speech signals in different scenarios, a more appropriate target gain can be determined for noise reduction to improve the noise reduction effect.

[0022] In one embodiment, the processing unit further includes an adaptive filter, and the multiple noise suppression modules include: a single microphone noise suppression module, and a residual nonlinear echo cancellation module; before the multiple noise suppression modules with different noise suppression modes respectively process the forward speech signal, the method further includes: inputting the forward speech signal into the adaptive filter to obtain the forward speech signal after adaptive filtering; wherein, if the forward speech signal is the noise frame, the adaptive filter adjusts its own filter coefficient based on the noise frame; the multiple noise suppression modules with different noise suppression modes respectively process the forward speech signal to obtain the gain output by each noise suppression module, including: inputting the forward speech signal after adaptive filtering into the residual nonlinear echo cancellation module for residual nonlinear echo cancellation processing to obtain a first gain; outputting the forward speech signal after adaptive filtering to the single microphone noise suppression module to obtain a second gain output by the single microphone noise suppression module after noise suppression.

[0023] The adaptive filter in the headset will adjust its own filter coefficients to adjust the filtering ability of noise. If the voice signal input to the adaptive filter is a voice frame, the human voice in the voice signal frame may be distorted. Therefore, in the embodiment of the present application, before the forward voice signal is input to the adaptive filter and each noise suppression module calculates the gain, it is first determined whether it is a noise frame. In the case of a noise frame, the coefficient of the adaptive filter is updated. After the filter is updated, the first gain is calculated to track the change of noise in real time, improve the noise reduction effect, and reduce voice distortion. Regardless of residual nonlinear echo elimination or single microphone noise suppression, the noise in the forward voice signal can be better suppressed to obtain a gain with a better noise reduction effect.

[0024] In one embodiment, the multiple noise suppression modules also include: an energy difference noise suppression module; the headset also includes a noise estimation module; the noise estimation module is also connected to the processing unit; the multiple noise suppression modules with different noise suppression methods respectively process the forward voice signal to obtain the gain output by each noise suppression module, and also include: based on the energy difference noise suppression module, the noise estimation output by the noise estimation module and the forward voice signal after adaptive filtering are processed to obtain a third gain; wherein the headset also includes a left headset, a right headset, a third microphone and a fourth microphone; the third microphone is arranged on the left headset, and the fourth microphone is arranged on the right headset; the multi-channel microphone voice signal also includes a third voice signal collected by the third microphone and a fourth voice signal collected by the fourth microphone; the The method also includes: before processing the noise estimate output by the noise estimation module and the forward speech signal after adaptive filtering based on the energy difference noise suppression module, obtaining the third speech signal and the fourth speech signal corresponding to the same frame of sound, and determining the maximum auxiliary microphone signal with higher energy from the two; calculating the third energy difference between the forward speech signal and the maximum auxiliary microphone signal; if the third energy difference is greater than or equal to the third preset energy difference threshold, not updating the noise estimate in the noise estimation module; if the third energy difference is less than or equal to the fourth preset energy difference threshold, updating the noise estimate in the noise estimation module based on the maximum auxiliary microphone signal; if the third energy difference is between the third preset energy difference threshold and the fourth preset energy difference threshold, updating the noise estimate in the noise estimation module based on the forward speech signal.

[0025] Although the forward speech signal can suppress part of the interfering speech after residual nonlinear echo elimination and single microphone noise suppression, it is difficult to completely eliminate the interfering signal. Therefore, in this embodiment, the speech signals collected by the third microphone and the fourth microphone are used as auxiliary microphone signals, the maximum auxiliary microphone signal is determined, and the third energy difference between the forward speech signal and the maximum auxiliary microphone signal is updated to determine the method for updating the noise estimation, so as to obtain a noise estimation that is more suitable for the currently collected speech signal, so that when the noise estimation is used to suppress the noise of the forward speech signal, the noise suppression effect can be improved.

[0026] In one embodiment, the preset gain selection condition is to select a minimum gain from multiple gains as the target gain.

[0027] In this embodiment, using the minimum gain for noise reduction has a certain noise reduction effect, and since the gain is minimum, the possibility of speech signal distortion during noise reduction due to excessive gain can be effectively reduced.

[0028] In a second aspect, an embodiment of the present application further provides a headset, comprising: a microphone for collecting sound; and a processing unit connected to the microphone, for executing the voice activity detection method as described in any one of the first aspects.

[0029] In one embodiment, the headset further includes a microphone rod, on which a first microphone and a second microphone are arranged; when collecting the same frame of sound, the first microphone is used to collect a first voice signal and the second microphone is used to collect a second voice signal; the processing unit is used to perform forward beamforming processing on the first voice signal and the second voice signal to obtain the forward voice signal; and, perform backward beamforming processing on the first voice signal and the second voice signal to obtain the backward voice signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 A schematic diagram of the structure of a headset provided in one embodiment of the present application;

[0032] Figure 2 A flow chart of a voice activity detection method provided in one embodiment of the present application;

[0033] Figure 3 A schematic diagram of a voice activity detection device provided in one embodiment of the present application.

[0034] Icon: left earphone 110; third microphone 111; right earphone 120; fourth microphone 121; headband 130; microphone rod 140; first microphone 141; second microphone 142; voice activity detection device 200; acquisition module 210; beamforming module 211; voice activity detection module 220; adaptive filter 230; residual nonlinear echo cancellation module 241; single microphone noise suppression module 242; energy difference noise suppression module 243; noise estimation module 244; noise reduction module 250. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] First, an embodiment of the present application provides a headset, which is equipped with a communication connection processing unit and at least two microphones. The microphone is used to collect sound and obtain a voice signal, and the processing unit is used to obtain the voice signal of the microphone and execute the voice activity detection method provided in the embodiment of the present application.

[0037] In one embodiment of the present application, the microphone may be disposed in the headset. In another embodiment of the present application, the microphone may also be an independent microphone and be connected to the headset for communication, which may be an indirect connection, a communication connection, a wireless connection, or a wired connection, which is not limited here.

[0038] See also Figure 1 , Figure 1 Schematic diagram of an earphone provided in an embodiment of the present application. In the embodiment of the present application, the earphone may be a headset, the headset includes a left earphone 110, a right earphone 120 and a headband 130, and the left earphone 110 and the right earphone 120 are connected by the headband 130.

[0039] In an embodiment of the present application, a microphone rod 140 is provided on one of the left earphone 110 or the right earphone 120, and the microphone can be provided on the microphone rod 140. For example, in one embodiment, a first microphone 141 and a second microphone 142 are provided on the microphone rod 140, and the voice signal collected by the first microphone 141 is a first voice signal, and the voice signal collected by the second microphone 142 is a second voice signal.

[0040] like Figure 1 As shown, in some embodiments, a third microphone 111 may be provided on the left earphone 110, and a fourth microphone may be provided on the right earphone, and the third microphone 111 and the fourth microphone 121 may be symmetrically distributed. In this embodiment, the voice signal collected by the third microphone 111 is the third voice signal, and the voice signal collected by the fourth microphone 121 is the fourth voice signal.

[0041] Among them, in an embodiment of the present application, the first microphone 141 and the second microphone 142 on the microphone rod 140 can be called main microphones, the third microphone 111 set on the left earphone 110 and the fourth microphone 121 set on the right earphone can be called auxiliary microphones, the main microphone is mainly used to collect the sound made by the headphone user, and the auxiliary microphone is mainly used to collect the noise of the user's environment.

[0042] See also Figure 2 , Figure 2 A flow chart of a voice activity detection method provided in an embodiment of the present application, the voice activity detection method comprising:

[0043] S110, obtaining multi-channel microphone voice signals corresponding to the same frame of sound.

[0044] In this embodiment, the same frame of sound may be a sound signal collected at the same time. The multi-channel microphone voice signal includes voice signals collected by different microphones or different processed voice signals.

[0045] For example, in an embodiment of the present application, a frame of multi-channel microphone voice signal may include a forward voice signal and a backward voice signal corresponding to the same frame of sound. Among them, the forward direction refers to the preset sound propagation direction of the headphone user, and the reverse direction is opposite to the forward direction. Specifically, the sound source is the user's mouth, and the microphone rod 140 is usually used to be placed in front of the user's mouth. On this basis, the position where the user's mouth may be located can be pre-configured, and then the direction of the preset user's mouth pointing to the microphone on the microphone rod 140 is configured as the forward direction. Thus, the forward voice signal can be characterized as a voice signal in the preset user's sound propagation direction. When the user speaks, the forward voice signal is more likely to be the sound of the user speaking. In the absence of other interference, the microphone can only collect the user's voice, and accordingly, the reverse voice signal is also mainly the characteristics of the user's voice. Therefore, the reverse voice signal and the forward voice signal have similar characteristics and have a high correlation. In the case of interference, the reverse voice signal will suppress the user's voice and enhance the noise in front of the user, thereby making the reverse voice signal and the forward voice signal highlight completely different features. Therefore, due to the characteristics of the forward speech signal and the backward speech signal, using the forward speech signal and the backward speech signal to perform voice activity detection helps to improve the accuracy of the detection result.

[0046] As mentioned above, the suppressed features and retained features of the forward speech signal and the backward speech signal are different, so the first microphone 141 and the second microphone 142 can be directionally collected for the forward and backward speech signals, respectively.

[0047] If the microphone collects voice signals non-directionally, then, in an embodiment of the present application, obtaining multi-channel microphone voice signals corresponding to the same frame of sound may include: obtaining a first voice signal collected by the first microphone 141 on the microphone rod 140 and a second voice signal collected by the second microphone 142 when collecting the same frame of sound; performing forward beamforming processing on the first voice signal and the second voice signal to obtain the forward voice signal; and performing backward beamforming processing on the first voice signal and the second voice signal to obtain the backward voice signal.

[0048] The beamforming technology requires at least two or more signals. Therefore, in the embodiment of the present application, the voice signals collected by the first microphone 141 and the second microphone 142 can be used for beamforming.

[0049] When beamforming is performed, the speech signals that need beamforming have obvious differences. Therefore, the first microphone 141 and the second microphone 142 should not be placed close together. Figure 1 As shown, in the embodiment of the present application, the first microphone 141 and the second microphone 142 can detect the preset distance on the microphone rod 140. Thus, the first voice signal and the second voice signal can be more obviously distinguished, so that the forward voice signal determined by the two can more accurately reflect the characteristics of the user's voice, and the backward voice signal can more accurately reflect the characteristics of the noise.

[0050] Furthermore, the first microphone 141 and the second microphone 142 can be spaced apart by 2 cm to 4 cm, that is, the preset distance ranges from 2 cm to 4 cm. Within this distance range, the first voice signal and the second voice signal can be more obviously distinguished, without causing excessive difference in characteristics between the two due to the distance being too far, resulting in inaccurate forward voice signals and backward voice signals.

[0051] The beamforming processing method can refer to the existing technology and will not be elaborated here.

[0052] In addition to the forward voice signal and the backward voice signal, in some embodiments of the present application, the multi-channel microphone voice signal also includes a third voice signal collected by the third microphone 111 and a fourth voice signal collected by the fourth microphone 121. Since the third microphone 111 and the fourth microphone 121 are arranged on the left earphone 110 and the right earphone 120, the third voice signal and the fourth voice signal can effectively reflect the noise conditions on both sides of the user, and then the third voice signal and the fourth voice signal can be used to assist in noise reduction.

[0053] S120, calculating the correlation and the first energy ratio between the forward speech signal and the backward speech signal.

[0054] The forward speech signal enhances the forward user speech features and suppresses the backward noise features. The backward speech signal is the opposite, suppressing the forward user speech features and enhancing the backward noise features. The features highlighted by the two are completely different, and when the two are similar, or the energy ratio is close, the features that characterize the two are similar, and there is a high probability that they are the same type of speech activities, for example, both are speech frames or both are noise frames. Therefore, the correlation and the first energy ratio of the forward speech signal and the backward speech signal can be calculated to determine the type of multi-channel microphone speech signals.

[0055] In one embodiment of the present application, the correlation between the forward speech signal and the backward speech signal is calculated by calculating the frequency domain coherence coefficient of the two. For example, the forward speech signal and the backward speech signal can be framed, windowed, and Fourier transformed from the time domain to the frequency domain, and then the frequency domain coherence coefficient of the forward speech signal and the backward speech signal in the frequency domain is calculated.

[0056] In an embodiment provided in the present application, a method for calculating a frequency domain coherence coefficient may include:

[0057]

[0058] Among them, Φ ii (ω),Φ kk () are the auto-power spectral densities of the forward speech signal and the backward speech signal respectively; Φ ik () is the cross power spectral density of the forward speech signal and the backward speech signal; ω is the angular frequency; δ1 is a value greater than 0; ω1 and ω2 are the upper and lower limits of the selected frequency range, respectively, the minimum value of ω1 is 0, and the maximum value of ω2 is 1 / 2 of the FFT frame length, C ik is the frequency domain coherence coefficient. Wherein, δ1 can be a smaller value, for example, it can be 10 -7 , used to avoid the denominator being 0.

[0059] The first energy ratio can be calculated as follows:

[0060]

[0061] Among them, Φ ii () is the auto-power spectrum density of the forward speech signal, Φ kk () is the autopower spectral density of the backward speech signal. Similarly, δ2 is a small value greater than 0, for example, 10 -7 , avoiding division by zero.

[0062] In other embodiments of the present application, the correlation and the first energy ratio may also be calculated by other methods, which are not limited here.

[0063] In an embodiment of the present application, calculating the correlation and the first energy ratio of the forward speech signal and the backward speech signal may include: calculating the correlation and the first energy ratio based on the speech signals in a preset frequency range in the forward speech signal and the backward speech signal; the preset frequency range matches the frequency range of human voice.

[0064] In this embodiment, the preset frequency range matches the frequency range of human voice. The preset frequency range can be the same as the frequency range of human voice, or it can be slightly larger than the frequency range of human voice. For example, the frequency range of human voice is 250Hz~3KHz, and the preset frequency range can be 250Hz~3KHz, or it can be 200Hz~3.2KHz.

[0065] In other embodiments, the intercepted frequency range may also be determined by the sampling range of the first microphone 141 and the second microphone 142. For example, some microphones cannot collect signals higher than 2.5 KHz, and the preset frequency range may be 200 Hz to 2.5 KHz. The frequency range is not limited here.

[0066] In this embodiment, intercepting a speech signal that matches the frequency range of human voice can make the features involved in the calculation of the correlation and the first energy ratio closer to the human voice, thereby improving the effectiveness and reliability of the correlation and the first energy ratio, and further improving the accuracy of voice activity detection.

[0067] S130: Determine types of the multi-channel microphone voice signals based on the correlation and the first energy ratio.

[0068] In one embodiment of the present application, determining the voice activity detection result based on correlation and a first energy ratio may include: if the first energy ratio is less than or equal to a first preset energy ratio threshold, determining that the type corresponding to each voice signal in the multiple microphone voice signals is a noise frame; if the first energy ratio is greater than or equal to a second preset energy ratio threshold, determining that the type corresponding to each voice signal in the multiple microphone voice signals is the voice frame.

[0069] In this embodiment, the second preset energy ratio threshold is greater than the first preset energy ratio threshold. The first preset energy threshold is a smaller value less than 0.5, for example, 0.11, 0.2, 0.3. When the first energy ratio is less than the first preset energy threshold, it indicates that the energy of the backward voice signal is much greater than the energy of the forward voice signal. Therefore, if the first energy ratio is less than or equal to the first preset energy ratio threshold, it is determined that the type corresponding to each voice signal in the multi-channel microphone voice signal is the noise frame.

[0070] The second preset energy threshold is a value greater than 0.5 and less than 1, for example, 0.75, 0.8, 0.9, etc. When the first energy ratio of the forward voice signal and the backward voice signal is greater than the second preset energy threshold, it indicates that the energies of the forward voice signal and the backward voice signal are similar, and they are most likely the same voice frames. Therefore, if the first energy ratio is greater than or equal to the second preset energy ratio threshold, it is determined that the type corresponding to each voice signal of the multi-channel microphone voice signal is a voice frame.

[0071] Among them, the multi-channel microphone voice signals are collected from the same frame of sound, and the type of the multi-channel microphone voice signal also represents the type of the frame of sound. Similarly, the type of the multi-channel microphone voice signal also represents the type of each voice signal in the multi-channel microphone voice signal, that is, the type of the multi-channel microphone voice signal is the type of the forward voice signal and the backward voice signal.

[0072] If the first energy ratio is between the first preset energy ratio threshold and the second preset energy ratio threshold, the type cannot be directly determined. Therefore, there is noise interference and the strength of the interference cannot be determined. Therefore, in an embodiment of the present application, the judgment can be continued in combination with the correlation.

[0073] That is, if the first energy ratio value is between the first preset energy ratio threshold and the second preset energy ratio threshold, it can be judged that: if the correlation is less than the preset correlation threshold, the type corresponding to each voice signal of the multi-channel microphone voice signal is determined to be a noise frame; if the correlation is greater than the preset correlation threshold, the type corresponding to each voice signal of the multi-channel microphone voice signal is determined to be a voice frame.

[0074] In this embodiment, the preset correlation threshold is in the range of 0 to 1. For example, the preset correlation threshold may be any value between 0.7 and 0.9.

[0075] If the first energy ratio is greater than the first preset energy threshold, it indicates that there is a noise signal, and if the correlation is less than the preset correlation threshold, it indicates that the noise signal is strong, so it can be directly determined as a noise frame. On the contrary, if the correlation is greater than the preset correlation threshold, it indicates that the noise signal has little influence, so it can be determined that the type corresponding to each voice signal of the multi-channel microphone voice signal is a voice frame.

[0076] Through the above method, when the first energy ratio is large or small, the speech frame and the noise frame are directly determined, and when the first energy ratio cannot directly determine the speech frame and the noise frame, the judgment is made in combination with the correlation. Through this method, different speech signals can be detected with high accuracy.

[0077] When the first energy ratio is greater than or equal to the first preset energy ratio threshold, it indicates that the energy of the forward voice signal and the backward voice signal are similar, and they are most likely the same voice signals. However, the interference noise emitted from the user side has similar characteristics in the forward and backward directions. Therefore, in this embodiment, the auxiliary microphone can be combined to further determine whether it is a voice frame or a noise frame.

[0078] In one embodiment of the present application, when the first energy ratio is greater than or equal to the first preset energy ratio threshold, the third voice signal and the fourth voice signal can be obtained and the second energy ratio between the third voice signal and the fourth voice signal can be determined; if the second energy ratio is greater than the fifth energy threshold, or the second energy ratio is less than the sixth energy threshold, the type of the multi-channel microphone voice signal is re-determined as a noise frame. The fifth energy threshold is greater than the sixth energy threshold.

[0079] In this embodiment, after the first microphone 141, the second microphone 142, the third microphone 111 and the fourth microphone 121 respectively collect the first voice signal, the second voice signal, the third voice signal and the fourth voice signal, the first voice signal, the second voice signal, the third voice signal and the fourth voice signal can be pre-processed by framing, windowing and Fourier transform to obtain frequency domain signals, and then subsequent processing is performed on the pre-processed frequency domain signals.

[0080] In this embodiment, the second energy ratio is the ratio between the third voice signal and the fourth voice signal, which can be used to characterize the size of the unilateral noise. If the second energy ratio is larger or smaller, it indicates that the side interference noise is larger. At this time, the collected voice signal is likely to be a voice signal of side noise. Therefore, the type of the multi-channel microphone voice signal can be re-determined as a noise frame. The signal that may be a voice frame can be further judged again by the voice signal of the auxiliary microphone, which can improve the accuracy of judging whether it is a voice frame.

[0081] In some embodiments of the present application, before noise reduction is performed, other methods may be combined to assist in determining the accuracy of voice activity detection, for example, using an existing neural network to calculate probability to output a voice activity detection result. The present application does not limit the number of voice activity detection methods used in noise reduction.

[0082] After determining the types of the multi-channel microphone voice signals, voice noise reduction can also be performed. In the embodiment of the present application, noise reduction is mainly performed on the user's voice. As mentioned above, the forward voice signal is mainly a feature of the voice signal of the user's voice. Therefore, in the embodiment of the present application, noise reduction can be performed on the forward voice signal to improve the noise reduction effect.

[0083] In one embodiment, the forward speech signal can be processed separately based on multiple noise suppression modules with different noise suppression methods to obtain the gain output by each noise suppression module; based on a preset gain selection condition, a target gain is determined from the gain output by each noise suppression module; and the forward speech signal is denoised using the target gain.

[0084] In this embodiment, the processing unit is configured with a plurality of noise suppression modules, and different noise suppression modes are configured in different noise suppression modules, so as to determine a gain that is more suitable for noise reduction in the current scene in different scenes.

[0085] In one embodiment, the processing unit further includes an adaptive filter. Before the forward speech signal is input into each noise suppression module, the forward speech signal can also be input into the adaptive filter to obtain the forward speech signal after adaptive filtering. Wherein, if the forward speech signal is a noise frame, the adaptive filter adjusts its own filter coefficient based on the noise frame, and the adaptive filter can be a Wiener filter or other types of adaptive filters, which are not limited here.

[0086] In this embodiment, the adaptive filter can determine whether to update in the following manner:

[0087]

[0088] Where update indicates whether the filter coefficient needs to be updated. When update=1, the filter coefficient needs to be updated. When update=0, no update is required. oth indicates the speech frame. S ik < is a case of noise frame, C ik >&S ik < is another case of a noise frame.

[0089] If the required speech signal exists in the speech signal, that is, when there is a human voice, the update of the adaptive filter parameters will cause the filter to be updated incorrectly, causing the human voice to be distorted. Therefore, in this embodiment, it can be determined whether the filter coefficient needs to be adjusted according to the type in the voice activity detection result, thereby reducing the human voice distortion caused by the adaptive filter.

[0090] The multiple noise suppression modules include: a single microphone noise suppression module and a residual nonlinear echo cancellation module. The single microphone noise suppression module can be a single microphone noise suppressor, an artificial intelligence-based noise suppressor, etc. The residual nonlinear echo cancellation module is used to perform residual nonlinear echo cancellation processing. The noise suppression processing performed by the above modules can refer to the prior art and will not be expanded here.

[0091] In this embodiment, the forward speech signal is processed respectively based on multiple noise suppression modules with different noise suppression modes to obtain the gain output by each noise suppression module, including: inputting the forward speech signal after adaptive filtering into the residual nonlinear echo cancellation module for residual nonlinear echo cancellation processing to obtain a first gain. And, inputting the forward speech signal after adaptive filtering into the single microphone noise suppression module to obtain a second gain output after the single microphone noise suppression module performs noise suppression.

[0092] For speech frames, the noise signal is small, so residual nonlinear echo cancellation can achieve better noise reduction effect. Therefore, the residual nonlinear echo cancellation module can be used to process the forward speech signal to obtain the first gain, so that the subsequent speech frame can have a better noise reduction effect. Similarly, single microphone noise suppression also gives a second gain with a better noise reduction effect.

[0093] Further, in one embodiment, the noise suppression module further includes an energy difference noise suppression module, and the processing unit further includes a noise estimation module; the noise estimation module is configured with an initial noise estimation. In this embodiment, the noise estimation output by the noise estimation module and the forward speech signal after the adaptive filtering can be processed based on the second Wiener filter adaptive filter to obtain a third gain. In some embodiments, the noise estimation module can be a part of the energy difference noise suppression module, which is not limited here.

[0094] In this embodiment, before the noise estimation output by the noise estimation module and the forward speech signal after the adaptive filtering are processed based on the energy difference noise suppression module, the third speech signal and the fourth speech signal corresponding to the same frame of sound can be obtained, and the maximum auxiliary microphone signal with higher energy can be determined from the two; the third energy difference between the forward speech signal and the maximum auxiliary microphone signal is calculated; if the third energy difference is greater than or equal to the third preset energy difference threshold, the noise estimation in the noise estimation module is not updated; if the third energy difference is less than or equal to the fourth preset energy difference threshold, the noise estimation in the noise estimation module is updated based on the maximum auxiliary microphone signal; if the third energy difference is between the third preset energy difference threshold and the fourth preset energy difference threshold, the noise estimation in the noise estimation module is updated based on the forward speech signal. Wherein, the third preset energy difference threshold is greater than the fourth preset energy difference threshold, and both are within 1, for example, the fourth preset energy difference threshold is 0.2, and the third preset energy difference threshold is 0.75.

[0095] In this embodiment, if the third energy difference is less than or equal to the fourth preset energy difference threshold, the current frame noise estimate may be updated based on the maximum auxiliary microphone signal and the previous frame noise estimate.

[0096] Although the forward speech signal can suppress some interfering speech after the first Wiener filter, it is difficult to completely eliminate the interfering signal. Since the wearer's speech intensity picked up by the microphone on the microphone rod 140 is greater than that of the auxiliary microphone, when there is only noise or interfering speech signal, the forward speech signal obtained by the two main microphone signals on the microphone rod 140 after adaptive filtering is smaller than that of the auxiliary microphone. In order to further eliminate the interfering signal and the noise signal, in the embodiment of the present application, the speech signal collected by the auxiliary microphone can be combined for noise reduction.

[0097] In this embodiment, due to the different directions of the noise source, the third microphone 111 and the fourth microphone 121 are arranged on both sides of the user, which makes it possible for there to be a more obvious difference between the third voice signal and the fourth voice signal. Therefore, the voice signal collected closer to the noise source can be selected. The closer to the noise source, the greater the energy of the collected voice signal. Therefore, the maximum auxiliary microphone signal can be determined by the energy of the third voice signal and the fourth voice signal, so as to perform noise reduction based on the maximum auxiliary microphone signal.

[0098] In one embodiment, the third energy difference can be calculated as follows:

[0099]

[0100] Among them, Φ jj (ω),Φ yy () are the auto-power spectral densities of the forward speech signal and the auxiliary microphone speech signal, ΔΦ PLDNE (ω) is the third energy difference.

[0101] Next, the energy difference between the forward voice signal and the maximum auxiliary microphone signal is calculated to obtain the third energy difference. If the third energy difference is greater than the third preset energy difference threshold, it indicates that the forward voice signal is much greater than the maximum auxiliary microphone signal. It is considered that only the wearer's voice signal exists in the collected voice signal at this time, and there is no need to update the noise estimate. When the third energy difference is less than the fourth preset energy difference threshold, then the forward voice signal and the maximum auxiliary microphone signal, that is, there are only noise and interference signals with a high probability. Therefore, the maximum auxiliary microphone signal can be used to update the noise estimate. When the third energy difference is between the third preset energy difference threshold and the fourth preset energy difference threshold, it is considered that this is noisy speech, and the forward voice signal can be used to update the noise estimate. In this way, a noise estimate that is more in line with the current forward voice signal can be obtained, and then a third gain with better noise reduction effect can be obtained. Noise estimation and Wiener filtering can refer to the prior art and will not be expanded here.

[0102] In other embodiments of the present application, more gains may be calculated and used, which is not limited here.

[0103] In an embodiment of the present application, the preset gain selection condition may be to select the minimum gain from multiple gains as the target gain.

[0104] In this embodiment, since the aforementioned gains already have a certain noise reduction effect, and even some gains correspond to stronger noise reduction effects, on this basis, using the maximum gain may cause the user's voice to be distorted during the noise reduction process. Therefore, the minimum gain can be used as the target gain to reduce the possibility of distortion.

[0105] In some embodiments, if the headset is used in a noisy scene, the maximum gain may be selected as the target gain to achieve the maximum noise reduction effect.

[0106] In other embodiments, an intermediate value among multiple gains may also be selected as the target gain to take into account both noise reduction effect and distortion reduction.

[0107] In one embodiment, the noise reduction of the forward speech signal may be performed in the following manner:

[0108] ftF(ω)′=F(ω)*min(G out1 (ω), out2 (ω), out2 (ω)

[0109] Among them, ftF(ω)′ is the frequency band short-time spectrum of the forward speech signal after noise reduction; ftF(ω) is the frequency band short-time spectrum of the forward speech signal; the minimum value of ω is 0, and the maximum value is 1 / 2 of the FFT frame length. out1 (ω), G out2 (ω), G out2 (ω) are the first gain, the second gain and the third gain respectively, min out1 (ω), out2 (ω), out2 (ω)) represents the minimum value among the first gain, the second gain and the third gain. Finally, the signal output in the time domain after the final noise reduction suppression is obtained by inverse Fourier transform, thereby completing the noise reduction of the speech signal.

[0110] For ease of understanding, an example is provided here to illustrate the solution of the present application. In this example, the earphone is a head-mounted earphone, the earphone is connected to a microphone rod 140, a first microphone 141 and a second microphone are arranged on the microphone rod 140, the first microphone 141 and the second microphone 142 are 2 cm apart on the microphone rod 140, the left earphone 110 of the earphone is arranged with a third microphone 111 and the right earphone 120 is arranged with a fourth microphone 121.

[0111] When the headset is working, the first microphone 141, the second microphone 142, the third microphone 111 and the fourth microphone 121 collect sound at the same time, that is, collect the same frame of sound. The first voice signal to the fourth voice signal are obtained respectively, and then the first voice signal to the fourth voice signal are framed, windowed and Fourier transformed to obtain the corresponding frequency domain signals. Then, the first voice signal and the second voice signal in the frequency domain are processed by forward and backward beamforming in turn to obtain the forward voice signal and the backward voice signal.

[0112] Then, voice activity detection is performed based on the forward voice signal and the backward voice signal. The correlation and the first energy ratio between the two are first calculated, and then compared with the first preset energy threshold, the second preset energy threshold and the preset correlation threshold to determine whether this frame of sound is a noise frame or a voice frame. For details, please refer to the above content and will not be expanded here.

[0113] After determining the noise frame or the speech frame, noise reduction can be performed. In one embodiment of the present application, the forward speech signal can be first input into an adaptive filter for filtering. The adaptive filter can adjust its own filter coefficient according to whether the input signal is a noise frame to adjust the filtering ability of different speech signals. Then, the speech signal after adaptive filtering is respectively input into the residual nonlinear echo cancellation module, the single microphone noise suppression module and the energy difference noise suppression module to obtain the first gain, the second gain and the third gain output by each suppression module. Finally, the third gain is used to reduce the noise of the forward speech signal, and the noise reduction result is inverse Fourier transformed to obtain the noise reduction result in the time domain, and finally the noise reduction result in the time domain is output.

[0114] Based on the same inventive concept, the present application also provides a voice activity detection device, see Figure 3 , Figure 3 The figure is a schematic diagram of a voice activity detection device provided by an embodiment of the present application. The voice activity detection device 200 can be applied to a processing unit of a headset, and the voice activity detection device 200 includes: an acquisition module 210 and a voice activity detection module 220 .

[0115] The acquisition module 210 is used to obtain multi-channel microphone voice signals corresponding to the same frame of sound. One frame of multi-channel microphone voice signals includes a forward voice signal and a backward voice signal corresponding to the same frame of sound. The forward voice signal is a voice signal in the propagation direction of the preset user voice, and the backward voice signal is a voice signal in the opposite direction of the preset user voice propagation direction.

[0116] The voice activity detection module 220 is used to calculate the correlation and the first energy ratio of the forward voice signal and the backward voice signal; determine the type of the multi-channel microphone voice signal based on the correlation and the first energy ratio, and the type of the multi-channel microphone voice signal is a voice frame or a noise frame.

[0117] The earphones are head-mounted earphones, which include a microphone rod 140, on which a first microphone 141 and a second microphone 142 are arranged; an acquisition module includes a beamforming module 211, and the acquisition module 210 is specifically used to acquire a first voice signal collected by the first microphone 141 on the microphone rod 140 and a second voice signal collected by the second microphone 142 when collecting the same frame of sound; forward beamforming processing is performed on the first voice signal and the second voice signal according to the beamforming module 211 to obtain a forward voice signal; and backward beamforming processing is performed on the first voice signal and the second voice signal according to the beamforming module 211 to obtain a backward voice signal.

[0118] The voice activity detection module 220 is specifically used for: if the first energy ratio is less than or equal to the first preset energy ratio threshold, then determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is a noise frame; if the first energy ratio is greater than or equal to the second preset energy ratio threshold, then determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is a voice frame; the second preset energy ratio threshold is greater than the first preset energy ratio threshold; if the first energy ratio is between the first preset energy ratio threshold and the second preset energy ratio threshold, then judging: if the correlation is less than the preset correlation threshold, then determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is a noise frame; if the correlation is greater than the preset correlation threshold, then determining that the type corresponding to each voice signal in the multi-channel microphone voice signal is a voice frame.

[0119] The earphone includes a left earphone 110, a right earphone 120, a third microphone 111 and a fourth microphone 121; the third microphone 111 is arranged on the left earphone, the fourth microphone 121 is arranged on the right earphone, and the multi-channel microphone voice signal also includes a third voice signal collected by the third microphone 111 and a fourth voice signal collected by the fourth microphone 121. The voice activity detection module 220 can be specifically used to obtain the third voice signal and the fourth voice signal and determine the second energy ratio between the third voice signal and the fourth voice signal when the first energy ratio is greater than or equal to the first preset energy ratio threshold; if the second energy ratio is greater than the fifth energy threshold, or the second energy ratio is less than the sixth energy threshold, then the type of the multi-channel microphone voice signal is re-determined to be a noise frame; the fifth energy threshold is greater than the sixth energy threshold.

[0120] The voice activity detection module 220 may be specifically configured to calculate the correlation and the first energy ratio based on the voice signals in a preset frequency range in the forward voice signal and the backward voice signal; the preset frequency range matches the frequency range of human voice.

[0121] The voice activity detection device 200 also includes a plurality of noise suppression modules with different noise suppression modes. The voice activity detection module 220 is also used to output the forward voice signal to the plurality of noise suppression modules with different noise suppression modes for processing, and obtain the gain output by each noise suppression module; the noise reduction module determines the target gain from the gain output by each noise suppression module based on the preset gain selection condition; and uses the target gain to reduce the noise of the forward voice signal.

[0122] The voice activity detection device 200 also includes an adaptive filter 230, and the multiple noise suppression modules include: a single microphone noise suppression module 242 and a residual nonlinear echo elimination module 241; the voice activity detection module 220 is also used to input the forward voice signal into the adaptive filter 230 to obtain the forward voice signal after adaptive filtering; wherein, if the forward voice signal is a noise frame, the adaptive filter adjusts its own filter coefficient based on the noise frame.

[0123] The adaptive filter 230 is also used to input the forward speech signal after adaptive filtering into the residual nonlinear echo elimination module 241 for residual nonlinear echo elimination processing to obtain a first gain; and output the forward speech signal after adaptive filtering to the single microphone noise suppression module 242 to obtain a second gain output by the single microphone noise suppression module after noise suppression.

[0124] The multiple noise suppression modules also include: an energy difference noise suppression module 243; the processing unit also includes a noise estimation module 244; the energy difference noise suppression module 243 is used to process the noise estimation output by the noise estimation module 244 and the forward speech signal after adaptive filtering to obtain a third gain.

[0125] The acquisition module 210 is also used to acquire the third voice signal and the fourth voice signal corresponding to the same frame of sound. The noise estimation module 244 is specifically used to determine the maximum auxiliary microphone signal with higher energy from the third voice signal and the fourth voice signal; calculate the third energy difference between the forward voice signal and the maximum auxiliary microphone signal; if the third energy difference is greater than or equal to the third preset energy difference threshold, the noise estimation in the noise estimation module 244 is not updated; if the third energy difference is less than or equal to the fourth preset energy difference threshold, the noise estimation in the noise estimation module 244 is updated based on the maximum auxiliary microphone signal; if the third energy difference is between the third preset energy difference threshold and the fourth preset energy difference threshold, the noise estimation in the noise estimation module 244 is updated based on the forward voice signal.

[0126] The noise reduction module 250 may be specifically used for a preset gain selection condition of selecting a minimum gain from multiple gains as a target gain.

[0127] Based on the same inventive concept, an embodiment of the present application further provides a readable storage medium, in which instructions are stored. The instructions can be executed by one or more processing units to implement the voice activity detection method provided in the above embodiment.

[0128] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD (digital video disc)), or a semiconductor medium (e.g., an SSD (Solid State Disk)).

[0129] If the control method of voice activity detection is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The functional modules in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0131] The above embodiments can be freely combined without conflict, and the embodiments obtained by the combination are included in the protection scope of this application.

[0132] The above detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0133] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0134] In the description of this application, it is also necessary to explain that, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0135] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A voice activity detection method, characterized in that: Applied to a processing unit of a headset, the voice activity detection method comprises: Acquire multiple microphone voice signals corresponding to the same frame of sound, wherein one frame of the multiple microphone voice signals includes a forward voice signal and a backward voice signal corresponding to the same frame of sound, wherein the forward voice signal is a voice signal in a preset user voice propagation direction, and the backward voice signal is a voice signal in a direction opposite to the preset user voice propagation direction; Calculating the correlation and the first energy ratio of the forward speech signal and the backward speech signal; The type of the multi-channel microphone voice signal is determined based on the correlation and the first energy ratio, and the type of the multi-channel microphone voice signal is a voice frame or a noise frame.

2. The voice activity detection method according to claim 1, characterized in that: The earphone is a headset, and the headset comprises a microphone rod, on which a first microphone and a second microphone are arranged; The step of obtaining the multi-channel microphone voice signals corresponding to the same frame of sound includes: Acquire a first voice signal collected by the first microphone and a second voice signal collected by the second microphone on the microphone rod when collecting the same frame of sound; Performing forward beamforming processing on the first speech signal and the second speech signal to obtain the forward speech signal; Performing backward beamforming processing on the first speech signal and the second speech signal to obtain the backward speech signal.

3. The voice activity detection method according to claim 2, characterized in that: On the microphone pole, the first microphone and the second microphone are spaced apart by a preset distance.

4. The voice activity detection method according to claim 3, characterized in that: The preset distance ranges from 2 cm to 4 cm.

5. The voice activity detection method according to claim 1, characterized in that: The determining of the voice activity detection result based on the correlation and the first energy ratio includes: If the first energy ratio is less than or equal to a first preset energy ratio threshold, determining that the type corresponding to each voice signal in the multi-channel microphone voice signals is the noise frame; If the first energy ratio is greater than or equal to a second preset energy ratio threshold, determining that the type corresponding to each voice signal of the multi-channel microphone voice signal is the voice frame; and the second preset energy ratio threshold is greater than the first preset energy ratio threshold; If the first energy ratio value is between the first preset energy ratio threshold and the second preset energy ratio threshold, determine: If the correlation is less than a preset correlation threshold, determining that the type corresponding to each voice signal of the multiple microphone voice signals is the noise frame; If the correlation is greater than the preset correlation threshold, it is determined that the type corresponding to each voice signal of the multiple microphone voice signals is the voice frame.

6. The voice activity detection method according to claim 5, characterized in that: The earphone comprises a left earphone, a right earphone, a third microphone and a fourth microphone; the third microphone is arranged on the left earphone, and the fourth microphone is arranged on the right earphone; The multi-channel microphone voice signal also includes a third voice signal collected by the third microphone and a fourth voice signal collected by the fourth microphone; The voice activity detection method further comprises: When the first energy ratio is greater than or equal to the first preset energy ratio threshold, acquiring the third speech signal and the fourth speech signal and determining a second energy ratio between the third speech signal and the fourth speech signal; If the second energy ratio is greater than the fifth energy threshold, or the second energy ratio is less than the sixth energy threshold, the type of the multi-channel microphone voice signal is re-determined as the noise frame; the fifth energy threshold is greater than the sixth energy threshold.

7. The voice activity detection method according to claim 1, characterized in that: Calculating the correlation and the first energy ratio of the forward speech signal and the backward speech signal, comprising: The correlation and the first energy ratio are calculated based on speech signals in a preset frequency range in the forward speech signal and the backward speech signal; the preset frequency range matches the frequency range of human voice.

8. The voice activity detection method according to any one of claims 1 to 7, characterized in that: The processing unit is configured with a plurality of noise suppression modules having different noise suppression modes; After determining the types of the multi-channel microphone voice signals based on the correlation and the first energy ratio, the voice activity detection method further includes: Processing the forward speech signal respectively based on the plurality of noise suppression modules with different noise suppression modes to obtain a gain output by each noise suppression module; Based on a preset gain selection condition, determining a target gain from the gains output by each of the noise suppression modules; The forward speech signal is denoised using the target gain.

9. The voice activity detection method according to claim 8, characterized in that: The processing unit further comprises an adaptive filter, and the plurality of noise suppression modules comprises: a single microphone noise suppression module and a residual nonlinear echo cancellation module; Before the forward speech signal is processed respectively based on the plurality of noise suppression modules having different noise suppression modes, the method further includes: Inputting the forward speech signal into the adaptive filter to obtain the forward speech signal after adaptive filtering; wherein, if the forward speech signal is the noise frame, the adaptive filter adjusts its own filter coefficient based on the noise frame; The method of processing the forward speech signal based on the plurality of noise suppression modules with different noise suppression modes respectively to obtain the gain output by each noise suppression module comprises: Inputting the adaptively filtered forward speech signal into the residual nonlinear echo cancellation module for residual nonlinear echo cancellation processing to obtain a first gain; The forward speech signal after the adaptive filtering is output to the single-microphone noise suppression module to obtain a second gain output by the single-microphone noise suppression module after noise suppression.

10. The voice activity detection method according to claim 9, characterized in that: The plurality of noise suppression modules further include: an energy difference noise suppression module; the processing unit further includes a noise estimation module; The method further comprises: processing the forward speech signal based on the plurality of noise suppression modules with different noise suppression modes respectively to obtain the gain output by each noise suppression module, and further comprising: processing the noise estimation output by the noise estimation module and the forward speech signal after the adaptive filtering based on the energy difference noise suppression module to obtain a third gain; The earphone further comprises a left earphone, a right earphone, a third microphone and a fourth microphone; the third microphone is arranged on the left earphone, and the fourth microphone is arranged on the right earphone; the multi-channel microphone voice signal further comprises a third voice signal collected by the third microphone and a fourth voice signal collected by the fourth microphone; The method also includes: before processing the noise estimate output by the noise estimation module and the forward speech signal after the adaptive filtering based on the energy difference noise suppression module, obtaining the third speech signal and the fourth speech signal corresponding to the same frame of sound, and determining the maximum auxiliary microphone signal with higher energy from the two; calculating the third energy difference between the forward speech signal and the maximum auxiliary microphone signal; if the third energy difference is greater than or equal to the third preset energy difference threshold, not updating the noise estimate in the noise estimation module; if the third energy difference is less than or equal to the fourth preset energy difference threshold, updating the noise estimate in the noise estimation module based on the maximum auxiliary microphone signal; if the third energy difference is between the third preset energy difference threshold and the fourth preset energy difference threshold, updating the noise estimate in the noise estimation module based on the forward speech signal.

11. The voice activity detection method according to claim 8, characterized in that: The preset gain selection condition is to select the minimum gain from multiple gains as the target gain.

12. A headset, characterized in that: include: Microphone, used to collect sound; A processing unit, connected to the microphone, is used to execute the voice activity detection method according to any one of claims 1 to 11.

13. The headset according to claim 12, characterized in that: The headset further comprises a microphone rod, on which a first microphone and a second microphone are arranged; When collecting the same frame of sound, the first microphone is used to collect a first voice signal and the second microphone is used to collect a second voice signal; The processing unit is used to perform forward beamforming processing on the first voice signal and the second voice signal to obtain the forward voice signal; And, performing backward beamforming processing on the first speech signal and the second speech signal to obtain the backward speech signal.

Images