Self-talking detection method and device, storage medium and earphone equipment

Abstract

The embodiment of the invention discloses a self-talking detection method and device, a storage medium and earphone equipment. The method comprises the following steps: receiving a first voice signal and a second voice signal through a first microphone and a second microphone, and aligning the intensity of the first voice signal and the intensity of the second voice signal, and calculating a single-frame energy value of the first voice signal, and a low-frequency energy difference value and a correlation value of the first voice signal and the second voice signal, and determining whether the first voice signal is a self-talking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value. According to the scheme provided by the embodiment of the invention, the two voice signals are acquired through the two microphones in the earphone, and the energy values, the low-frequency-band energy difference and the high-frequency-band correlation of the two signals are analyzed, so that a higher-precision self-talking detection effect is realized.

Description

Technical Field

[0001] The present application relates to the technical field of audio signals, and in particular to a self-talk detection method, device, storage medium and headphone device. Background Art

[0002] With the progress of society and the improvement of people's living standards, headphones have become an indispensable companion in daily life. In particular, TWS (True Wireless Stereo) headphones have been widely recognized in the market for their convenience, compactness, and ease of use. If TWS headphones can detect that the wearer is talking and control music playback, active noise reduction, and external transparency mode, etc., it can significantly improve the wearer's experience in conversation, singing, and reading.

[0003] In view of the above situation, there are two main implementation schemes in the existing technology. One is to collect as much data as possible by using multiple sensors (such as microphones, bone conduction sensors and accelerometers, etc.) for fusion judgment. The typical approach is to combine the signal of the microphone array and the signal of the bone conduction sensor to first determine whether there is speech corresponding to the bone vibration, and then use the accelerometer to detect the head vibration to further confirm the speech state. Another scheme is to continuously improve the detection algorithm to enhance the information extraction capability, such as time domain feature analysis, frequency domain feature analysis and speech feature analysis in the field of signal processing, and subsequent deep learning methods represented by deep neural networks (DNN) and convolutional neural networks (CNN).

[0004] The applicant found that multi-sensor fusion can indeed improve detection accuracy, but it requires more sensors, which increases hardware costs, system complexity, and device power consumption. Deep learning algorithms require a large amount of data to train models, and the model size is generally large, occupying a large storage space and power consumption, which is unacceptable in small embedded devices such as headphones. Therefore, in practical applications, algorithms based on a small number of sensors and low computational complexity are more needed to achieve high-precision self-talk detection. Summary of the invention

[0005] The embodiments of the present application provide a self-talk detection method, apparatus, storage medium and headphone device, which obtain two voice signals through two microphones in the headphone, analyze the energy values, low-frequency energy differences and high-frequency correlations of the two signals, and thereby achieve a more accurate self-talk detection effect.

[0006] The embodiment of the present application provides a self-talk detection method, which is applied to a headphone device, wherein the headphone device includes a first microphone and a second microphone, including:

[0007] receiving a first voice signal and a second voice signal through the first microphone and the second microphone;

[0008] Aligning the strengths of the first speech signal and the second speech signal;

[0009] Calculating a single-frame energy value of the first speech signal, a low-frequency energy difference value between the first speech signal and the second speech signal, and a correlation value;

[0010] Determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value.

[0011] In one embodiment, aligning the strengths of the first speech signal and the second speech signal includes:

[0012] Acquire an arrangement of the first microphone and the second microphone;

[0013] calculating an alignment coefficient according to an arrangement of the first microphone and the second microphone;

[0014] The strength of the first speech signal is weakened according to the alignment coefficient.

[0015] In one embodiment, determining whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value, and the correlation value includes:

[0016] If the single-frame energy value is greater than the energy threshold, determining whether the current speech frame of the first speech signal is a self-speaking speech frame according to the low-frequency energy difference value;

[0017] If the single-frame energy value is not greater than the energy threshold, determining whether the current speech frame of the first speech signal is a self-speaking speech frame according to the correlation value.

[0018] In one embodiment, the method further comprises:

[0019] When it is determined that the current speech frame of the first speech signal is a self-speaking speech frame, obtaining a target number of self-speaking speech frames determined as the self-speaking speech frames in a subsequent first preset number of continuous speech frames;

[0020] Determining whether the target quantity is greater than a second preset quantity;

[0021] If yes, it is determined that the first speech signal is a self-speaking signal.

[0022] In one embodiment, the calculation process of the single-frame energy value of the first speech signal includes:

[0023] Obtain the energy spectral density S of the first speech signal T (f):

[0024] S T (f)=|T′(f)| 2

[0025] Wherein, T′(f) is the single frame signal spectrum of the first speech signal after alignment processing;

[0026] Calculate the single frame energy value E of the first speech signal T :

[0027] E T =∑ f＝0 S T (f).

[0028] In one embodiment, the process of calculating the low-frequency energy difference between the first speech signal and the second speech signal includes:

[0029] Obtain the energy spectral density S of the second speech signal F (f):

[0030] S F (f)=|F(f)| 2

[0031] Wherein, F(f) is the single-frame signal spectrum of the second speech signal;

[0032] Calculate the low-frequency energy difference ΔE between the first speech signal and the second speech signal:

[0033]

[0034] Among them, f min ~f max is the low frequency band range for calculating the energy difference.

[0035] In one embodiment, the calculation process of the correlation value between the first speech signal and the second speech signal includes:

[0036] Obtain the cross power spectrum S of the first speech signal and the second speech signal TF (f):

[0037]

[0038] in, YesS F The conjugate complex number of (f) is used to calculate the cross-spectral coherence function C by the following formula: TF (f):

[0039]

[0040] Among them, C TF The value range of (f) is between [0,1]. TF When (f) = 1, it means that the first speech signal and the second speech signal are highly correlated. TFWhen (f)=0, it indicates that there is no correlation between the first speech signal and the second speech signal. The correlation value C of the first speech signal and the second speech signal is calculated by the following formula:

[0041]

[0042] Among them, f min2 ~f max2 is the high frequency band range for calculating the correlation value, and N is the corresponding number of frequency points.

[0043] The embodiment of the present application further provides a self-talk detection device, which is applied to an earphone device, wherein the earphone device includes a first microphone and a second microphone, including:

[0044] A receiving unit, configured to receive a first voice signal and a second voice signal through the first microphone and the second microphone;

[0045] an alignment unit, configured to align the strengths of the first speech signal and the second speech signal;

[0046] A calculation unit, used to calculate a single-frame energy value of the first speech signal, a low-frequency energy difference value between the first speech signal and the second speech signal, and a correlation value;

[0047] A determination unit is used to determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value.

[0048] The embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed on a computer, the computer executes the self-speaking detection method as described above.

[0049] An embodiment of the present application further provides an earphone device, comprising a memory and a processor, wherein the processor executes the self-talk detection method as described above by calling a computer program stored in the memory.

[0050] The self-talk detection method, device, storage medium and headphone device provided in the embodiments of the present application can receive the first voice signal and the second voice signal through the first microphone and the second microphone, align the strengths of the first voice signal and the second voice signal, calculate the single-frame energy value of the first voice signal, the low-frequency energy difference value and the correlation value between the first voice signal and the second voice signal, and determine whether the first voice signal is a self-talk signal based on the single-frame energy value, the low-frequency energy difference value and the correlation value. The solution provided in the embodiments of the present application obtains two voice signals through two microphones in the headphones, analyzes the energy value, low-frequency energy difference and high-frequency correlation of the two signals, and thereby achieves a more accurate self-talk detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of a self-talk detection method provided in an embodiment of the present application.

[0053] Figure 2 A schematic diagram of the structure of the earphone device provided in an embodiment of the present application.

[0054] Figure 3 A second flow chart of the self-talk detection method provided in an embodiment of the present application.

[0055] Figure 4 A schematic diagram of the structure of a self-talk detection device provided in an embodiment of the present application.

[0056] Figure 5 Another structural schematic diagram of the earphone device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0058] It should be noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the existence of other identical elements in the process, method, article or device that includes the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0059] The embodiments of the present application provide a self-speaking detection method, device, storage medium and headphone device. Specifically, the self-speaking detection method of the embodiments of the present application can be executed by a computer device or a server, wherein the computer device can be a terminal. The terminal can be a smart phone, a tablet computer, a laptop computer, a touch screen, a game console, a personal computer (PC, Personal Computer), a personal digital assistant (Personal Digital Assistant, PDA), a smart home and other devices, and the terminal can also include a client, which can be a media playback client or an instant messaging client, etc.

[0060] See also Figure 1 , the specific process of the self-speaking detection method can be as follows:

[0061] Step 101: receiving a first voice signal and a second voice signal through a first microphone and a second microphone.

[0062] In one embodiment, the self-talk detection method is applied to a headphone device, the headphone device includes a first microphone and a second microphone, for details, see Figure 2 , the headset 100 can be a wireless headset or a wired headset, and the first microphone 110 can be a Talk microphone, that is, a call microphone, which is mainly used to capture and transmit the user's voice during a voice call. In the design of the headset, the Talk microphone is usually placed close to the user's mouth to ensure that the user's voice can be clearly captured. This microphone plays an important role in application scenarios such as calls, voice chats, and video conferencing. The second microphone 120 can be a FF microphone, that is, a feedforward microphone, which is mainly used in active noise reduction (ANC) headsets to capture external noise signals. The FF microphone is usually placed on the outside of the headset and can capture all the noise in the external environment of the headset. By capturing these noise signals, the FF microphone can work with the noise reduction system to generate reverse sound waves equal to the external noise, neutralize the noise, and thus achieve the effect of noise reduction.

[0063] In one embodiment, after the first voice signal and the second voice signal are received by the first microphone and the second microphone, noise reduction filtering can be performed to filter out environmental noise, further reduce the interference of background noise, and improve call quality. For example, a dual-microphone array is formed by the first microphone (Talk Mic) and the second microphone (FF Mic), and the sound signal is weighted, delayed, and processed by a beamforming algorithm, thereby enhancing the sound signal in the target direction and suppressing the sound signal in the non-target direction, so as to achieve a clear call experience.

[0064] Furthermore, when filtering the first speech signal and the second speech signal, methods such as Wiener filtering and adaptive filtering can be used. Among them, Wiener filtering is a linear filtering method based on the minimum mean square error criterion, and its purpose is to estimate the original signal as accurately as possible from the observed signal in the presence of noise interference. Specifically, the filter coefficient can be calculated according to the Wiener-Hoff equation, and then the obtained filter coefficient is convolved with the first speech signal or the second speech signal to obtain the filtered signal. Adaptive filtering is a filtering technology that can automatically adjust the filter parameters to adapt to the changes in signals and noise. In the headphone microphone signal noise reduction, adaptive filtering can continuously optimize the performance of the filter according to the real-time changes of the signal and noise, so as to better filter out the noise. Specifically, the filter parameters can be initialized first, and then at each sampling moment, the input signal (i.e., the first speech signal or the second speech signal) and the reference signal are obtained, and the output of the filter is calculated, thereby calculating the error signal, which reflects the difference between the filter output and the expected signal (reference signal). Finally, according to the adaptive algorithm, the error signal and the input signal are used to update the filter coefficients. After repeating the above steps, the filter parameters can be continuously adjusted to make the filter adapt to the changes in the signal and noise, thereby achieving effective noise reduction for the first speech signal or the second speech signal.

[0065] Step 102: align the strengths of the first speech signal and the second speech signal.

[0066] In one embodiment, the Talk microphone is arranged at the bottom of the earphone rod, close to the user's mouth, and the FF microphone is arranged on the upper outer side of the earphone rod. Figure 2 As shown in the figure, if the Talk microphone signal strength is much higher than the FF microphone signal strength, the subsequent calculation result of the energy difference may be dominated by the first voice signal, resulting in an inability to accurately reflect the relationship between the wearer's voice and environmental noise or other interfering sounds. By weakening the first voice signal to align with the second voice signal strength, the energy difference calculation can be made more balanced and feature information related to the wearer's voice can be extracted more effectively.

[0067] Furthermore, factors such as the positional relationship, distance, sensitivity of the Talk microphone and the FF microphone, and the acoustic structure of the headset will affect the initial intensity relationship between the first voice signal and the second voice signal. Different headphone designs may result in different differences in the signal strengths collected by the Talk microphone and the FF microphone under the same sound input. For example, the acoustic cavity design of the headset may affect the propagation and collection of sound. If the acoustic environment where the Talk microphone is located makes it have a stronger amplification effect on the human voice signal, and the acoustic characteristics of the FF microphone cause the sound it collects to be relatively weak, then it is necessary to determine the degree of attenuation of the Talk microphone signal based on this design difference. In this embodiment, the arrangement of the Talk microphone and the FF microphone can be obtained first, and then the alignment coefficient can be calculated based on the arrangement of the Talk microphone and the FF microphone, and then the intensity of the first voice signal can be weakened based on the alignment coefficient.

[0068] Among them, the above-mentioned alignment coefficient can be calculated and measured in a laboratory environment. For example, after establishing a test environment, an audio playback device is arranged in the environment to play sound signals of different frequencies and intensities, simulating the wearer's speech sound and environmental noise. Wear the TWS headset to be calibrated on the tester's head, and collect the signals of the Talk microphone and FF microphone at the same time under different experimental environments. For the collected signals, preprocessing is first performed, such as removing DC bias, performing appropriate filtering and other operations to improve the signal quality. Then analyze the intensity characteristics of the dual-microphone signals in different sound scenes, and calculate the energy value, frequency distribution and other parameters of the dual-microphone signals when the wearer simulates speaking and not speaking. Finally, based on the experimental data, find the Talk microphone signal attenuation coefficient that enables the dual-microphone signals to be well aligned in various scenarios.

[0069] Step 103, calculating the single-frame energy value of the first speech signal, the low-frequency energy difference value and the correlation value between the first speech signal and the second speech signal.

[0070] In one embodiment, the single-frame energy value of the first voice signal reflects the energy of the voice signal in the frame and is a measure of the strength of the voice signal. In voice processing, the energy value can be used as an important basis for judging whether a voice signal exists and its strength. For the first voice signal (Talk microphone signal), the purpose of calculating its single-frame energy value is to obtain the energy information contained in the signal in the current frame, so as to combine it with other parameters (such as low-frequency energy difference, correlation value, etc.) to judge whether it is the wearer's own speech.

[0071] The purpose of calculating the low-frequency energy difference is to use the energy difference between the Talk microphone signal and the FF microphone signal in the low-frequency band to determine whether there is a wearer's voice. Generally, the wearer's voice has relatively strong energy in the low-frequency band, while the energy distribution of interference factors such as environmental noise in the low-frequency band is different from that of the wearer's voice. This embodiment can highlight the characteristics of the wearer's voice and reduce the impact of environmental noise and other factors on self-talk detection by calculating the energy difference between the two in the low-frequency band.

[0072] The high-frequency correlation value is used to measure the similarity between the Talk microphone signal and the FF microphone signal in the high-frequency band. In self-talk detection, when the wearer speaks, due to the influence of the sound propagation path and acoustic characteristics, the Talk microphone signal and the FF microphone signal will show a certain correlation in the high-frequency band, while the correlation of interference factors such as environmental noise in the high-frequency band is usually weak. Therefore, this embodiment can assist in determining whether the wearer is talking by himself by calculating the high-frequency correlation value, thereby improving the accuracy of detection.

[0073] Step 104: Determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value, and the correlation value.

[0074] In one embodiment, if the single-frame energy value is greater than the energy threshold, whether the current speech frame of the first speech signal is a self-speaking speech frame is determined based on the low-frequency energy difference; if the single-frame energy value is not greater than the energy threshold, whether the current speech frame of the first speech signal is a self-speaking speech frame is determined based on the correlation value.

[0075] Specifically, the single-frame energy value is used as the basis for preliminary judgment of the strength of the voice signal. If the single-frame energy value is too large or too small, it may indicate that the signal is abnormal (for example, if it is too large, it may be strong noise interference, and if it is too small, it may be a weak or lost signal). It can be further analyzed in combination with other parameters in subsequent judgments. The low-frequency energy difference is mainly used to determine whether there is wearer voice at a high volume. When the single-frame energy value is greater than the energy threshold, the low-frequency energy difference is used as the basis. If the low-frequency energy difference is greater than 0, the current voice frame is judged to be a self-speaking voice frame, because the energy advantage of the wearer's voice in the low-frequency band is more obvious at this time. Correspondingly, if the low-frequency energy difference is less than or equal to 0, the current voice frame is judged to be a non-self-speaking voice frame. The high-frequency correlation value plays an important role in low-volume conditions. When the single-frame energy value is less than the energy threshold, it indicates that the human voice is relatively small and the energy difference analysis is prone to misjudgment. At this time, the high-frequency correlation value is used as the basis. If it is within the preset range, the current voice frame is judged to be a self-speaking voice frame. This is because at a low volume, the correlation characteristics of the wearer's voice in the high-frequency band are more prominent, which can be used to make up for the shortcomings of the low-frequency energy difference judgment. Correspondingly, if it is not within the above range, the current voice frame is judged to be a non-self-speaking voice frame.

[0076] In one embodiment, in order to eliminate interference from environmental noise and the wearer's cough, etc., it is also necessary to analyze multiple consecutive frames of speech. Specifically, when it is confirmed that the current speech frame of the first speech signal is a self-speaking speech frame, the target number of self-speaking speech frames in the subsequent first preset number of consecutive speech frames is obtained, and it is determined whether the target number is greater than the second preset number. If so, the first speech signal is determined to be a self-speaking signal. For example, when the frame of speech is determined to be self-speaking, the results of the subsequent L consecutive frames of speech are paid attention to. If the total number of self-speaking speech frames in the L frames is greater than L1, the first speech signal is determined to be a self-speaking signal, that is, the headphone device is currently in the wearer's self-speaking state. If the total number of self-speaking speech frames in the L frames is less than L1, the first speech signal is determined to be a non-self-speaking signal.

[0077] The value of L affects the delay of the self-talk detection algorithm. If L takes a larger value, the total number of frames analyzed is larger, and the self-talk is judged to be slower. Correspondingly, the value of L1 affects the sensitivity of the self-talk detection algorithm. If L1 takes a larger value, the self-talk judgment requirement is more stringent, and it is difficult to judge the wearer's short sentences as self-talk. If L1 takes a smaller value, the self-talk judgment requirement is looser, and the wearer's cough, external interference voice and environmental noise may be mistakenly judged as self-talk. Therefore, the above factors can be considered when setting the values ​​of L and L1.

[0078] As can be seen from the above, the self-talk detection method provided in the embodiment of the present application can receive the first voice signal and the second voice signal through the first microphone and the second microphone, align the strengths of the first voice signal and the second voice signal, calculate the single-frame energy value of the first voice signal, the low-frequency energy difference value and the correlation value of the first voice signal and the second voice signal, and determine whether the first voice signal is a self-talk signal based on the single-frame energy value, the low-frequency energy difference value and the correlation value. The solution provided in the embodiment of the present application obtains two voice signals through two microphones in the headset, analyzes the energy value, low-frequency energy difference and high-frequency correlation of the two signals, and thereby achieves a more accurate self-talk detection effect.

[0079] See also Figure 3 , is another flow chart of the self-talk detection method provided in the embodiment of the present application. The specific flow of the method can be as follows:

[0080] Step 201: Receive a first voice signal and a second voice signal through a first microphone and a second microphone.

[0081] Step 202: align the strengths of the first speech signal and the second speech signal.

[0082] Specifically, firstly, a spectrum analysis is performed on a single frame signal, i.e., a first voice signal, collected by the Talk microphone, i.e., the first microphone, to obtain its spectrum T(f). Correspondingly, a spectrum analysis is performed on a single frame signal, i.e., a second voice signal, collected by the FF microphone, i.e., the second microphone, to obtain a spectrum F(f). The above spectrum analysis can be implemented by using a signal processing technique such as Fourier transform to convert a time domain signal into a frequency domain signal so as to process and analyze the signal in the frequency domain.

[0083] Then, the spectrum of the first speech signal is aligned according to the alignment coefficient A to obtain the attenuated spectrum of the first speech signal T′(f), which is calculated as follows:

[0084]

[0085] Among them, T′(f) is the first speech signal after alignment processing, and A is the alignment coefficient, which is determined by the dual microphone arrangement of the headset. This operation directly adjusts the amplitude of the first speech signal in the frequency domain, achieving the intensity reduction determined by the arrangement. Since the intensity of the first speech signal has been adjusted to a level similar to that of the second speech signal, the subsequent energy difference calculation results can better reflect the characteristics of the wearer's voice, reducing the energy difference calculation deviation caused by signal strength imbalance, thereby improving the accuracy of judging the wearer's voice based on energy difference in the self-talk detection algorithm.

[0086] Step 203: Acquire the energy spectral density of the first speech signal, and calculate the single-frame energy value of the first speech signal according to the energy spectral density of the first speech signal.

[0087] Specifically, the energy spectral density S of the first speech signal T The calculation formula of (f) is as follows:

[0088] S T (f)=|T′(f)| 2

[0089] The single frame energy value E T It is the integral of the energy spectral density over the entire frequency range and is calculated as:

[0090] E T =Σ f＝0 S T (f)

[0091] Since the frequency range is discrete in actual calculation, the integral is approximated by summing up. The physical meaning of this formula is to add up the energy contribution of the first speech signal at each frequency to obtain the total energy of the frame signal.

[0092] Step 204: Obtain the energy spectral density of the second speech signal, and calculate the low-frequency energy difference according to the energy spectral density of the first speech signal and the energy spectral density of the second speech signal.

[0093] Correspondingly, the energy spectral density S of the second speech signal F The calculation formula of (f) is as follows:

[0094] S F (f)=|F(f)| 2

[0095] Then the low-frequency energy difference ΔE can be calculated by the following formula:

[0096]

[0097] Among them, f min ~f max To calculate the low-frequency band range of the energy difference, this formula indicates that within the selected low-frequency band range, the energy spectral density of the first voice signal is subtracted from the energy spectral density of the second voice signal, and the difference at each frequency is accumulated, and the result is the low-frequency energy difference. If ΔE>0, it means that the energy of the first voice signal in the low-frequency band is higher than that of the second voice signal, and the wearer's voice may exist; conversely, if ΔE≤0, the possibility of the wearer's voice existing is small, and this embodiment needs to be further judged in combination with other parameters.

[0098] Step 205: Obtain the cross-power spectrum of the first speech signal and the second speech signal, and calculate the cross-spectral coherence function according to the cross-power spectrum.

[0099] Step 206: Calculate the correlation value between the first speech signal and the second speech signal according to the cross-spectral coherence function.

[0100] In one embodiment, it is necessary to first calculate the auto-power spectrum and cross-power spectrum of the first speech signal and the second speech signal, and then calculate the cross-spectral coherence function to measure the correlation between the two signals. Specifically, the auto-power spectrum of the first speech signal and the second speech signal is S T (f) S F (f), and the cross power spectrum S of the first speech signal and the second speech signal TF (f) The calculation formula is as follows:

[0101]

[0102] in, YesS F The conjugate complex number of (f), the cross power spectrum S TF (f) reflects the relationship between the first speech signal and the second speech signal in the frequency domain, and then based on the S TF(f) Calculate the cross-spectral coherence function C TF (f):

[0103]

[0104] Among them, C TF The value range of (f) is between [0,1]. TF When (f) = 1, it means that the two signals are completely coherent at this frequency, that is, the correlation is extremely strong; when C TF When (f) = 0, it means that the two signals are irrelevant at this frequency, that is, there is no correlation. Then the high-frequency correlation value C of the first speech signal and the second speech signal can be calculated by the following formula:

[0105]

[0106] Among them, f min2 ~f max2 To calculate the high frequency band range of the correlation value, N is the corresponding number of frequency points. This formula means that within the selected high frequency band range, the cross-spectral coherence function values ​​at each frequency are accumulated and divided by the number of frequency points, and the average value obtained is the high frequency correlation value C. By comparing with the set correlation threshold range, it can be determined whether the wearer's voice exists. If C is within the threshold range, it means that the high frequency band correlation is strong and the wearer's voice may exist; if it is not within the threshold range, the wearer's voice is less likely to exist.

[0107] Step 207: Determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value, and the correlation value.

[0108] Specifically, when the single frame energy value E T When it is greater than the energy threshold, it indicates that the human voice is too loud. At this time, the low-frequency energy difference ΔE is used as the standard. If ΔE>0, it is a self-speaking speech frame. If ΔE≤0, it is not a self-speaking speech frame. When the single-frame energy value E T When it is less than the energy threshold, it indicates that the human voice is too small and the energy difference analysis is prone to misjudgment. The high-frequency correlation value C is used as the standard. If C is within [C min ,C max ], it is a self-speaking speech frame. If C is not within [C min ,C max ] range, it is not a self-speaking speech frame.

[0109] Through the self-talk detection scheme adopted in the embodiment of the present application, it is possible to achieve high-accuracy self-talk detection of TWS headphones using only dual-microphone hardware and a relatively simple detection algorithm, reducing the hardware threshold and algorithm overhead for implementing self-talk detection. In addition, the algorithm can accurately judge environmental noise, interfering human voices from all directions of the wearer, and the wearer's cough, and has good robustness. At a relatively high volume, self-talk judgment is made based on the energy difference in the low frequency band, and at a relatively low volume, self-talk judgment is made based on the correlation in the high frequency band. In this way, whether the wearer speaks loudly or quietly, the algorithm can give correct results. In addition, the above algorithm parameters are flexible, and the reaction speed and sensitivity of the self-talk detection algorithm can be adjusted, such as the upper and lower thresholds of the high-frequency correlation value, the total number of frames L for continuous detection, and the frame number threshold L1.

[0110] All of the above technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

[0111] As can be seen from the above, the self-talk detection method provided in the embodiment of the present application can receive the first voice signal and the second voice signal through the first microphone and the second microphone, align the intensities of the first voice signal and the second voice signal, obtain the energy spectrum density of the first voice signal, calculate the single-frame energy value of the first voice signal according to the energy spectrum density of the first voice signal, obtain the energy spectrum density of the second voice signal, calculate the low-frequency energy difference value according to the energy spectrum density of the first voice signal and the energy spectrum density of the second voice signal, obtain the cross-power spectrum of the first voice signal and the second voice signal, calculate the cross-spectral coherence function according to the cross-spectral coherence function, calculate the correlation value of the first voice signal and the second voice signal according to the cross-spectral coherence function, and determine whether the first voice signal is a self-talk signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value. The solution provided in the embodiments of the present application addresses the problem that the current self-talk detection algorithm is prone to misdetection of external interfering human voices and the wearer's coughing. By extracting the two voice signals obtained by the two microphones in the headset, analyzing the energy values, low-frequency energy differences and high-frequency correlations of the two signals and performing a comprehensive analysis, the self-talk detection algorithm can give correct detection results in various complex situations such as external interfering sounds, the wearer's coughing and whispering, thereby improving the accuracy of self-talk detection.

[0112] In order to implement the above method, an embodiment of the present invention further provides a self-speaking detection device, which can be integrated into a computer device such as a mobile phone, a personal computer, a tablet computer, etc.

[0113] For example, Figure 4 FIG. 1 is a schematic diagram of a first structure of a self-talk detection device provided by an embodiment of the present invention. The self-talk detection device may include:

[0114] A receiving unit 301, configured to receive a first voice signal and a second voice signal through the first microphone and the second microphone;

[0115] An alignment unit 302, configured to align the strengths of the first speech signal and the second speech signal;

[0116] A calculation unit 303, configured to calculate a single-frame energy value of the first speech signal, a low-frequency energy difference value between the first speech signal and the second speech signal, and a correlation value;

[0117] The determination unit 304 is configured to determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value, and the correlation value.

[0118] The self-talk detection device proposed in the embodiment of the present invention can receive the first voice signal and the second voice signal through the first microphone and the second microphone, align the strengths of the first voice signal and the second voice signal, calculate the single-frame energy value of the first voice signal, the low-frequency energy difference value and the correlation value between the first voice signal and the second voice signal, and determine whether the first voice signal is a self-talk signal based on the single-frame energy value, the low-frequency energy difference value and the correlation value. The solution provided in the embodiment of the present application obtains two voice signals through two microphones in the headset, analyzes the energy value, low-frequency energy difference and high-frequency correlation of the two signals, and thereby achieves a more accurate self-talk detection effect.

[0119] All of the above technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

[0120] An embodiment of the present application further provides a headset device, comprising a memory and a processor, wherein the processor is configured to execute the process in the self-speaking detection method provided in the present embodiment by calling a computer program stored in the memory.

[0121] For example, see Figure 5 The computer device 400 may include a memory 401, a processor 402 and other components. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0122] In this embodiment, the processor 402 in the computer device loads the executable code corresponding to the process of one or more applications into the memory 401, and the processor 402 runs the application stored in the memory 401, thereby executing the steps in the above-mentioned self-speaking detection method in this embodiment.

[0123] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0124] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the self-speaking detection methods provided in the embodiments of the present application. For example, the instructions can execute the steps in the self-speaking detection method described above in the present embodiment.

[0125] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0126] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0127] Since the instructions stored in the storage medium can execute the steps in any one of the self-talking detection methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the self-talking detection methods provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0128] It should be noted that, for the self-talk detection method described in the embodiment of the present application, a person skilled in the art can understand that the whole or part of the process of the self-talk detection method described in the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, such as a memory, and executed by at least one processor, and the execution process may include the process of the embodiment of the self-talk detection method. Among them, the storage medium can be a disk, an optical disk, a read-only memory, a random access memory (Random Access Memory, RAM), etc.

[0129] The above is a detailed introduction to a self-talk detection method, device, storage medium and headphone device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A self-talk detection method, applied to an earphone device, the earphone device comprising a first microphone and a second microphone, characterized in that: include: receiving a first voice signal and a second voice signal through the first microphone and the second microphone; Aligning the strengths of the first speech signal and the second speech signal; Calculating a single-frame energy value of the first speech signal, a low-frequency energy difference value between the first speech signal and the second speech signal, and a correlation value; Determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value.

2. The self-talk detection method according to claim 1, characterized in that: The aligning the strengths of the first speech signal and the second speech signal includes: Acquire an arrangement of the first microphone and the second microphone; calculating an alignment coefficient according to an arrangement of the first microphone and the second microphone; The strength of the first speech signal is weakened according to the alignment coefficient.

3. The self-talk detection method according to claim 1, characterized in that: The determining whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value comprises: If the single-frame energy value is greater than the energy threshold, determining whether the current speech frame of the first speech signal is a self-speaking speech frame according to the low-frequency energy difference value; If the single-frame energy value is not greater than the energy threshold, determining whether the current speech frame of the first speech signal is a self-speaking speech frame according to the correlation value.

4. The self-talk detection method according to claim 3, characterized in that: The method further comprises: When it is determined that the current speech frame of the first speech signal is a self-speaking speech frame, obtaining a target number of self-speaking speech frames determined as the self-speaking speech frames in a subsequent first preset number of continuous speech frames; Determining whether the target quantity is greater than a second preset quantity; If yes, it is determined that the first speech signal is a self-speaking signal.

5. The self-talk detection method according to claim 1, characterized in that: The calculation process of the single-frame energy value of the first speech signal includes: Obtain the energy spectral density S of the first speech signal T (f): S T (f)=|T′(f)| 2 Wherein, T′(f) is the single-frame signal spectrum of the first speech signal after alignment processing, and the single-frame energy value E of the first speech signal is calculated. T : E T =∑ f＝0 S T (f)。 6. The self-talk detection method according to claim 5, characterized in that: The calculation process of the low-frequency energy difference between the first speech signal and the second speech signal includes: Obtain the energy spectral density S of the second speech signal F (f): S F (f)=|f(f)| 2 Wherein, F(f) is the single-frame signal spectrum of the second speech signal; Calculate the low-frequency energy difference ΔE between the first speech signal and the second speech signal: Among them, f min ~f max is the low frequency band range for calculating the energy difference.

7. The self-talk detection method according to claim 6, characterized in that: The calculation process of the correlation value between the first speech signal and the second speech signal includes: Obtain the cross power spectrum S of the first speech signal and the second speech signal TF (f): in, YesS F The conjugate complex number of (f) is used to calculate the cross-spectral coherence function C by the following formula: TF (f): Among them, C TF The value range of (f) is between [0,1]. TF When (f) = 1, it means that the first speech signal and the second speech signal are highly correlated. TF When (f)=0, it indicates that there is no correlation between the first speech signal and the second speech signal. The correlation value C of the first speech signal and the second speech signal is calculated by the following formula: Among them, f min2 ~f max2 is the high frequency band range for calculating the correlation value, and N is the corresponding number of frequency points.

8. A self-talk detection device, applied to an earphone device, the earphone device comprising a first microphone and a second microphone, characterized in that: include: A receiving unit, configured to receive a first voice signal and a second voice signal through the first microphone and the second microphone; an alignment unit, configured to align the strengths of the first speech signal and the second speech signal; A calculation unit, used to calculate a single-frame energy value of the first speech signal, a low-frequency energy difference value between the first speech signal and the second speech signal, and a correlation value; A determination unit is used to determine whether the first speech signal is a self-speaking signal according to the single-frame energy value, the low-frequency energy difference value and the correlation value.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A headset device, comprising a memory and a processor, characterized in that: The processor executes the method according to any one of claims 1 to 7 by calling the computer program stored in the memory.

Images