A voice activity detection method and device, an electronic device, and a storage medium

Abstract

The application provides a voice activity detection method and device, an electronic device and a storage medium. The method comprises: obtaining a to-be-detected audio signal; performing long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value; comparing the long-time feature value with a preset threshold value to determine a voice activity detection result. In the embodiment of the application, the voice activity detection is performed based on the long-time feature value, which can improve the accuracy and stability of the voice activity detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice detection, in particular to a voice activity detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] Voice activity detection (VAD) is an important front-end module in audio technology, which is mainly used to detect whether there is a voice activity segment in audio data. It can provide functions such as voice wake-up as an independent module, and can also be used as a front-end component of command word recognition and continuous speech recognition to form a low-power multifunctional language recognition system. Voice activity detection has detection and judgment methods based on energy decision, short-time zero-crossing rate, mel-frequency cepstral coefficient and other voice signal features. However, the accuracy and stability of voice activity detection using the above features are poor. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a voice activity detection method and device, an electronic device and a storage medium to improve the accuracy and stability of voice activity detection.

[0004] The present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a voice activity detection method, comprising: obtaining a to-be-detected audio signal; performing long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value; comparing the long-time feature value with a preset threshold value to determine a voice activity detection result.

[0006] Since the long-time feature value preserves the connection of the voice signal in a long time dimension, it can reflect the long-time non-stationarity of the voice signal. Therefore, in the embodiments of the present application, voice activity detection is performed based on the extracted long-time feature value, which can improve the accuracy and stability (or robustness) of voice activity detection.

[0007] In some possible implementation manners of the technical solution provided in the above first aspect, the long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value comprises: simulating the long-time feature of the to-be-detected audio signal by using a forgetting factor to obtain the long-time feature value.

[0008] Since the long-time feature value calculation process needs to use all frame data in a long time period, a large memory space is needed for storage and a huge amount of calculation is needed, therefore, in the embodiments of the present application, the long-time feature of the to-be-detected audio signal is simulated by using a forgetting factor, which can reduce the calculation amount of calculating the long-time feature value and reduce the delay of voice activity detection.

[0009] In some possible implementation manners, the long-term feature value is an LTSF (Long-Term Spectral Flatness, long-term spectral flatness) feature value; and the step of obtaining the long-term feature value of the to-be-detected audio signal by using the forgetting factor to simulate the long-term feature of the to-be-detected audio signal comprises: determining a spectral energy of the to-be-detected audio signal based on a first forgetting factor; determining an arithmetic mean of the spectral energy and a geometric mean of the spectral energy based on a second forgetting factor; wherein the spectral energy, the arithmetic mean of the spectral energy, and the geometric mean of the spectral energy are the long-term feature; and obtaining the LTSF feature value based on the arithmetic mean of the spectral energy and the geometric mean of the spectral energy.

[0010] The LTSF feature value obtained through the foregoing improvement does not need to be calculated based on all frame data, but is generated by using the first forgetting factor and the second forgetting factor to simulate the spectral energy, the arithmetic mean of the spectral energy, and the geometric mean of the spectral energy, so that the calculation amount of the LTSF feature value can be reduced, and the power consumption and the storage space requirement of the electronic device can be reduced.

[0011] In some possible implementation manners, the long-term feature value is an LTSV (Long-Term Signal Variability, long-term signal variability) feature value; and the step of obtaining the long-term feature value of the to-be-detected audio signal by using the forgetting factor to simulate the long-term feature of the to-be-detected audio signal comprises: determining a spectral energy of the to-be-detected audio signal based on a first forgetting factor and a second forgetting factor; wherein the spectral energy is the long-term feature; determining an entropy of a current frame and an average entropy of the current frame based on the spectral energy; and obtaining the LTSV feature value based on the entropy of the current frame and the average entropy of the current frame.

[0012] The LTSV feature value obtained through the foregoing improvement does not need to be calculated based on all frame data, but is generated by using the first forgetting factor and the second forgetting factor to simulate the spectral energy, so that the calculation amount of the LTSV feature value can be reduced, and the power consumption and the storage space requirement of the electronic device can be reduced.

[0013] In some possible implementation manners, the step of obtaining the long-term feature value of the to-be-detected audio signal by using the forgetting factor to simulate the long-term feature of the to-be-detected audio signal comprises: simulating the long-term feature of the to-be-detected audio signal by using the forgetting factor; processing the long-term feature based on a variance algorithm to obtain a long-term variance feature value; and wherein the long-term variance feature value is the long-term feature value.

[0014] In the embodiments of the present application, the long-time variance characteristic value obtained through the variance algorithm can make the voice and the environmental noise more distinguishable, thereby improving the stability of the voice activity detection.

[0015] In some possible implementation manners, in combination with the technical solutions provided by the first aspect, the long-time variance characteristic value is a characteristic value based on LTSF; and a calculation expression of the long-time variance characteristic value is:

[0016] S(n,ω k )=a1*S(n-1,ω k )+(1-a2)*|X(n,ω k )| 2 ;

[0017] AM(m,ω k )=a2*AM(m-1,ω k )+(1-a2)*S(n,ω k );

[0018] Var(n,ω k )=(S(n,ω k )-AM(n,ω k )) 2 / S(n,ω k );

[0019] Var_s(n,ω k )=a2*Var_s(n-1,ω k )+(1-a2)*Var(n,ω k );

[0020]

[0021] wherein m represents the mth frame, n represents the nth frame, ω k represents the kth frequency point corresponding to the Fourier coefficient of the to-be-detected audio signal obtained through fast Fourier transform, X(n,ω k ) represents the spectrum corresponding to parameters n and ω k , S(n,ω k ) represents the long-time spectrum energy corresponding to parameters n and ω k , S(n-1,ω k ) represents the long-time spectrum energy corresponding to parameters n-1 and ω k , AM(m,ω k ) represents the arithmetic mean of the energy corresponding to parameters m and ω k , AM(m-1,ω k ) represents the arithmetic mean of the energy corresponding to parameters m-1 and ω k , and Var(n,ω k) represents parameter n, ω k corresponding variance value; Var_s(n, ω k ) represents smoothing Var(n, ω k ); correspondingly, Var_s(n-1, ω k ) represents smoothing Var(n-1, ω k ), and Var(n-1, ω k ) represents parameter n-1, ω k corresponding variance value; Var_mean(n) represents the average value of Var_s(n, ω k ) in the frequency axis of the nth frame, and bins represent the total number of frequency points; Var_tf(n) represents the long-time variance feature value; a1 represents a first forgetting factor; and a2 represents a second forgetting factor.

[0022] In some possible implementation manners, the long-time variance feature value is an LTSV-based feature value, and a calculation expression of the long-time variance feature value is:

[0023] S(n, ω k ) = a1 * S(n-1, ω k ) + (1-a1) * |X(n, ω k )| 2 ;

[0024] S(l, ω k ) = a2 * S(n-1, ω k ) + (1-a2) * S(n, ω k );

[0025] Var(n, ω k ) = (S(n, ω k ) - S(l, ω k )) 2 / S(n, ω k );

[0026] Var_s(n, ω k ) = a2 * Var_s(n-1, ω k ) + (1-a2) * Var(n, ω k );

[0027]

[0028] wherein m represents the mth frame, n represents the nth frame, l represents the lth frame, ω k represents the kth frequency point corresponding to a Fourier coefficient of the to-be-detected audio signal obtained through fast Fourier transform; X(n, ω k ) represents parameter n, ωk corresponding spectrum; S(n, ω k ) denotes parameters n, ω k corresponding long-term spectrum energy; S(n-1, ω k ) denotes parameters n-1, ω k corresponding long-term spectrum energy; S(l, ω k ) denotes parameters l, ω k corresponding long-term spectrum energy; Var(n, ω k ) denotes parameters n, ω k corresponding variance value; Var_s(n, ω k ) denotes smoothing of Var(n, ω k ); correspondingly, Var_s(n-1, ω k ) denotes smoothing of Var(n-1, ω k ); Var(n-1, ω k ) denotes parameters n-1, ω k corresponding variance value; Var_mean(n) denotes an average value of Var_s(n, ω k ) on a frequency axis in the nth frame, and bins denotes a total number of frequency points; Var_tf(n) denotes the long-term variance feature value; a1 denotes a first forgetting factor; and a2 denotes a second forgetting factor.

[0029] With reference to the technical scheme provided in the first aspect above, in some possible implementation manners, before the long-term feature value is compared with the preset threshold value, the method further includes: dynamically adjusting the threshold value based on a feature frequency of the long-term feature value and background noise energy.

[0030] In the embodiments of the present application, the threshold value is dynamically adjusted based on the feature frequency of the long-term feature value and the background noise energy, so as to improve the accuracy and stability of voice activity detection.

[0031] With reference to the technical scheme provided in the first aspect above, in some possible implementation manners, the long-term feature value is an LTSF feature value or an LTSV feature value.

[0032] Since the LTSF feature value and the LTSV feature value both retain the connection of a voice signal in a long time dimension and can reflect the long-term non-stationarity of the voice signal, in the embodiments of the present application, the voice activity detection is performed based on the extracted LTSF feature value or LTSV feature value, so as to improve the accuracy and stability (or robustness) of voice activity detection.

[0033] In a second aspect, an embodiment of the present application provides a voice activity detection apparatus, comprising: an acquisition module configured to acquire a to-be-detected audio signal; a feature extraction module configured to perform long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value; and a decision module configured to compare the long-time feature value with a preset threshold value and determine a voice activity detection result.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, which are connected; the memory is configured to store a program; and the processor is configured to invoke the program stored in the memory to execute the method provided in the first aspect and / or some possible implementation manners of the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the method provided in the first aspect and / or some possible implementation manners of the first aspect when being run by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 A module block diagram of an electronic device provided by an embodiment of the present application.

[0038] Figure 2 A step flowchart of a voice activity detection method provided by an embodiment of the present application.

[0039] Figure 3 A schematic diagram of three ways of calculating LTSF feature values provided by an embodiment of the present application.

[0040] Figure 4 A schematic diagram of a voice activity detection method provided by an embodiment of the present application.

[0041] Figure 5 A flowchart of dynamically adjusting a threshold value provided by an embodiment of the present application.

[0042] Figure 6 A module block diagram of a voice activity detection apparatus provided by an embodiment of the present application.

[0043] Icon: 100-electronic device; 110-processor; 120-memory; 200-voice activity detection apparatus; 210-acquisition module; 220-feature extraction module; 230-decision module. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0045] Please refer to Figure 1 The present embodiment provides a schematic structural block diagram of an electronic device 100 applying a voice activity detection method and apparatus. In the present embodiment, the electronic device 100 can be, but is not limited to, a personal computer (PC), a smart phone, a Bluetooth headset, a Bluetooth speaker, a tablet computer, a personal digital assistant (PDA), a mobile Internet device (MID), etc. Of course, the above-mentioned devices are only used to facilitate the understanding of the present embodiment, and should not be regarded as a limitation to the present embodiment.

[0046] In structure, the electronic device 100 can include a processor 110 and a memory 120.

[0047] The processor 110 is directly or indirectly electrically connected with the memory 120 to realize the transmission or interaction of data. For example, these elements can be electrically connected with each other through one or more communication buses or signal lines. The voice activity detection apparatus includes at least one software module which can be stored in the memory 120 in the form of software or firmware (Firmware) or solidified in the operating system (OS) of the electronic device 100. The processor 110 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the voice activity detection apparatus, to realize the voice activity detection method. The processor 110 can execute the computer program after receiving the execution instruction.

[0048] The processor 110 can be an integrated circuit chip with signal processing capability. The processor 110 can also be a general-purpose processor, for example, a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a discrete gate or transistor logic, a discrete hardware component, and can implement or execute the disclosed methods, steps, and logic block diagrams in the embodiments of the present application. In addition, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0049] The memory 120 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), and an electrically erasable programmable read-only memory (EEPROM). The memory 120 is used to store a program, and the processor 110 executes the program after receiving an execution instruction.

[0050] It should be noted that Figure 1 The structure shown is only a schematic, and the electronic device 100 provided by the embodiments of the present application can also have fewer or more components, or have a different configuration from Figure 1 that shown. In addition, Figure 1 each component shown can be implemented by software, hardware, or a combination thereof. Figure 1

[0051] Please refer to Figure 2 , Figure 2 the step flow chart of the voice activity detection method provided by the embodiments of the present application, which is applied to Figure 1 the electronic device 100 shown. It should be noted that the voice activity detection method provided by the embodiments of the present application is not limited by the order shown in Figure 2 and the following, and the method comprises steps S101-S103.

[0052] Step S101: Acquire an audio signal to be detected.

[0053] ​The electronic device can collect an audio signal in a current environment in real time as the to-be-detected audio signal, or collect a historical audio signal as the to-be-detected audio signal when it is necessary to analyze the historical audio signal. Of course, the to-be-detected audio signal can also come from other devices, such as a microphone or other audio collection device.

[0054] The to-be-detected audio signal can be a PCM (Pulse Code Modulation) data stream. Before subsequent processing, windowing processing is performed to extract a PCM data stream of a fixed time length as the to-be-detected audio signal.

[0055] Step S102: performing long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value.

[0056] The long-time feature value is an LTSF feature value or an LTSV feature value.

[0057] The LTSF feature value is calculated by the following formula:

[0058]

[0059] In formulas (1)-(4), m, n, and p represent frame numbers, that is, m represents the mth frame (a current index frame), n represents the nth frame, and p represents the pth frame; ω k represents a Fourier coefficient corresponding to the kth frequency point obtained by performing fast Fourier transform on the to-be-detected audio signal; M represents the total number of frames of the to-be-detected audio signal; R represents the total number of frames; X(p, ω k ) represents a spectrum corresponding to parameters p and ω k ; S(n, ω k ) represents a spectrum energy corresponding to parameters n and ω k ; AM(m, ω k ) represents an arithmetic mean of energy corresponding to parameters m and ω k ; GM(m, ω k ) represents a geometric mean of energy corresponding to parameters m and ω k ; and L x (m) is an LTSF feature value corresponding to parameter m.

[0060] The LTSV feature value is calculated by the following formula:

[0061]

[0062]

[0063] In formulas (5)-(8), m, n, p, and l represent frame numbers, that is, m represents the mth frame, n represents the nth frame, p represents the pth frame, and l represents the lth frame; ωk represents the Fourier coefficient corresponding to the kth frequency point obtained by fast Fourier transform of the audio signal to be detected; M represents the total number of frames of the audio signal to be detected; R represents the total number of frames; X(p, ω k ) represents the spectrum corresponding to parameters p, ω k ; S(n, ω k ) represents the spectrum energy corresponding to parameters n, ω k ; S(l, ω k ) represents the spectrum energy corresponding to parameters l, ω k ; and I(m, k) represents the entropy corresponding to parameters m and k. represents the average entropy corresponding to parameters m and k; and LSTV(m) represents the LTSV feature value corresponding to parameter m.

[0064] After the electronic device obtains the signal to be detected, the electronic device extracts long-time features from the signal to be detected to obtain long-time feature values for voice activity detection. Before extracting the features from the signal to be detected, the electronic device also needs to perform fast Fourier transform (FFT) on the signal to be detected.

[0065] Step S103: Comparing the long-time feature values with preset threshold values to determine a voice activity detection result.

[0066] Finally, the long-time feature values are compared with preset threshold values. In the embodiments of the present application, the comparison process can be performed by a decision state machine. It should be noted that the voice activity detection result can be directly determined based on the size relationship between the long-time feature values and the preset threshold values. The voice activity detection result includes voice and environmental noise.

[0067] For example, when the long-time feature value is greater than the preset threshold value, the output voice activity detection result is voice, and when the long-time feature value is less than the preset threshold value, the output voice activity detection result is environmental noise.

[0068] It should be noted that the LTSF feature value and the LTSV feature value both retain the connection of the voice signal in a long-time dimension and can reflect the long-time non-stationarity of the voice signal. Therefore, in the embodiments of the present application, voice activity detection is performed based on the extracted LTSF feature value or LTSV feature value, which can improve the accuracy and stability (or robustness) of voice activity detection.

[0069] The inventor has found in practice that the original LTSF feature value and LTSV feature value calculation process needs to use all frame data in a long time period, needs a large memory space for storage and needs a huge amount of calculation. Based on this, as an implementation, a forgetting factor smoothing is used to approximate the long-time feature. That is, the step S102 can specifically include: using a forgetting factor to simulate the long-time feature of the audio signal to be detected to obtain the long-time feature value.

[0070] When the extracted long-time feature value is the LTSF feature value, the step specifically includes: determining the spectral energy of the audio signal to be detected based on a first forgetting factor; determining the arithmetic mean of the spectral energy and the geometric mean of the spectral energy based on a second forgetting factor; wherein the spectral energy, the arithmetic mean of the spectral energy and the geometric mean of the spectral energy are the long-time feature; obtaining the LTSF feature value based on the arithmetic mean of the spectral energy and the geometric mean of the spectral energy.

[0071] The LTSF feature value obtained by the above step can be realized by the following formula:

[0072] S(n,ω k )=a1*S(n-1,ω k )+(1-a2)*|X(n,ω k )| 2 (9)

[0073] GM(m,ω k )=a2*GM(m-1,ω k )+(1-a2)*log 10 S(n,ω k ) (10)

[0074] AM(m,ω k )=a2*AM(m-1,ω k )+(1-a2)*S(n,ω k ) (11)

[0075] L x (m)=∑ k (log 10 GM(m,ω k )-log 10 AM(m,ω k )) (12)

[0076] In the formulas (9)-(10), a1 represents the first forgetting factor; a2 represents the second forgetting factor; the values of a1 and a2 are both close to 1. In the embodiments of the present application, a1=0.96 and a2=0.90. GM(m-1,ω k ) represents the parameters m-1, ωk The corresponding geometric mean of energy; AM(m-1,ω) k ) represents parameters m-1, ω k The arithmetic mean of the corresponding energies.

[0077] It should be noted that the explanations of other parameters can be found in the explanations of the same parameters in formulas (1) to (4), and will not be repeated here.

[0078] Compared to the original method of calculating LTSF features, the improved LTSF feature values ​​do not require calculation on all frame data. Instead, they are generated by simulating the spectral energy, the arithmetic mean of the spectral energy, and the geometric mean of the spectral energy through the first and second forgetting factors. This method reduces the computational load when calculating LTSF feature values, thereby reducing the power consumption and storage space requirements of electronic devices.

[0079] Specifically, when the extracted long-term feature value is the LTSV feature value, the above steps use a forgetting factor to simulate the long-term features of the audio signal to be detected and obtain the long-term feature value. This can specifically include: determining the spectral energy of the audio signal to be detected based on the first forgetting factor and the second forgetting factor; wherein the spectral energy is the long-term feature; determining the entropy of the current frame and the average entropy of the current frame based on the spectral energy; and obtaining the LTSV feature value based on the entropy of the current frame and the average entropy.

[0080] The LTSV eigenvalues ​​obtained using the above steps can be realized using the following formula:

[0081] S(n,ω k )=a1*S(n-1,ω k )+(1-a1)*|X(n,ω k )| 2 (13)

[0082] S(l,ω k )=a2*S(n-1,ω k )+(1-a2)*S(n,ω k (14)

[0083] I(m,k)=-S(l,ω k )×log(S(l,ω k (15)

[0084]

[0085] In formulas (13) to (17), a1 represents the first forgetting factor; a2 represents the second forgetting factor; the values ​​of a1 and a2 are both close to 1. In the embodiments of this application, a1 = 0.96, a2 = 0.90. S(n,ωk denotes parameters n, ω k corresponding long-term spectral energy; S(n-1, ω k denotes parameters n-1, ω k corresponding long-term spectral energy.

[0086] It should be noted that the explanation of other parameters can refer to the explanation of the same parameters in formulas (5)-(8), which will not be repeated here, and the expression form of formula (16) is the same as that of formula (7), and the expression form of formula (17) is the same as that of formula (8).

[0087] The LTSV eigenvalue obtained through the above improvement is calculated in comparison with the original LTSV eigenvalue, without the need to calculate all frame data, but to simulate the generation of spectral energy through the first forgetting factor and the second forgetting factor. In this way, the calculation amount of calculating the LTSV eigenvalue can be reduced, and the power consumption and storage space requirement of the electronic device can be reduced.

[0088] In order to further improve the discrimination of the extracted long-term eigenvalue and improve the stability of the voice activity detection, the step of obtaining the long-term eigenvalue by using the forgetting factor to simulate the long-term feature of the audio signal to be detected can further include: using the forgetting factor to simulate the long-term feature of the audio signal to be detected; processing the long-term feature based on a variance algorithm to obtain a long-term variance eigenvalue; wherein the long-term variance eigenvalue is the long-term eigenvalue.

[0089] When the extracted long-term eigenvalue is the LTSF eigenvalue, the above step can specifically include: determining the spectral energy of the audio signal to be detected based on the first forgetting factor; determining the arithmetic mean of the spectral energy based on the second forgetting factor; processing the spectral energy and the arithmetic mean of the spectral energy based on a variance algorithm to obtain a long-term variance eigenvalue.

[0090] The long-term variance eigenvalue obtained by the above step can be realized by the following formula:

[0091] S(n, ω k ) = a1 * S(n-1, ω k ) + (1-a2) * |X(n, ω k )| 2 (18)

[0093] AM(m, ω k ) = a2 * AM(m-1, ω k ) + (1-a2) * S(n, ω k ) (19)

[0094] Var(n, ω k ) = (S(n, ωk )-AM(n,ω k )) 2 / S(n,ω k ) (20)

[0095] Var_s(n,ω k )=a2*Var_s(n-1,ω k )+(1-a2)*Var(n,ω k ) (21)

[0096]

[0097] wherein, formula (18) is same as formula (9), formula (19) is same as formula (11), and the parameters herein can be referred to each other.

[0098] In formula (20)-(23), Var(n,ω k ) represents a variance value corresponding to parameters n, ω k ; Var_s(n,ω k ) represents smoothing of Var(n,ω k ); correspondingly, Var_s(n-1,ω k ) represents smoothing of Var(n-1,ω k ); Var(n-1,ω k ) represents a variance value corresponding to parameters n-1, ω k ; Var_mean(n) represents an average value of Var_s(n,ω k ) in a frequency axis of the nth frame; and Var_tf(n) represents a long-time variance feature value. The above formula (20)-(23) correspond to a variance algorithm.

[0099] When the extracted long-time feature value is an LTSV feature value, the above steps can specifically include: determining a spectral energy of the audio signal to be detected based on a first forgetting factor; and processing the spectral energy based on a variance algorithm to obtain a long-time variance feature value.

[0100] The long-time variance feature value obtained by the above steps can be realized by the following formula:

[0101] S(n,ω k )=a1*S(n-1,ω k )+(1-a1)*|X(n,ω k )| 2 (24)

[0102] S(l,ω k )=a2*S(n-1,ω k) + (1 - a2) * S(n, ω k ) (25)

[0103] Var(n, ω k ) = (S(n, ω k ) - S(l, ω k )) 2 / S(n, ω k ) (26)

[0104] Var_s(n, ω k ) = a2 * Var_s(n - 1, ω k ) + (1 - a2) * Var(n, ω k ) (27)

[0105]

[0106] Wherein, formula (24) and formula (13) are the same, formula (25) and formula (14) are the same, the parameters described herein can be referred to each other. In addition, formula (26)-(29) and formula (20)-(23) are the same parameters, which can be referred to each other, and will not be repeated here. The above formula (26)-(29) correspond to the variance algorithm.

[0107] It can be seen that the long-time variance characteristic value obtained by the above variance algorithm can make the speech and environmental noise more distinguishable, and the process of calculating the long-time variance characteristic value by using the above algorithm also omits the logarithmic operation, thereby effectively reducing the calculation amount.

[0108] It can be seen that in the embodiments of the present application, the long-time characteristic value can be calculated in three ways, please refer to Figure 3 , taking the long-time characteristic value as the LTSF characteristic value as an example.

[0109] The first way: after obtaining the audio signal to be detected, the spectral energy is calculated by formula (1), the geometric mean is calculated by formula (2), the arithmetic mean is calculated by formula (3), and finally the LTSF characteristic value is calculated by formula (4).

[0110] The second way: after obtaining the audio signal to be detected, the spectral energy is calculated by formula (9), the geometric mean is calculated by formula (10), the arithmetic mean is calculated by formula (11), and finally the LTSF characteristic value is calculated by formula (12).

[0111] Thirdly, the spectral energy is calculated by using formula (18) after the audio signal to be detected is acquired; formula (18) is the same as formula (9), the arithmetic mean is calculated by using formula (19); formula (19) is the same as formula (11), then the variance value is calculated by using formula (20), smoothing is performed by using formula (21), the variance mean is calculated by using formula (22), and finally the LTSF characteristic value is calculated by using formula (23).

[0112] In the embodiment, the preset threshold value can be a fixed value, which can be preconfigured in the electronic device.

[0113] In an embodiment, the preset threshold value can be a dynamically adjusted value, that is, before the long-time characteristic value is compared with the preset threshold value, the method comprises: dynamically adjusting the threshold value based on the characteristic frequency of the long-time characteristic value and the background noise energy.

[0114] Referring to Figure 4 The voice activity detection method provided in the embodiment comprises improvement of the feature extraction process and dynamic adjustment of the threshold value. The dynamic adjustment of the threshold value is based on background noise tracking, and after the threshold value is adjusted, the voice activity detection result is output by decision of the decision state machine.

[0115] Referring to Figure 5 The specific process of dynamically adjusting the threshold value is as follows:

[0116] S1: the maximum upper limit value (max_fe) and the minimum lower limit value (min_fe) of the characteristic interval of the preset time length are calculated. The maximum upper limit value (max_fe) represents the limit value of 100% speech, and the minimum lower limit value (max_fe) represents the limit value of 100% environmental noise. Then, the maximum characteristic upper limit value (max_fe_thr) of the long-time characteristic value and the minimum characteristic lower limit value (min_fe_thr) of the long-time characteristic value are calculated based on the background noise energy. The background noise energy is the background noise energy acquired by the historical frame data. This step is equivalent to calculating the maximum characteristic upper limit value (max_fe_thr) and the minimum characteristic lower limit value (min_fe_thr) of the long-time characteristic value by using the background noise energy acquired by the historical frame data.

[0117] The preset time length can be 0.2 seconds or 0.3 seconds, which is not limited in the application.

[0118] S2: Calculate the upper envelope, lower envelope, upper envelope fluctuation width, lower envelope fluctuation width and envelope width of the long-term feature value in the preset time length. Wherein, the upper envelope, lower envelope and envelope width of the long-term feature value are used to represent the characteristic frequency, that is, the change of the upper envelope, lower envelope and envelope width of the long-term feature value represents the change of the characteristic frequency. Specifically, when the envelope width is narrow and close to the upper limit, it represents continuous speech; when the envelope width is narrow and close to the lower limit, it represents continuous environmental noise; when the envelope width is wide, it represents that speech data and noise data exist at the same time.

[0119] S3: Judge the audio state corresponding to the current long-term feature value (the state includes continuous speech, continuous noise, speech noise conversion), and adjust the threshold value based on the judgment result. Specifically, it includes the following three adjustment conditions:

[0120] (1) envelop_wide < min_wide (the width when continuous speech or noise, a preset value). It represents that the current audio stream is in a continuous state, if envelop_up_diff > envelop_down_diff represents continuous speech stream, the threshold value is moved up, otherwise, it is continuous noise stream, the threshold value is moved down.

[0121] (2) envelop_wide > well_wide (ideal width when speech noise conversion, a preset value). It represents that the current audio stream is in a conversion state of speech and noise. The threshold value is calculated by the average of the upper envelope fluctuation width and the lower envelope fluctuation width.

[0122] (3) min_wide < envelop_wide < well_wide: it represents that the current audio stream is between the continuous state and the conversion state, after analysis according to the minimum envelope width representing noise characteristics (min_bg_wide, a preset value) and the current ideal speech envelope width (wide_select, a preset value), the threshold value is adjusted.

[0123] (3.1) Assuming wide_select = max_fe - min_fe_thr - well_wide; if the condition 0 < wide_select < min_bg_wide is met, the threshold value will be updated to the average of max_fe and min_fe_thr, and min_bg_wide will be updated to the value of wide_select.

[0124] (3.2) Assuming wide_select = max_fe_thr - min_fe - well_wide; if 0 < wide_select < min_bg_wide, the threshold value will be updated to the average of max_fe_thr and min_fe, and min_bg_wide will be updated to the value of wide_select.

[0125] (3.3) min_bg_wide = wide_fe_thr (ideal width threshold, a preset value) - well_wide; the threshold value will be updated to the average of max_fe_thr and min_fe_thr.

[0126] It can be seen that, in the embodiment of the application, the threshold value is dynamically adjusted based on the feature frequency and the background noise energy of the long-time feature value, so as to improve the accuracy and stability of the voice activity detection.

[0127] The voice activity detection method using formula (18) to formula (23) is used for experiments. The voice detection results measured in the test data of 18 kinds of environmental noise backgrounds, a total of 2868 test voice data (about 13 GB) are as follows:

[0128]

[0129] Compared with the current voice activity detection method, the comparison results are as follows:

[0130]

[0131] From the test results, it can be seen that the voice activity detection method of the above example significantly improves the detection accuracy and the stability of the method in different responsible noise environments. At the same time, the method can greatly reduce the storage space and calculation amount requirements of the algorithm, and becomes a kind of efficient, high-robustness and low-consumption voice detection method, which is easy to realize in embedded devices and low-power devices.

[0132] Please refer to Figure 6 , based on the same inventive concept, the embodiment of the application also provides a voice activity detection device 200, which comprises:

[0133] The acquisition module 210 is configured to acquire a to-be-detected audio signal.

[0134] The feature extraction module 220 is configured to perform long-time feature extraction on the to-be-detected audio signal to obtain a long-time feature value.

[0135] The decision module 230 is configured to compare the long-time feature value with a preset threshold value, and determine a voice activity detection result.

[0136] Optionally, the feature extraction module 220 is specifically configured to simulate long-time features of the audio signal to be detected by using a forgetting factor, to obtain the long-time feature values.

[0137] Optionally, the long-time feature values are LTSF feature values; the feature extraction module 220 is further configured to determine spectral energy of the audio signal to be detected based on a first forgetting factor; determine an arithmetic mean of the spectral energy and a geometric mean of the spectral energy based on a second forgetting factor; wherein the spectral energy, the arithmetic mean of the spectral energy and the geometric mean of the spectral energy are the long-time features; and obtain the LTSF feature values based on the arithmetic mean of the spectral energy and the geometric mean of the spectral energy.

[0138] Optionally, the long-time feature values are LTSV feature values; the feature extraction module 220 is further configured to determine spectral energy of the audio signal to be detected based on a first forgetting factor and a second forgetting factor; wherein the spectral energy is the long-time feature; determine entropy of a current frame and average entropy of the current frame based on the spectral energy; and obtain the LTSV feature values based on the entropy of the current frame and the average entropy of the current frame.

[0139] Optionally, the feature extraction module 220 is further configured to simulate long-time features of the audio signal to be detected by using a forgetting factor; and process the long-time features based on a variance algorithm to obtain long-time variance feature values; wherein the long-time variance feature values are the long-time feature values.

[0140] Optionally, the apparatus further comprises a threshold adjustment module. The threshold adjustment module is configured to dynamically adjust a preset threshold value based on a feature frequency of the long-time feature values and background noise energy, before the long-time feature values are compared with the threshold value.

[0141] It should be noted that, for the convenience and brevity of description, the specific working process of the system, apparatus and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0142] Based on the same inventive concept, the embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, the computer program being executed to perform the method provided in the foregoing embodiments when being executed.

[0143] The storage medium can be any available medium or media that is / are accessible by a computer, or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy diskette, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), etc.

[0144] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic, for example, the division of the units is merely a logical function division, and another division mode can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0145] In addition, the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0146] Furthermore, the functional modules in each of the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0147] In this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0148] The above description is only for the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A voice activity detection method, characterized by, The method comprises: obtaining an audio signal to be detected; extracting long-time features of the audio signal to be detected to obtain long-time feature values; the long-time feature values are LTSF feature values or LTSV feature values; comparing the long-time feature values with preset threshold values to determine a voice activity detection result; the extracting of the long-time features of the audio signal to be detected to obtain the long-time feature values comprises: simulating long-time features of the audio signal to be detected by using a forgetting factor to obtain the long-time feature values; wherein the long-time feature values comprise long-time variance feature values; when the long-time variance feature values are the LTSF feature values, obtaining the long-time variance feature values comprises: determining spectral energy of the audio signal to be detected based on a first forgetting factor; determining an arithmetic mean of the spectral energy based on a second forgetting factor; and processing the spectral energy and the arithmetic mean of the spectral energy based on a variance algorithm to obtain the long-time variance feature values; when the long-time variance feature values are LTSF feature values, the calculation expression of the long-time variance feature values comprises: ； ； wherein, denotes the frame, denotes the frame; denotes the frequency bin corresponding to the Fourier coefficient of the audio signal to be detected obtained by the fast Fourier transform; denotes the , corresponding spectrum; denotes the , corresponding long-term spectral energy; denotes the , corresponding long-term spectral energy; denotes the , corresponding arithmetic mean of the energies; denotes the , corresponding arithmetic mean of the energies; denotes the first forgetting factor; when the long-time variance feature values are LTSV feature values, obtaining the long-time variance feature values comprises: determining spectral energy of the audio signal to be detected based on the first forgetting factor; and processing the spectral energy based on the variance algorithm to obtain the long-time variance feature values; when the long-time variance feature values are LTSV feature values, the calculation expression of the long-time variance feature values comprises: ； ； wherein, denotes the frame, denotes the frame, denotes the frame; denotes the frequency bin corresponding to the Fourier coefficient of the fast Fourier transform of the audio signal to be detected; denotes the , corresponding spectrum; denotes the , corresponding long-term spectral energy; denotes the , corresponding long-term spectral energy; denotes the , corresponding long-term spectral energy; denotes the first forgetting factor; denotes the second forgetting factor.

2. The method of claim 1, wherein, the long-time feature values are LTSF feature values; the simulating of the long-time features of the audio signal to be detected by using the forgetting factor to obtain the long-time feature values comprises: determining spectral energy of the audio signal to be detected based on a first forgetting factor; determining an arithmetic mean and a geometric mean of the spectral energy based on a second forgetting factor; wherein the spectral energy, the arithmetic mean of the spectral energy, and the geometric mean of the spectral energy are the long-time features; obtaining the LTSF feature values based on the arithmetic mean and the geometric mean of the spectral energy.

3. The method of claim 1, wherein, the long-time feature values are LTSV feature values; the simulating of the long-time features of the audio signal to be detected by using the forgetting factor to obtain the long-time feature values comprises: determining spectral energy of the audio signal to be detected based on a first forgetting factor and a second forgetting factor; wherein the spectral energy is the long-time feature; determining entropy of a current frame and average entropy of the current frame based on the spectral energy, and obtaining the LTSV feature values based on the entropy of the current frame and the average entropy of the current frame.

4. The method of claim 1, wherein, the simulating of the long-time features of the audio signal to be detected by using the forgetting factor to obtain the long-time feature values comprises: simulating long-time features of the audio signal to be detected by using a forgetting factor; processing the long-time features based on a variance algorithm to obtain long-time variance feature values; wherein the long-time variance feature values are the long-time feature values.

5. The method of claim 4, wherein, the long-time variance feature values are LTSF feature values; the calculation expression of the long-time variance feature values further comprises: ； ； ； ； wherein denotes a parameter , a corresponding variance value; denotes smoothing ; accordingly, denotes smoothing , denotes a parameter , a corresponding variance value; denotes a first frame average value in the frequency axis, denotes a total frequency point number; denotes the long-time variance eigenvalue; denotes a first forgetting factor; denotes a second forgetting factor.

6. The method of claim 4, wherein, The long-time variance eigenvalue is an LTSV-based eigenvalue; The calculation expression of the long-time variance eigenvalue further comprises: ； ； ； ； wherein denotes a parameter , a corresponding variance value; denotes smoothing ; accordingly, denotes smoothing , denotes a parameter , a corresponding variance value; denotes a first frame average value in the frequency axis, denotes a total frequency point number; denotes the long-time variance eigenvalue; denotes a first forgetting factor; denotes a second forgetting factor.

7. The method according to any one of claims 1 to 6, characterized in that, Before the comparing of the long-time eigenvalue with the preset threshold value, the method further comprises: Based on the characteristic frequency of the long-time eigenvalue and the background noise energy, the threshold value is dynamically adjusted.

8. A voice activity detection apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire a to-be-detected audio signal; A feature extraction module is configured to perform long-time feature extraction on the to-be-detected audio signal to obtain a long-time eigenvalue; The long-time eigenvalue is an LTSF eigenvalue or an LTSV eigenvalue; A decision module is configured to compare the long-time eigenvalue with a preset threshold value to determine a voice activity detection result. The feature extraction module is configured to simulate long-time features of the to-be-detected audio signal by using a forgetting factor to obtain the long-time eigenvalue. The long-time eigenvalue comprises a long-time variance eigenvalue; when the long-time variance eigenvalue is the LTSF eigenvalue, the feature extraction module is configured to determine spectral energy of the to-be-detected audio signal based on a first forgetting factor and determine an arithmetic mean of the spectral energy based on a second forgetting factor. The spectral energy and the arithmetic mean of the spectral energy are processed based on a variance algorithm to obtain the long-time variance eigenvalue. When the long-time variance eigenvalue is the LTSF eigenvalue, the calculation expression of the long-time variance eigenvalue comprises: ； ； wherein, denotes the frame, denotes the frame; denotes the frequency bin corresponding to the Fourier coefficient of the audio signal to be detected obtained by the fast Fourier transform; denotes the , corresponding spectrum; denotes the , corresponding long-term spectral energy; denotes the , corresponding long-term spectral energy; denotes the , corresponding arithmetic mean of the energies; denotes the , corresponding arithmetic mean of the energies; denotes the first forgetting factor; When the long-time variance eigenvalue is an LTSV-based eigenvalue, the feature extraction module is configured to determine spectral energy of the to-be-detected audio signal based on the first forgetting factor and process the spectral energy based on the variance algorithm to obtain the long-time variance eigenvalue. The long-time variance eigenvalue is an LTSV-based eigenvalue; the calculation expression of the long-time variance eigenvalue comprises: ； ； wherein, represents the frame, represents the frame, represents the frame; represents the frequency bin corresponding to the Fourier coefficient of the audio signal to be detected obtained by fast Fourier transform; represents the , corresponding spectrum; represents the , corresponding long-time spectrum energy; represents the , corresponding long-time spectrum energy; represents the , corresponding long-time spectrum energy; represents the first forgetting factor; represents the second forgetting factor.

9. An electronic device, comprising: Comprise: A processor and a memory, the processor and the memory are connected; The memory is configured to store a program; The processor is configured to run the program stored in the memory to execute the method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program executes the method in any one of claims 1-7 when being run by a computer.

Images