Speech detection method and apparatus, terminal device, and storage medium

Abstract

The application discloses a speech detection method and device, a terminal equipment and a computer readable storage medium. The method comprises the following steps: performing frame processing on an original audio signal to obtain each frame signal; calculating an energy gradient factor of each frame signal, wherein the energy gradient factor is the sum of energy gradients between adjacent two frequency points in each frequency point of the frame signal; determining a target energy gradient factor threshold corresponding to a target frame signal in each frame signal; and determining that the target frame signal is a speech signal when the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold. The application provides a speech detection scheme using frequency domain gradient information of a signal as a feature to improve the stability of a VAD algorithm in a noise scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio processing, and in particular relates to a voice detection method, apparatus, terminal equipment, and computer-readable storage medium. Background Art

[0002] With the rapid development of science and technology, audio is closely related to many aspects of people's lives, and audio processing technology is constantly improving and perfecting.

[0003] Currently, the VAD (Voice Activity Detection) algorithm is used to detect speech and non-speech segments from pure or noisy speech signals. It plays a crucial role in speech signal processing and is therefore widely used in speech enhancement, call noise reduction, speech recognition, and other fields. VAD algorithms typically consist of two parts: first, feature selection and extraction, and second, the distinction between speech and non-speech. However, VAD features primarily consist of short-term energy and short-term zero-crossing rate. These features are susceptible to noise interference, resulting in low stability in existing VAD algorithms in noisy environments. Summary of the Invention

[0004] The main purpose of the present invention is to provide a speech detection method, apparatus, terminal device, and computer-readable storage medium. The purpose is to provide a speech detection solution that utilizes the frequency domain gradient information of a signal as a feature to improve the stability of the VAD algorithm in noisy scenarios.

[0005] In order to achieve the above object, the present invention provides a speech detection method, which comprises:

[0006] Performing frame processing on the original audio signal to obtain each frame signal;

[0007] Calculating an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal;

[0008] Determining a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals;

[0009] When the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold, it is determined that the target frame signal is a speech signal.

[0010] Optionally, the step of determining a target energy gradient factor threshold corresponding to a target frame signal in each frame signal includes:

[0011] Acquiring a signal-to-noise ratio of a target frame signal in each of the frame signals;

[0012] A target energy gradient factor threshold corresponding to the target frame signal is calculated according to the signal-to-noise ratio of the target frame signal, wherein the target energy gradient factor threshold is smaller when the signal-to-noise ratio of the target frame signal is larger.

[0013] Optionally, the step of obtaining the signal-to-noise ratio of the target frame signal in each frame signal includes:

[0014] When the target frame signal is the first frame signal among the frame signals, calculating the signal-to-noise ratio of the target frame signal according to a preset initial noise energy estimate and the frequency energy of each frequency point of the target frame signal;

[0015] When the target frame signal is not the first frame signal among the frame signals, the signal-to-noise ratio of the target frame signal is determined according to a speech detection result of a first frame signal that precedes the target frame signal.

[0016] Optionally, the step of determining the signal-to-noise ratio of the target frame signal according to the speech detection result of the first frame signal arranged before the target frame signal includes:

[0017] When a speech detection result of a first frame signal arranged before the target frame signal is a speech signal, taking a first signal-to-noise ratio of the first frame signal as a signal-to-noise ratio of the target frame signal;

[0018] When the speech detection result of the first framed signal is a non-speech signal, a first noise energy estimation value of the first framed signal is calculated, and a signal-to-noise ratio of the target framed signal is calculated based on the first noise energy estimation value and the frequency energy of each frequency point of the target framed signal.

[0019] Optionally, the step of determining a first noise energy estimation value of the first framed signal includes:

[0020] determining a second noise energy estimation value of a second frame signal that precedes the first frame signal;

[0021] determining a first signal-to-noise ratio of the first framed signal;

[0022] A first noise energy estimation value of the first frame signal is calculated according to the second noise energy estimation value, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first frame signal.

[0023] Optionally, after the step of determining a target energy gradient factor threshold corresponding to a target frame signal in each frame signal, the method further includes:

[0024] When the target energy gradient factor is less than the target energy gradient factor threshold, determining third frame signals of a first preset number of consecutive frames arranged before the target frame signal, and detecting whether the energy gradient factors of the third frame signals are all less than the energy gradient factor threshold corresponding to each of the third frame signals;

[0025] If yes, determining that the target frame signal is a non-speech signal;

[0026] If not, the step of determining that the target frame signal is a speech signal is performed.

[0027] Optionally, the step of detecting whether the energy gradient factors of the third frame signals are all less than energy gradient factor thresholds corresponding to the third frame signals includes:

[0028] After decrementing the current count value by one, detecting whether the count value is less than or equal to zero, wherein before performing voice detection on the first frame signal in each of the frame signals, the count value is set to the first preset frame number;

[0029] If the count value is less than or equal to zero, determining that the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0030] If the count value is greater than zero, determining that the energy gradient factors of the third frame signals are not all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0031] After the step of determining that the target frame signal is a speech signal, the method further includes:

[0032] The count value is reset to the first preset frame number.

[0033] In addition, to achieve the above-mentioned object, the present invention further provides a speech detection device, the speech detection device comprising:

[0034] A frame processing module is used to perform frame processing on the original audio signal to obtain each frame signal;

[0035] An energy gradient factor module, configured to calculate an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal;

[0036] a target energy gradient factor threshold module, configured to determine a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals;

[0037] The speech signal module is configured to determine that the target frame signal is a speech signal when the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes: a memory, a processor, and a voice detection program stored on the memory and runnable on the processor. When the voice detection program of the terminal device is executed by the processor, the steps of the voice detection method described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a speech detection program is stored. When the speech detection program is executed by a processor, the steps of the speech detection method described above are implemented.

[0040] The embodiment of the present invention obtains each segment of the framed signal by framing the original audio signal, and then calculates the energy gradient factor of each framed signal, wherein the energy gradient factor of the framed signal is the sum of the energy gradients between two adjacent frequency points in each frequency point of the framed signal, determines the target energy gradient factor threshold corresponding to the target framed signal in each framed signal, judges the relationship between the energy gradient factor of the target framed signal and the target energy gradient factor threshold corresponding to the target framed signal, and determines that the target framed signal is a speech signal when the energy gradient factor of the target framed signal is greater than or equal to the target energy gradient factor threshold. In this way, the embodiment of the present invention provides a solution for speech detection using the frequency domain gradient information of the signal as a feature to improve the stability of the VAD algorithm in noisy scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the device structure of the hardware operating environment of the terminal device involved in the embodiment of the present invention;

[0042] Figure 2 1. A schematic flow chart of the steps of the first embodiment of the speech detection method of the present invention;

[0043] Figure 3 A schematic diagram of a short-time Fourier transform according to an embodiment of a speech detection method of the present invention;

[0044] Figure 4 Schematic diagram of quiet and speech point energies involved in an embodiment of a speech detection method of the present invention;

[0045] Figure 5 A schematic diagram of noise and speech frequency point energy involved in an embodiment of a speech detection method of the present invention;

[0046] Figure 6Schematic diagram of the VAD algorithm process involved in one embodiment of the speech detection method of the present invention;

[0047] Figure 7 A schematic diagram of original speech involved in an embodiment of a speech detection method of the present invention;

[0048] Figure 8 Schematic diagram of VAD judgment results in a quiet scene according to an embodiment of the speech detection method of the present invention;

[0049] Figure 9 Schematic diagram of VAD judgment results in a scenario where the signal-to-noise ratio is 20 decibels according to an embodiment of the speech detection method of the present invention;

[0050] Figure 10 Schematic diagram of VAD judgment results in a scenario where the signal-to-noise ratio is 10 decibels according to an embodiment of the speech detection method of the present invention;

[0051] Figure 11 Schematic diagram of VAD decision results in a scenario where the signal-to-noise ratio is 5 dB according to an embodiment of the speech detection method of the present invention;

[0052] Figure 12 Schematic diagram of VAD decision results in a scenario where the signal-to-noise ratio is 3 dB according to an embodiment of the speech detection method of the present invention;

[0053] Figure 13 FIG. 1 is a schematic diagram of functional modules of a speech detection device according to an embodiment of the present invention.

[0054] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment of the terminal device involved in the embodiment of the present invention.

[0057] It should be noted that the terminal device in the embodiment of the present invention can be a headset, a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.

[0058] like Figure 1As shown, the terminal device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art will understand that Figure 1 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0060] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a voice detection program.

[0061] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the voice detection program stored in the memory 1005 and perform the following operations:

[0062] Performing frame processing on the original audio signal to obtain each frame signal;

[0063] Calculating an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal;

[0064] Determining a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals;

[0065] When the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold, it is determined that the target frame signal is a speech signal.

[0066] Furthermore, the operation of determining a target energy gradient factor threshold corresponding to a target frame signal in each frame signal includes:

[0067] Acquiring a signal-to-noise ratio of a target frame signal in each of the frame signals;

[0068] A target energy gradient factor threshold corresponding to the target frame signal is calculated according to the signal-to-noise ratio of the target frame signal, wherein the target energy gradient factor threshold is smaller when the signal-to-noise ratio of the target frame signal is larger.

[0069] Furthermore, the operation of obtaining the signal-to-noise ratio of the target frame signal in each of the frame signals includes:

[0070] When the target frame signal is the first frame signal among the frame signals, calculating the signal-to-noise ratio of the target frame signal according to a preset initial noise energy estimate and the frequency energy of each frequency point of the target frame signal;

[0071] When the target frame signal is not the first frame signal among the frame signals, the signal-to-noise ratio of the target frame signal is determined according to a speech detection result of a first frame signal that precedes the target frame signal.

[0072] Furthermore, the operation of determining the signal-to-noise ratio of the target frame signal according to the speech detection result of the first frame signal arranged before the target frame signal includes:

[0073] When a speech detection result of a first frame signal arranged before the target frame signal is a speech signal, taking a first signal-to-noise ratio of the first frame signal as a signal-to-noise ratio of the target frame signal;

[0074] When the speech detection result of the first framed signal is a non-speech signal, a first noise energy estimation value of the first framed signal is calculated, and a signal-to-noise ratio of the target framed signal is calculated based on the first noise energy estimation value and the frequency energy of each frequency point of the target framed signal.

[0075] Further, the operation of determining the first noise energy estimation value of the first framed signal includes:

[0076] determining a second noise energy estimation value of a second frame signal that precedes the first frame signal;

[0077] determining a first signal-to-noise ratio of the first framed signal;

[0078] A first noise energy estimation value of the first frame signal is calculated according to the second noise energy estimation value, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first frame signal.

[0079] Furthermore, after the step of determining the target energy gradient factor threshold corresponding to the target frame signal in each frame signal, the processor 1001 may also be configured to call a speech detection program stored in the memory 1005 to perform the following operations:

[0080] When the target energy gradient factor is less than the target energy gradient factor threshold, determining third frame signals of a first preset number of consecutive frames arranged before the target frame signal, and detecting whether the energy gradient factors of the third frame signals are all less than the energy gradient factor threshold corresponding to each of the third frame signals;

[0081] If yes, determining that the target frame signal is a non-speech signal;

[0082] If not, the step of determining that the target frame signal is a speech signal is performed.

[0083] Furthermore, the operation of detecting whether the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals includes:

[0084] After decrementing the current count value by one, detecting whether the count value is less than or equal to zero, wherein before performing voice detection on the first frame signal in each of the frame signals, the count value is set to the first preset frame number;

[0085] If the count value is less than or equal to zero, determining that the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0086] If the count value is greater than zero, determining that the energy gradient factors of the third frame signals are not all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0087] After the step of determining that the target frame signal is a speech signal, the processor 1001 may also be configured to call a speech detection program stored in the memory 1005 to perform the following operations:

[0088] The count value is reset to the first preset frame number.

[0089] Based on the above structure, various embodiments of the speech detection method are proposed.

[0090] Please refer to Figure 2 , Figure 2This is a flow chart of the first embodiment of the voice detection method of the present invention. It should be noted that although a logical order is shown in the flow chart, in some cases, the voice detection method of the present invention can of course also perform the steps shown or described in a different order than here. In this embodiment, the execution subject of the voice detection method can be a personal computer, a smart phone or other device, which is not limited in this embodiment. For the sake of ease of description, the following description of each embodiment is omitted. In this embodiment, the voice detection method includes:

[0091] Step S10, performing frame processing on the original audio signal to obtain each frame signal;

[0092] An input audio signal (hereinafter referred to as an original audio signal for distinction) is framed to obtain multiple segments of framed audio signals (hereinafter referred to as framed signals for distinction).

[0093] In one feasible implementation, the input signal is framed in the time domain. The length of each frame and the overlap length can be set as needed. The frame length is usually between 7.5ms and 30ms. Each frame data is short-time Fourier transformed to obtain frequency domain data. Subsequent processing is performed in the frequency domain. Figure 3 As shown in , the original audio signal is framed and each frame signal is subjected to short-time Fourier transform processing to obtain each frame signal in the frequency domain. Figure 7 The original speech shown can be input as the original audio signal.

[0094] It should be noted that the original audio signal can be divided into 10,000 frames or any number of frames.

[0095] Step S20, calculating the energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of the energy gradients between two adjacent frequency points in each frequency point of the framed signal;

[0096] An energy gradient factor of each frame signal is calculated, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the frame signal.

[0097] It should be noted that, through observation, we can find that there are obvious resonance peaks in the speech signal spectrum, while there are no resonance peaks in quiet scenes or quasi-steady-state noise scenes. At the same time, there are obvious intervals between the resonance peaks, which can be inferred that the energy gradient value between frequency points will be larger, while the gradient value in quiet scenes will be smaller, such as Figure 4 On the other hand, in the case of stationary or quasi-stationary noise, as Figure 5 As shown in FIG, the superimposed signal-to-noise ratio is 5DB Gaussian noise, and the energy between the frequency points in the non-speech segment is relatively flat, so the calculated intra-frame energy gradient value is still small.

[0098] In one possible implementation, the energy gradient factor is denoted as eg and is defined as follows:

[0099]

[0100] Where, X k is the FFT value of the kth frequency point, |X k | 2 For its energy, |X k | 2 -|X k-1 | 2 It is the discrete energy gradient, N is the number of FFT points, and L is the number of frequency points calculated. It should be noted that since the frequency range of human voice is mainly below 4KHz, L usually only needs to be calculated to 4K HZ, so:

[0101]

[0102] In order to avoid calculation errors caused by interference in individual frame signals, eg is smoothed and filtered:

[0103] e.g. smooth (n)=λ·eg(n-1)+(1-λ)·eg(n)

[0104] n represents the frame number of each frame signal obtained by performing frame processing on the original audio signal.

[0105] Step S30, determining a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals;

[0106] For any frame signal among the frame signals (hereinafter referred to as a target frame signal for distinction), a target energy gradient factor threshold corresponding to the target frame signal is calculated.

[0107] Step S40: When the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold, determine that the target frame signal is a speech signal.

[0108] The energy gradient factor of the target frame signal is compared with the target energy gradient factor threshold, and when the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold, the target frame signal is determined to be a speech signal.

[0109] In one feasible implementation, the target energy gradient factor threshold is denoted as eg th , in eg n >=eg th When VAD n =1, it is determined that the target frame signal with frame number n is a speech signal.

[0110] It should be noted that when VAD=1, the current frame signal is determined to be a speech signal, and when VAD=0, the current frame signal is determined to be a non-speech signal.

[0111] In this way, the embodiment of the present invention obtains each segment of the framed signal by framing the original audio signal, and then calculates the energy gradient factor of each framed signal, wherein the energy gradient factor of the framed signal is the sum of the energy gradients between two adjacent frequency points in each frequency point of the framed signal, determines the target energy gradient factor threshold corresponding to the target framed signal in each framed signal, judges the relationship between the energy gradient factor of the target framed signal and the target energy gradient factor threshold corresponding to the target framed signal, and determines that the target framed signal is a speech signal when the energy gradient factor of the target framed signal is greater than or equal to the target energy gradient factor threshold. In this way, the embodiment of the present invention provides a solution for speech detection using the frequency domain gradient information of the signal as a feature to improve the stability of the VAD algorithm in noisy scenarios.

[0112] Furthermore, based on the first embodiment of the speech detection method of the present invention, a second embodiment of the speech detection method of the present invention is proposed.

[0113] In this embodiment, step S30 includes:

[0114] Step S301, obtaining a signal-to-noise ratio of a target frame signal in each of the frame signals;

[0115] For a target frame signal in each frame signal, a signal-to-noise ratio of the target frame signal is obtained.

[0116] In one possible implementation, ratio n It represents the ratio of signal to noise in the frame signal with frame number n, that is, the signal-to-noise ratio of the frame signal of the nth frame.

[0117] Step S302 : calculating a target energy gradient factor threshold corresponding to the target frame signal according to the signal-to-noise ratio of the target frame signal, wherein the larger the signal-to-noise ratio, the smaller the target energy gradient factor threshold.

[0118] An energy gradient factor threshold corresponding to the target frame signal is calculated according to the signal-to-noise ratio of the target frame signal, wherein the relationship between the signal-to-noise ratio of the target frame signal and the target energy gradient factor threshold is: when the signal-to-noise ratio is larger, the target energy gradient factor threshold is smaller.

[0119] In a feasible implementation, the energy gradient factor threshold is denoted as eg th , the value is calculated as follows:

[0120]

[0121] Among them, k and c are constant coefficients, and Δ is generally set to a small constant value to avoid division by zero.

[0122] Furthermore, in a feasible embodiment, step S301 includes:

[0123] Step A10: When the target frame signal is the first frame signal among the frame signals, the signal-to-noise ratio of the target frame signal is calculated based on a preset initial noise energy estimate and the frequency energy of each frequency point of the target frame signal.

[0124] When the target frame signal is the frame signal that is ranked first among the frame signals, a preset noise energy estimate value (hereinafter referred to as the initial noise energy estimate value for distinction) is determined, and the signal-to-noise ratio of the target frame signal is calculated based on the initial noise energy estimate value and the frequency energy of each frequency point of the target frame signal.

[0125] In one feasible implementation, an arrangement order of the framed signals is determined, and it is detected whether the target framed signal is the framed signal arranged first among the framed signals. When the target framed signal is the framed signal arranged first among the framed signals, a signal-to-noise ratio of the target framed signal is calculated based on a preset initial noise energy estimate and the frequency energy of each frequency point of the target framed signal. The signal-to-noise ratio calculation formula of the framed signal is:

[0126]

[0127] The target frame signal at this time is the frame signal arranged in the first place, that is, the first signal-to-noise ratio is the signal-to-noise ratio of the frame signal with frame number n equal to 1. Then in the above formula, when calculating ratio1, E n-1 E0 is the initial noise energy estimation value preset based on the empirical value. It should be noted that E n Indicates the noise energy estimation value of the target frame signal with frame number n.

[0128] Step A20: When the target frame signal is not the first frame signal among the frame signals, determine the signal-to-noise ratio of the target frame signal according to the speech detection result of the first frame signal that precedes the target frame signal.

[0129] When the target frame signal is not the frame signal arranged first among the frame signals, the frame signal arranged before the target frame signal (hereinafter referred to as the first frame signal for distinction) is determined, and the signal-to-noise ratio of the target frame signal is determined based on the speech detection result of the first frame signal.

[0130] In one feasible implementation, when the speech detection result of the first frame signal is a speech signal, that is, the first frame signal is a speech signal, the target energy gradient factor threshold of the first frame signal is used as the target energy gradient factor threshold of the target frame signal; when the speech detection result of the first frame signal is a speech signal, that is, the first frame signal is a non-speech signal, the target energy gradient factor threshold of the target frame signal is recalculated according to the calculation formula of the target energy gradient factor threshold.

[0131] Furthermore, in a feasible embodiment, in step A20, the step of “determining the signal-to-noise ratio of the target frame signal based on the speech detection result of the first frame signal arranged before the target frame signal” includes:

[0132] Step A201, when the speech detection result of the first frame signal arranged before the target frame signal is a speech signal, taking the first signal-to-noise ratio of the first frame signal as the signal-to-noise ratio of the target frame signal;

[0133] When the speech detection result of the first frame signal arranged before the target frame signal is a speech signal, the signal-to-noise ratio of the first frame signal (hereinafter referred to as the first signal-to-noise ratio for distinction) is obtained, and the first signal-to-noise ratio is used as the signal-to-noise ratio of the target frame signal.

[0134] In a feasible implementation, when the speech detection result of the 11th frame is a speech signal, the first signal-to-noise ratio of the 10th frame is obtained, and the first signal-to-noise ratio is used as the signal-to-noise ratio of the 12th frame.

[0135] Step A202, when the speech detection result of the first frame signal is a non-speech signal, calculate the first noise energy estimation value of the first frame signal, and calculate the signal-to-noise ratio of the target frame signal based on the first noise energy estimation value and the frequency energy of each frequency point of the target frame signal.

[0136] When the speech detection result of the first frame signal is a non-speech signal, a noise energy estimate value of the first frame signal (hereinafter referred to as the first noise energy estimate value for distinction) is calculated, and then the signal-to-noise ratio of the target frame signal is calculated based on the first noise energy estimate value and the frequency energy of each frequency point of the target frame signal.

[0137] In one feasible implementation, when the speech detection result of the 11th frame is a non-speech signal, the first noise energy estimate of the 11th frame is calculated, and then the first noise energy estimate and the frequency energy of each frequency point of the 12th frame signal are substituted into the signal-to-noise ratio calculation formula to obtain the signal-to-noise ratio of the 12th frame.

[0138] Furthermore, in a feasible embodiment, in step A202, the step of "determining a first noise energy estimation value of the first framed signal" includes:

[0139] Step B10, determining a second noise energy estimation value of a second frame signal that precedes the first frame signal;

[0140] A frame signal arranged before the first frame signal (hereinafter referred to as the second frame signal for distinction) is determined, and a noise energy estimation value of the second frame signal (hereinafter referred to as the second noise energy estimation value for distinction) is obtained.

[0141] In one feasible implementation, if the second frame signal is the 10th frame signal, the signal-to-noise ratio of the 10th frame signal, the noise energy estimation value of the 9th frame signal, and the frequency energy of each frequency point of the 10th frame signal are substituted into the calculation formula of the noise energy estimation value to obtain the noise energy estimation value of the 10th frame. n The calculation formula is:

[0142]

[0143] Among them, β is a constant coefficient selected based on experimental experience.

[0144] In one possible implementation, Figure 8 As shown in Figure 2, the VAD algorithm determines the input signal in a quiet scene; Figure 9 As shown in Figure 2, the VAD algorithm determines the input signal in a scenario where the signal-to-noise ratio is 20 dB. Figure 10 As shown in Figure 2, the VAD algorithm determines the input signal in a scenario where the signal-to-noise ratio is 10 dB. Figure 11 As shown in Figure 2, the VAD algorithm determines the input signal in a scenario where the signal-to-noise ratio is 5 dB. Figure 12 As shown in FIG, the decision result of the VAD algorithm on the input signal in the scenario where the signal-to-noise ratio is 3 dB. Therefore, the improved VAD algorithm proposed in this application can realize the decision of speech signals in various scenarios.

[0145] Step B20, determining a first signal-to-noise ratio of the first framed signal;

[0146] A signal-to-noise ratio (SNR) of a first frame signal (hereinafter referred to as a first SNR for distinction) is obtained.

[0147] In one feasible implementation, if the first frame signal is the 11th frame signal, then the noise energy estimation value of the 10th frame signal and the frequency energy of each frequency point of the 11th frame signal are substituted into the signal-to-noise ratio calculation formula to obtain the first signal-to-noise ratio of the 11th frame signal.

[0148] Step B30: Calculate a first noise energy estimate of the first frame signal according to the second noise energy estimate, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first frame signal.

[0149] A first noise energy estimation value of the first frame signal is calculated according to the second noise energy estimation value, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first frame signal.

[0150] In one feasible implementation, the noise energy estimation value of the 10th frame, the signal-to-noise ratio of the 11th frame, and the frequency energy of each frequency point of the 11th frame are substituted into the noise energy estimation formula to obtain a first noise energy estimation value of the 11th frame.

[0151] In this embodiment, the speech detection method of the present invention obtains the signal-to-noise ratio of the target frame signal in each frame signal, and then calculates the energy gradient factor threshold corresponding to the target frame signal based on the signal-to-noise ratio of the target frame signal, wherein the relationship between the signal-to-noise ratio of the target frame signal and the target energy gradient factor threshold is: the larger the signal-to-noise ratio, the smaller the target energy gradient factor threshold; when the target frame signal is the frame signal ranked first among the frame signals, the signal-to-noise ratio of the target frame signal is calculated based on the initial noise energy estimate and the frequency energy of each frequency point of the target frame signal; when the target frame signal is not the frame signal ranked first among the frame signals, the first frame signal ranked before the target frame signal is determined, and based on the energy of the target energy gradient factor threshold. A signal-to-noise ratio (SNR) of a target framed signal is determined based on a speech detection result of the first framed signal; when the speech detection result of the first framed signal is a speech signal, a first SNR of the first framed signal is obtained, and the first SNR is used as the SNR of the target framed signal; when the speech detection result of the first framed signal is a non-speech signal, a first noise energy estimate of the first framed signal is calculated, and then the SNR of the target framed signal is calculated based on the first noise energy estimate and the frequency energy of each frequency point of the target framed signal, and a second noise energy estimate of the second framed signal is determined; and finally, a first noise energy estimate of the first framed signal is calculated based on the first SNR of the first framed signal, the second noise energy estimate, the first SNR, and the frequency energy of each frequency point of the first framed signal.

[0152] In this way, the present invention avoids the inconvenience of manually adjusting the threshold according to the differences in products and input signals to optimize the effect by realizing the adaptation of the energy gradient factor threshold, and solves the problem that the existing fixed threshold cannot adapt well to and take into account speech detection in different noise scenarios.

[0153] Furthermore, based on the first embodiment and / or the second embodiment of the speech detection method of the present invention, a third embodiment of the speech detection method of the present invention is proposed.

[0154] In this embodiment, after step S30, the following steps are further included:

[0155] Step C10, when the target energy gradient factor is less than the target energy gradient factor threshold, determining third frame signals arranged for a first preset number of consecutive frames before the target frame signal, and detecting whether the energy gradient factors of each of the third frame signals are less than the energy gradient factor threshold corresponding to each of the third frame signals;

[0156] When the target energy gradient factor of the target frame signal is less than the target energy gradient factor threshold corresponding to the target frame signal, determine the frame signal (hereinafter referred to as the third frame signal for distinction) of the consecutive frame number (hereinafter referred to as the first preset frame number for distinction) arranged before the target frame signal, and detect whether the energy gradient factors of each third frame signal are less than the energy gradient factor threshold corresponding to each third frame signal.

[0157] In a feasible implementation, the first preset number of frames is usually set to about 10-50 frames. For example, vad-hold is set to 10, and the current target frame signal is the 11th frame signal. Then, it is detected whether the energy gradient factors of the first frame to the 10th frame signals are all less than the target energy gradient factor thresholds corresponding to the first frame to the 10th frame signals.

[0158] Step C20: If yes, determine that the target frame signal is a non-speech signal;

[0159] If the energy gradient factors of the third frame signals arranged before the target frame signal for the first preset number of consecutive frames are all smaller than the energy gradient factor thresholds corresponding to the third frame signals, then the current target frame signal is determined to be a non-speech signal.

[0160] In a feasible implementation, if the target frame signal is the 11th frame, then when it is detected that the frame signals of the first 10 frames are all non-speech signals, the speech detection result of the 11th frame is determined to be a non-speech signal.

[0161] Step C30: If not, then executing the step of determining that the target frame signal is a speech signal.

[0162] If the energy gradient factors of the third frame signals arranged before the target frame signal for a first preset number of consecutive frames are all less than the energy gradient factor thresholds corresponding to the third frame signals, the current target frame signal is determined to be a speech signal.

[0163] In a feasible implementation, if the target frame signal is the 11th frame, then when it is detected that the frame signals of the first 10 frames are not all non-speech signals, the speech detection result of the 11th frame is determined to be a speech signal.

[0164] Furthermore, in a feasible embodiment, in step C10, the step of “detecting whether the speech detection results of the first preset number of consecutive frame signals before the target frame signal are all non-speech signals” includes:

[0165] Step C101, after decrementing a current count value by one, detecting whether the count value is less than or equal to zero, wherein before performing voice detection on the first frame signal in each of the frame signals, the count value is set to the first preset number of frames;

[0166] When the target energy gradient factor of the target frame signal is less than the target energy gradient factor threshold corresponding to the target frame signal, the count value used for speech detection (hereinafter referred to as the count value for distinction) is decremented by one, wherein, when the current target frame signal is the frame signal ranked first among the frame signals, the count value is the first preset frame number.

[0167] In one feasible implementation, a variable vad-hold is set, which is the above-mentioned count value. When performing speech detection on the first frame signal, the count value is set to the first preset frame number. For example, when the target energy gradient factor of the target frame signal is less than the target energy gradient factor threshold, vad-hold is decremented by 1, and vad-hold is obtained to be equal to 9. Therefore, when the detection results of the current 10 frames of the frame signal are all non-speech signals, vad-hold = 0.

[0168] Step C102: If the count value is less than or equal to zero, determining that the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0169] When the count value is less than or equal to zero, it is determined that the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals.

[0170] In a feasible implementation, if the current target frame signal is the 13th frame signal and the detected count value is -2, then the count value -2 is less than 0, and it can be determined that the energy gradient factors of the 10 frames arranged before the 13th frame signal are all less than the energy gradient factor thresholds corresponding to the first 10 frame signals.

[0171] Step C103: if the count value is greater than zero, determining that the energy gradient factors of the third frame signals are not all less than the energy gradient factor thresholds corresponding to the third frame signals;

[0172] When the count value is greater than zero, it is determined that the energy gradient factors of the third frame signals are all less than energy gradient factor thresholds corresponding to the third frame signals.

[0173] In a feasible implementation, if the current target frame signal is the 13th frame signal and the detected count value is 7, then the count value 7 is greater than 0, and it can be determined that the energy gradient factors of the 10 frames arranged before the 13th frame signal are all less than the energy gradient factor thresholds corresponding to the first 10 frame signals.

[0174] In one possible implementation, Figure 6 As shown in the VAD algorithm flow chart, first, the input signal is framed in the time domain to obtain frame signals of each frame, and then STFT (short-time Fourier transform) is performed to convert the time domain frame signals into frequency domain frame signals. Then, the energy gradient factor eg of each frame signal is calculated to determine whether the eg of the frame signal is less than the energy gradient factor threshold corresponding to the frame signal. If the eg of the frame signal is less than the energy gradient factor threshold corresponding to the frame signal, then vad-hold is decremented by one, and when the next frame signal is not less than the energy gradient factor threshold, it is determined whether vad-hold is less than or equal to 0. If vad-hold is less than or equal to 0, VAD=0 is determined, and the frame signal of the frame is a non-speech signal; if vad-hold is greater than 0, VAD=1 is determined, and the frame signal of the frame is a speech signal. If the eg of the frame signal is not less than the energy gradient factor threshold corresponding to the frame signal, then VAD=1 is determined, and the frame signal of the frame is a speech signal. When VAD=1, vad-hold-init is assigned to vad-hold. It should be noted that vad-hold-init is the first preset frame number.

[0175] In the step S40, after the step of "determining that the target frame signal is a speech signal", the method further includes:

[0176] Step D10: resetting the count value to the first preset frame number.

[0177] When it is determined that the current target frame signal is a speech signal, the count value is reset to a first preset frame number.

[0178] In a feasible implementation, if the current target frame signal is the 13th frame and the count value is 9, then when the energy gradient factor of the 13th frame is greater than the target energy gradient factor threshold corresponding to the 13th frame, the 13th frame signal is determined to be a speech signal, and the count value is reset to 10.

[0179] In this embodiment, the method for speech detection of the present invention determines the third frame signals arranged before the target frame signal for a first preset number of consecutive frames when the target energy gradient factor of the target frame signal is less than the target energy gradient factor threshold corresponding to the target frame signal, and detects whether the energy gradient factors of each third frame signal are less than the energy gradient factor threshold corresponding to each third frame signal; if the energy gradient factors of each third frame signal arranged before the target frame signal for a first preset number of consecutive frames are less than the energy gradient factor threshold corresponding to each third frame signal, then the current target frame signal is determined to be a non-speech signal. When the energy gradient factors of the third frame signals arranged for a first preset number of consecutive frames before the target frame signal are not all smaller than the energy gradient factor threshold corresponding to each third frame signal, the current target frame signal is determined to be a speech signal; when the target energy gradient factor of the target frame signal is less than the target energy gradient factor threshold corresponding to the target frame signal, the count value used for speech detection is decremented by one, wherein, before speech detection is performed on the frame signal arranged first in each of the frame signals, the count value is set to the first preset frame number; when the count value is less than or equal to zero, it is determined that the energy gradient factors of the third frame signals are all less than the energy gradient factor threshold corresponding to each third frame signal; when the count value is greater than zero, it is determined that the energy gradient factors of the third frame signals are not all smaller than the energy gradient factor threshold corresponding to each third frame signal; when it is determined that the current target frame signal is a speech signal, the count value is reset to the first preset frame number.

[0180] In this way, since even a very short speech segment will last for several frames, the duration of the speech for several consecutive frames is used as a supplementary constraint, that is, the relationship between the energy gradient factor of the frame signal of the first preset number of consecutive frames before the current target frame signal and its corresponding energy gradient factor threshold is set as a limiting condition, thereby avoiding frequent VAD jumps.

[0181] In addition, an embodiment of the present invention further provides a speech detection device.

[0182] Please refer to Figure 13 , Figure 13 FIG. 1 is a functional module diagram of an embodiment of a speech detection device according to the present invention. Figure 13 As shown, the speech detection device of the present invention includes:

[0183] The frame processing module 10 is used to perform frame processing on the original audio signal to obtain each frame signal;

[0184] An energy gradient factor module 20 is configured to calculate an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal;

[0185] A target energy gradient factor threshold module 30 is configured to determine a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals;

[0186] The speech signal module 40 is configured to determine that the target frame signal is a speech signal when the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold.

[0187] Furthermore, the target energy gradient factor threshold module 30 further includes:

[0188] a signal-to-noise ratio unit, configured to obtain a signal-to-noise ratio of a target frame signal in each of the frame signals;

[0189] The target energy gradient factor threshold unit is used to calculate the target energy gradient factor threshold corresponding to the target frame signal according to the signal-to-noise ratio of the target frame signal, wherein the larger the signal-to-noise ratio of the target frame signal, the smaller the target energy gradient factor threshold.

[0190] Furthermore, the signal-to-noise ratio unit includes:

[0191] a first signal-to-noise ratio subunit, configured to calculate the signal-to-noise ratio of the target frame signal according to a preset initial noise energy estimate and frequency energy of each frequency point of the target frame signal when the target frame signal is the frame signal ranked first among the frame signals;

[0192] The second signal-to-noise ratio subunit is configured to determine the signal-to-noise ratio of the target frame signal according to a speech detection result of a first frame signal arranged before the target frame signal when the target frame signal is not the frame signal arranged first among the frame signals.

[0193] Furthermore, the second signal-to-noise ratio subunit is further configured to determine a second noise energy estimate of a second framed signal that precedes the first framed signal; determine a first signal-to-noise ratio of the first framed signal; and calculate a first noise energy estimate of the first framed signal based on the second noise energy estimate, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first framed signal.

[0194] Furthermore, the speech detection device of the present invention further includes:

[0195] The first signal-to-noise ratio module is configured to calculate a first signal-to-noise ratio of the target frame signal according to a preset initial noise energy estimation value and frequency energy of each frequency point of the target frame signal.

[0196] Furthermore, the speech detection device of the present invention further includes:

[0197] a frame signal detection module configured to, when the target energy gradient factor is less than the target energy gradient factor threshold, determine third frame signals arranged for a first preset number of consecutive frames preceding the target frame signal, and detect whether the energy gradient factors of the third frame signals are all less than the energy gradient factor threshold corresponding to each of the third frame signals;

[0198] a non-speech signal module, configured to determine that if the target frame signal is a non-speech signal;

[0199] The voice signal module is configured to, if not, execute the step of determining that the target frame signal is a voice signal.

[0200] Furthermore, the frame signal detection module includes:

[0201] a counting unit, configured to decrement a current count value by one and then detect whether the count value is less than or equal to zero, wherein the count value is set to the first preset number of frames before performing voice detection on the first frame signal in each of the frame signals;

[0202] a first detection unit, configured to determine, if the count value is less than or equal to zero, that the energy gradient factors of the third frame signals are all less than energy gradient factor thresholds corresponding to the third frame signals;

[0203] a second detection unit, configured to determine, if the count value is greater than zero, that the energy gradient factors of the third frame signals are not all less than energy gradient factor thresholds corresponding to the third frame signals;

[0204] The speech detection device of the present invention further includes:

[0205] The reset module is configured to reset the count value to the first preset number of frames.

[0206] The present invention also provides a computer storage medium on which a speech detection program is stored. When the speech detection program is executed by a processor, the steps of the speech detection program method described in any of the above embodiments are implemented.

[0207] The specific embodiments of the computer storage medium of the present invention are basically the same as the above-mentioned embodiments of the speech detection program method of the present invention, and will not be described in detail here.

[0208] The present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the speech detection method of the present invention as described in any of the above embodiments, which will not be described in detail here.

[0209] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0210] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0211] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions for enabling a terminal device (which can be a TWS headset, etc.) to execute the methods described in each embodiment of the present invention.

[0212] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A speech detection method, characterized in that: The speech detection method comprises the following steps: Performing frame processing on the original audio signal to obtain each frame signal; Calculating an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal; Determining a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals; When the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold, it is determined that the target frame signal is a speech signal.

2. The speech detection method according to claim 1, wherein: The step of determining the target energy gradient factor threshold corresponding to the target frame signal in each frame signal comprises: Acquiring a signal-to-noise ratio of a target frame signal in each of the frame signals; A target energy gradient factor threshold corresponding to the target frame signal is calculated according to the signal-to-noise ratio of the target frame signal, wherein the target energy gradient factor threshold is smaller when the signal-to-noise ratio of the target frame signal is larger.

3. The speech detection method according to claim 2, wherein: The step of obtaining the signal-to-noise ratio of the target frame signal in each frame signal comprises: When the target frame signal is the first frame signal among the frame signals, calculating the signal-to-noise ratio of the target frame signal according to a preset initial noise energy estimate and the frequency energy of each frequency point of the target frame signal; When the target frame signal is not the first frame signal among the frame signals, the signal-to-noise ratio of the target frame signal is determined according to a speech detection result of a first frame signal that precedes the target frame signal.

4. The speech detection method according to claim 3, wherein: The step of determining the signal-to-noise ratio of the target frame signal according to the speech detection result of the first frame signal arranged before the target frame signal comprises: When a speech detection result of a first frame signal arranged before the target frame signal is a speech signal, taking a first signal-to-noise ratio of the first frame signal as a signal-to-noise ratio of the target frame signal; When the speech detection result of the first framed signal is a non-speech signal, a first noise energy estimation value of the first framed signal is calculated, and a signal-to-noise ratio of the target framed signal is calculated based on the first noise energy estimation value and the frequency energy of each frequency point of the target framed signal.

5. The speech detection method according to claim 4, wherein: The step of calculating the first noise energy estimation value of the first framed signal comprises: determining a second noise energy estimation value of a second frame signal that precedes the first frame signal; determining a first signal-to-noise ratio of the first framed signal; A first noise energy estimation value of the first frame signal is calculated according to the second noise energy estimation value, the first signal-to-noise ratio, and the frequency energy of each frequency point of the first frame signal.

6. The speech detection method according to any one of claims 1 to 5, characterized in that: After the step of determining the target energy gradient factor threshold corresponding to the target frame signal in each frame signal, the method further includes: When the energy gradient factor of the target frame signal is less than the target energy gradient factor threshold, determining third frame signals arranged for a first preset number of consecutive frames before the target frame signal, and detecting whether the energy gradient factors of the third frame signals are all less than the energy gradient factor threshold corresponding to each of the third frame signals; If yes, determining that the target frame signal is a non-speech signal; If not, the step of determining that the target frame signal is a speech signal is performed.

7. The speech detection method according to claim 6, wherein: The step of detecting whether the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals comprises: After decrementing the current count value by one, detecting whether the count value is less than or equal to zero, wherein before performing voice detection on the first frame signal in each of the frame signals, the count value is set to the first preset frame number; If the count value is less than or equal to zero, determining that the energy gradient factors of the third frame signals are all less than the energy gradient factor thresholds corresponding to the third frame signals; If the count value is greater than zero, determining that the energy gradient factors of the third frame signals are not all less than the energy gradient factor thresholds corresponding to the third frame signals; After the step of determining that the target frame signal is a speech signal, the method further includes: The count value is reset to the first preset frame number.

8. A speech detection device, characterized in that: The speech detection device comprises: A frame processing module is used to perform frame processing on the original audio signal to obtain each frame signal; An energy gradient factor module, configured to calculate an energy gradient factor of each of the framed signals, wherein the energy gradient factor is the sum of energy gradients between two adjacent frequency points in each frequency point of the framed signal; a target energy gradient factor threshold module, configured to determine a target energy gradient factor threshold corresponding to a target frame signal in each of the frame signals; The speech signal module is configured to determine that the target frame signal is a speech signal when the energy gradient factor of the target frame signal is greater than or equal to the target energy gradient factor threshold.

9. A terminal device, characterized in that: The terminal device includes: a memory, a processor, and a voice detection program stored in the memory and executable on the processor. When the voice detection program is executed by the processor, the steps of the voice detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a speech detection program, which, when executed by a processor, implements the steps of the speech detection method according to any one of claims 1 to 7.

Images