Sound signal detection method and device, computer readable storage medium, terminal

Abstract

A sound signal detection method and device, a computer readable storage medium and a terminal, the method comprising: determining a plurality of sound signals collected by a plurality of microphones from a same input signal within a first preset time period; determining a correlation coefficient, an error energy cumulative value and an energy difference cumulative value between each two sound signals of the plurality of sound signals; and if one or more of the following conditions are met, determining that wind noise exists in the input signal: at least one of the correlation coefficients is less than a preset correlation threshold, at least one of the error energy cumulative values is greater than a first preset difference value, and at least one of the energy difference cumulative values is greater than a second preset difference value. The present application can accurately determine whether wind noise exists in the sound signal collected by the microphone, and reduce unnecessary voice activity detection caused by excessive environmental wind noise.

Description

Technical Field

[0001] This invention relates to the field of speech processing technology, and in particular to a method and apparatus for detecting sound signals, a computer-readable storage medium, and a terminal. Background Technology

[0002] Voice activity detection (VAD), also known as speech activity detection, is a technique used in speech processing. Specifically, VAD involves identifying and eliminating long periods of silence from an audio signal stream. This technique has wide applications; for example, it can simplify speech processing and remove non-speech segments in VoIP applications, avoiding the encoding and transmission of non-speech data packets, thereby saving computation time and bandwidth resources without degrading service quality. Therefore, researching how to improve the accuracy of extracting speech signals from an audio signal stream is of significant value.

[0003] In existing technologies, voice activity detection typically involves two stages: the first stage is preliminary threshold decision, where the sound signal captured by the microphone is initially judged to a threshold value. If the sound signal exceeds the threshold, the second stage is triggered. The second stage is voice detection, which can be performed frame-by-frame on the sound signal. Once voice is detected, subsequent functions such as speech recognition are triggered. However, the drawback of these existing technologies is that during the preliminary threshold decision in the first stage, if there is excessive wind noise in the environment, even if there is no voice signal in the captured sound signal, the wind noise signal can still easily exceed the threshold and trigger the second stage. This results in the second stage performing voice detection on the wind noise signal, thereby reducing detection accuracy, wasting resources, and increasing power consumption. Other existing technologies also use the method of directly increasing the threshold value in the first stage, but this approach may result in the threshold value being set too high, preventing even genuine voice signals from exceeding the threshold, which also fails to guarantee detection accuracy.

[0004] Therefore, there is an urgent need for a sound signal detection method that can accurately determine whether the sound signal collected by the microphone over a period of time is a wind noise signal, reduce unnecessary voice activity detection caused by excessive environmental wind noise, improve detection accuracy, reduce bandwidth waste, and lower power consumption. Summary of the Invention

[0005] The technical problem solved by this invention is how to accurately determine whether the sound signal collected by the microphone is a wind noise signal, avoid unnecessary voice activity detection in scenarios with excessive wind noise, improve the accuracy of voice signal detection, reduce bandwidth waste, and reduce power consumption.

[0006] To address the aforementioned technical problems, this invention provides a sound signal detection method, comprising the following steps: determining multiple sound signals acquired by multiple microphones from the same input signal within a first preset time period; determining the correlation coefficient, cumulative error energy value, and cumulative energy difference value between every two sound signals in the multiple sound signals, wherein the correlation coefficient is used to indicate the similarity between the sound signals; determining that wind noise exists in the input signal if one or more of the following conditions are met: at least one of the correlation coefficients between every two sound signals in the multiple sound signals is less than a preset correlation threshold; at least one of the cumulative error energy values ​​between every two sound signals in the multiple sound signals is greater than a first preset difference; and at least one of the cumulative energy difference values ​​between every two sound signals in the multiple sound signals is greater than a second preset difference; wherein, when acquiring the sound signals, the spatial positions of each microphone are different.

[0007] Optionally, the sound signal detection method further includes: determining that the input signal is a speech signal when it is determined that there is no wind noise in the input signal and the extreme value of the amplitude of the input signal exceeds a preset threshold range.

[0008] Optionally, after determining that the input signal is a speech signal, the method further includes: extracting keywords from the input signal and performing a search.

[0009] Optionally, determining the correlation coefficient between any two audio signals in the multiple audio signals includes: for each audio signal, performing frame processing using a second preset duration to obtain multiple frames of audio sub-signals corresponding to different acquisition time periods; calculating the initial correlation coefficient between audio sub-signals corresponding to the same acquisition time period in each pair of audio signals; and determining the correlation coefficient between the two audio signals based on the obtained multiple initial correlation coefficients.

[0010] Optionally, the initial correlation coefficient between the sound sub-signals corresponding to the same acquisition time period in each pair of sound signals can be calculated using the following formula:

[0011]

[0012] Where: corrcoef is used to indicate the initial correlation coefficient between the sound sub-signals corresponding to the same acquisition time period in two sound signals, x 1_frame x2 _frame These are used to indicate two frames of audio sub-signals corresponding to the same acquisition time period in two audio signals, respectively. cov() is used to indicate the covariance calculation function, and var() is used to indicate the variance calculation function.

[0013] Optionally, determining the correlation coefficient between the two audio signals based on the obtained multiple initial correlation coefficients includes: taking the average of the obtained multiple initial correlation coefficients as the correlation coefficient between the two audio signals; or taking the median of the obtained multiple initial correlation coefficients as the correlation coefficient between the two audio signals.

[0014] Optionally, each audio signal is acquired using a preset sampling frequency. Determining the cumulative error energy value between every two audio signals in the multiple audio signals includes: for every two audio signals in the multiple audio signals, determining the audio signal difference between sampling points belonging to the same acquisition time in the two audio signals; determining the error energy between sampling points belonging to the same acquisition time based on the audio signal difference, and calculating the sum of the error energy between multiple pairs of sampling points belonging to the same acquisition time in the two signals as the cumulative error energy value between the two audio signals.

[0015] Optionally, the following formula can be used to determine the error energy between sampling points belonging to the same acquisition time based on the difference in the sound signals, and to calculate the sum of the error energies between multiple pairs of sampling points belonging to the same acquisition time in the two signals:

[0016] Delta = x1(t) - x2(t);

[0017] Delta_RSSI=[x1(t)-x2(t)] 2 ;

[0018]

[0019] Where t indicates different acquisition times, x1(t) and x2(t) indicate the audio signals of the sampling points belonging to the same acquisition time in the two audio signals respectively; Delta indicates the audio signal difference between each pair of sampling points belonging to the same acquisition time in the two audio signals; Delta_RSSI indicates the error energy between each pair of sampling points belonging to the same acquisition time in the two audio signals; S_RSSI indicates the sum of the error energy between n pairs of sampling points belonging to the same acquisition time in the two audio signals, where n indicates the number of sampling points belonging to the same acquisition time in the two audio signals.

[0020] Optionally, each audio signal is acquired using a preset sampling frequency; determining the cumulative energy difference between any two audio signals in the multiple audio signals includes: for each pair of audio signals in the multiple audio signals, determining the energy values ​​of the sampling points belonging to the same acquisition time in the two audio signals; calculating the difference in energy values ​​between the sampling points belonging to the same acquisition time in the two audio signals, and then calculating the sum of the differences in energy values ​​between multiple pairs of sampling points belonging to the same acquisition time in the two signals, as the cumulative energy difference between the two audio signals.

[0021] Optionally, the following formula can be used to calculate the difference in energy values ​​between sampling points belonging to the same acquisition time in each pair of the multiple audio signals, and then calculate the sum of the differences in energy values ​​between multiple pairs of sampling points belonging to the same acquisition time in the two audio signals:

[0022] RSSI_x1(t)=x1(t) 2 ;

[0023] RSSI_x2(t)=x2(t) 2 ;

[0024] RSSI_Delta(t)=RSSI_x1(t)-RSSI_x2(t);

[0025]

[0026] Where t indicates different acquisition times, x1(t) and x2(t) indicate the audio signals of the sampling points belonging to the same acquisition time in the two audio signals, respectively; RSSI_x1(t) and RSSI_x2(t) indicate the energy values ​​of each pair of sampling points belonging to the same acquisition time in the two audio signals, respectively; RSSI_Delta(t) indicates the difference in energy values ​​between each pair of sampling points belonging to the same acquisition time in the two audio signals; RSSI_S indicates the sum of the differences in energy values ​​between n pairs of sampling points belonging to the same acquisition time in the two audio signals, where n indicates the number of pairs of sampling points belonging to the same acquisition time in the two audio signals.

[0027] Optionally, the preset correlation threshold, the first preset difference, and the second preset difference are determined based on one or more of the following: wind noise intensity in the acquisition space, sound wave incident angle, microphone spatial position, and microphone frequency response.

[0028] Optionally, before determining the correlation coefficient, cumulative error energy value, and cumulative energy difference value between every two audio signals in the multi-channel audio signals, the method further includes: performing downsampling processing on each audio signal separately.

[0029] This invention also provides a sound signal detection device, comprising: a multi-channel sound signal determination module, configured to determine multiple sound signals acquired by multiple microphones from the same input signal within a first preset time period; a wind noise decision coefficient determination module, configured to determine the correlation coefficient, error energy accumulation value, and energy difference accumulation value between every two sound signals in the multi-channel sound signals, wherein the correlation coefficient is used to indicate the similarity between the sound signals; and a wind noise detection module, configured to determine that wind noise exists in the input signal when one or more of the following conditions are met: at least one of the correlation coefficients between every two sound signals in the multi-channel sound signals is less than a preset correlation threshold; at least one of the error energy accumulation values ​​between every two sound signals in the multi-channel sound signals is greater than a first preset difference value; and at least one of the energy difference accumulation values ​​between every two sound signals in the multi-channel sound signals is greater than a second preset difference value.

[0030] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, executes the steps of the above-described sound signal detection method.

[0031] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the above-described sound signal method when running the computer program.

[0032] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0033] In this embodiment of the invention, firstly, multiple sound signals acquired by multiple microphones from the same input signal within a first preset time period are determined; then, three wind noise decision coefficients are determined, namely, the correlation coefficient between every two sound signals in the multiple sound signals, the cumulative error energy value, and the cumulative energy difference value; then, the aforementioned three wind noise decision coefficients are used to determine whether wind noise exists in the input signal, that is, if one or more of the following conditions are met, it is determined that wind noise exists in the input signal: at least one of the correlation coefficients between every two sound signals in the multiple sound signals is less than a preset correlation threshold, at least one of the cumulative error energy values ​​is greater than a first preset difference value, and at least one of the cumulative energy difference values ​​is greater than a second preset difference value. Compared to existing technologies that use preliminary threshold judgment methods for voice activity detection, the disadvantage is that if the wind noise in the environment is too high, the wind noise signal can still easily exceed the threshold, leading to the wind noise signal being misjudged as a voice signal, resulting in insufficient detection accuracy, resource waste, and increased power consumption. The embodiments of the present invention calculate three wind noise judgment coefficients for multiple sound signals collected by multiple microphones from the same input signal, and based on the comparison results of the wind noise judgment coefficients with each preset threshold, it can accurately determine whether there is wind noise (mainly wind noise signal) in the input signal, reduce unnecessary voice activity detection caused by excessive environmental wind noise, improve detection accuracy, reduce bandwidth resource waste, and reduce power consumption.

[0034] Furthermore, the sound signal detection method further includes: determining the input signal as a speech signal when it is determined that there is no wind noise in the input signal and the extreme value of the amplitude of the input signal exceeds a preset threshold range. Compared with the existing technology that adopts a two-step method of preliminary threshold value judgment and speech detection, in scenarios with excessive wind noise, the wind noise signal is likely to exceed the threshold value and trigger the second stage, resulting in the second stage performing speech detection on the wind noise signal, thereby reducing detection accuracy and wasting resources; the embodiment of the present invention adopts a three-step method: before performing threshold value judgment, wind noise signal detection is performed on the input signal; only when it is determined that there is no wind noise in the input signal is threshold value judgment performed, and finally the input signal with no wind noise and the extreme value of the amplitude exceeding the preset threshold range is determined as the real speech signal, thereby effectively improving the accuracy of speech signal detection.

[0035] Further, determining the correlation coefficient between any two audio signals in the multiple audio signals includes: for each audio signal, performing frame segmentation processing using a second preset duration to obtain multiple frames of audio sub-signals corresponding to different acquisition time periods; calculating the initial correlation coefficient between the audio sub-signals corresponding to the same acquisition time period in each pair of audio signals; and determining the correlation coefficient between the two audio signals based on the obtained multiple initial correlation coefficients. The average or median value of the obtained multiple initial correlation coefficients can be used as the correlation coefficient between the two audio signals. In this embodiment of the invention, by performing frame segmentation processing on each audio signal before calculating the correlation coefficient, the computational efficiency and accuracy can be effectively improved.

[0036] Furthermore, the preset correlation threshold, the first preset difference, and the second preset difference are determined based on one or more of the following: wind noise intensity in the acquisition space, sound wave incident angle, microphone spatial position, and microphone frequency response. In this embodiment of the invention, the preset thresholds or preset differences compared with the three wind noise decision coefficients during the wind noise decision process are determined by comprehensively considering one or more factors such as the acquisition environment, acquisition equipment, and sound wave characteristics, thereby reducing errors and improving the accuracy of wind noise decision.

[0037] Furthermore, before determining the correlation coefficient, cumulative error energy, and cumulative energy difference between any two audio signals in the multi-channel audio signals, the method further includes downsampling each audio signal separately. Since the amount of data in the acquired original audio signals is large, directly calculating the decision coefficients based on the original audio signals would occupy a large amount of storage space and reduce computational efficiency. This embodiment of the invention first downsamples the acquired original audio signals, which can compress the data volume, thereby improving the efficiency of subsequent calculations (detection), reducing the storage space occupied by the data, and reducing computational overhead. Attached Figure Description

[0038] Figure 1 This is a flowchart of the first sound signal detection method in an embodiment of the present invention;

[0039] Figure 2 yes Figure 1 A flowchart of a specific implementation of step S12;

[0040] Figure 3 yes Figure 1 A flowchart of another specific implementation of step S12;

[0041] Figure 4 yes Figure 1 A flowchart of another specific implementation of step S12;

[0042] Figure 5 This is a flowchart of the second sound signal detection method in this embodiment of the invention;

[0043] Figure 6 This is a schematic diagram of the structure of a sound signal detection device according to an embodiment of the present invention. Detailed Implementation

[0044] As mentioned earlier, research on how to improve the accuracy of extracting speech signals from audio signal streams is of great value.

[0045] In existing technologies, speech activity detection typically involves two stages: the first stage is preliminary threshold decision, which involves setting a preliminary threshold value for the sound signal captured by the microphone. If the sound signal exceeds the threshold value, the second stage is triggered. The second stage is speech detection, which can be performed frame-by-frame on the sound signal. Once speech is detected, subsequent functions such as speech recognition are triggered. Other existing technologies also employ the method of directly increasing the threshold value of the first stage to detect speech signals from the sound signal.

[0046] The inventors of this invention have discovered that the shortcomings of the prior art are as follows: In the process of preliminary threshold decision-making in the first stage, if the wind noise in the environment is too high, even if there is no speech signal in the collected sound signal, the wind noise signal can still easily exceed the threshold value and trigger the second stage, which leads to the second stage performing speech detection on the wind noise signal, thereby reducing the detection accuracy, causing resource waste and increased power consumption; while the method of directly increasing the threshold value may result in the actual speech signal not being able to exceed the threshold value due to the threshold value being set too high, thus failing to guarantee the detection accuracy.

[0047] Therefore, there is an urgent need for a sound signal detection method that can accurately determine whether there is wind noise in the sound signal collected by the microphone over a period of time, reduce unnecessary voice activity detection caused by excessive environmental wind noise, improve detection accuracy, reduce bandwidth waste, and lower power consumption.

[0048] In this embodiment of the invention, firstly, multiple sound signals acquired by multiple microphones from the same input signal within a first preset time period are determined; then, three wind noise decision coefficients are determined, namely, the correlation coefficient between every two sound signals in the multiple sound signals, the cumulative error energy value, and the cumulative energy difference value; then, the aforementioned three wind noise decision coefficients are used to determine whether wind noise exists in the input signal, that is, if one or more of the following conditions are met, it is determined that wind noise exists in the input signal: at least one of the correlation coefficients between every two sound signals in the multiple sound signals is less than a preset correlation threshold, at least one of the cumulative error energy values ​​is greater than a first preset difference value, and at least one of the cumulative energy difference values ​​is greater than a second preset difference value. Compared to existing technologies that use preliminary threshold judgment methods for voice activity detection, the disadvantage is that if the wind noise in the environment is too high, the wind noise signal can still easily exceed the threshold, leading to the wind noise signal being misjudged as a voice signal, resulting in insufficient detection accuracy, resource waste, and increased power consumption. The embodiments of the present invention calculate three wind noise judgment coefficients for multiple sound signals collected by multiple microphones from the same input signal, and based on the comparison results of the wind noise judgment coefficients with each preset threshold, it can accurately determine whether there is wind noise in the input signal, reduce unnecessary voice activity detection caused by excessive environmental wind noise, improve detection accuracy, reduce bandwidth resource waste, and reduce power consumption.

[0049] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Reference Figure 1 , Figure 1 This is a flowchart of the first sound signal detection method in an embodiment of the present invention. The first sound signal detection method may include steps S11 to S13:

[0051] Step S11: Determine the multiple audio signals acquired by multiple microphones from the same input signal within a first preset time period;

[0052] Step S12: Determine the correlation coefficient, cumulative error energy value, and cumulative energy difference value between every two audio signals in the multi-channel audio signals;

[0053] Step S13: If one or more of the following conditions are met, it is determined that wind noise exists in the input signal: at least one of the correlation coefficients between any two audio signals in the multi-channel audio signals is less than a preset correlation threshold; at least one of the cumulative error energy values ​​between any two audio signals in the multi-channel audio signals is greater than a first preset difference; and at least one of the cumulative energy difference values ​​between any two audio signals in the multi-channel audio signals is greater than a second preset difference.

[0054] When collecting the sound signal, the spatial positions of each microphone are different.

[0055] It is understood that, in specific implementations, the method can be implemented using a software program that runs in a processor integrated within the chip or chip module; or, the method can be implemented using hardware or a combination of hardware and software.

[0056] In the specific implementation of step S11, the sound signal, broadly speaking, can be an analog signal used to describe various types of sound information generated by the vibration of objects in nature, and it changes continuously over time. Sound signals can be divided into speech signals, wind noise signals, or noise signals, etc.; speech signals, as a major type of sound signal, have become an important means of obtaining and disseminating information in human society. The formation process of speech is as follows: air is expelled from the lungs into the larynx, passes through the vocal cords into the vocal tract, and finally radiates sound waves from the mouth, forming speech.

[0057] In specific implementations, when acquiring the sound signal, the number of microphones is at least two, but three or more microphones can be used to acquire the sound signal depending on the needs of different application scenarios. In some non-limiting embodiments, the microphone can be two microphones in an existing conventional headphone chip, one as a reference microphone and the other as an error microphone; it can also be a microphone or microphone array in existing conventional smartphones, tablets, intelligent voice dialogue robots, intelligent translation devices, and other electronic devices with voice communication or voice detection (recognition) functions (generally speaking, in the non-low power mode of a smartphone chip, voice detection is achieved through a microphone array, while in the low power mode, there is often only one microphone).

[0058] It should be noted that each microphone collects the same input signal within the first preset time period to obtain one audio signal. Multiple microphones collect the same input signal within the first preset time period to obtain multiple audio signals. The number of microphones is the same as the number of audio signals collected.

[0059] The length of the first preset duration is directly related to the amount of data in each acquired audio signal. It is understood that the first preset duration should not be set too long, otherwise the acquired audio signals will contain too much audio information, resulting in a large data volume and potentially reducing detection efficiency. Conversely, the first preset duration should not be set too short, otherwise the acquired audio signals will contain too little audio information, resulting in a small data volume and potentially reducing detection accuracy. In some non-limiting embodiments, the first preset duration can be set to approximately 40 milliseconds.

[0060] In the specific implementation of step S12, the correlation coefficient can be used to represent the similarity between sound signals; the cumulative error energy value can be used to represent the energy difference between sound signals, and can be determined based on the signal difference between sound signals; the cumulative energy difference value can also be used to represent the energy difference between each sound signal, and can be determined based on the energy value of the sound signal.

[0061] Furthermore, before determining the correlation coefficient, cumulative error energy value, and cumulative energy difference value between any two audio signals in the multiple audio signals, the method further includes downsampling each audio signal separately. Since the amount of data in the acquired original audio signals is large, directly calculating the decision coefficient based on the original audio signals would occupy a large amount of storage space and reduce computational efficiency. In this embodiment of the invention, downsampling the acquired original audio signals first can compress the data volume, thereby improving the efficiency of subsequent calculations (detection), reducing the storage space occupied by the data, and reducing computational overhead.

[0062] In the specific implementation of step S13, it is understood that when the sound signal is collected, due to the different spatial positions of each microphone (for example, in a scenario where a person is wearing headphones and making a phone call, the two microphones are located at the left and right ear positions respectively), and because wind noise in the environment has a certain degree of randomness, when the input signal is mainly wind noise, the sound signals collected by each microphone will differ significantly, have low correlation, and exhibit large energy differences. Consequently, the accumulated error energy value and the accumulated energy difference value will be large. Conversely, when wind noise is weak or absent, the sound signals collected by each microphone are approximately similar within the effective bandwidth, exhibiting high correlation, and the accumulated error energy value and the accumulated energy difference value will be small. Therefore, by reasonably setting the thresholds for the above three decision coefficients, it is possible to accurately determine whether wind noise exists in the input signal (i.e., whether it is mainly or entirely wind noise), avoiding misjudging the speech signal as wind noise.

[0063] Furthermore, in some non-limiting embodiments, the preset correlation threshold, the first preset difference, and the second preset difference can be determined based on one or more of the following influencing factors: wind noise intensity in the acquisition space, sound wave incident angle, microphone spatial position, and microphone frequency response.

[0064] Among them, wind noise intensity is used to indicate the noise intensity in a specific space where the sound signal is collected. The noise can refer to other sounds besides speech (such as wind noise, footsteps, door closing sounds, and vehicle noise during operation); the sound wave incident angle can be the angle formed between the sound wave and the horizontal plane of the other medium when the sound wave enters from one medium; the spatial position of the microphone can refer to the absolute position of the microphone at the time of collection, or the relative position of the microphone to the terminal at the time of collection; the frequency response of the microphone, also known as the effective frequency range, reflects the main frequency range in which the microphone operates and measures the microphone's ability to uniformly reproduce signals in high, medium, and low frequency bands.

[0065] In practical implementation, in order to set various thresholds as reasonably as possible, reduce errors, and improve the accuracy of wind noise judgment, it should be noted that: when collecting the multiple sound signals, the greater the wind noise intensity in the collection space, and / or the greater the difference in the sound wave incident angle of each microphone, and / or the greater the spatial difference between each microphone room, and / or the greater the difference in the frequency response of each microphone, then: the greater the difference between the sound signals collected by each microphone, the smaller the correlation coefficient, and the greater the cumulative value of error energy and the cumulative value of energy difference; under such circumstances, the correlation threshold should be smaller, the first preset difference should be larger, and the second preset difference should be larger.

[0066] Furthermore, after determining that wind noise exists in the input signal, the input signal can be directly discarded; alternatively, the difference between the correlation coefficient and the preset correlation threshold, the difference between the cumulative error energy value and the first preset difference, and the difference between the cumulative energy difference value and the second preset difference can be comprehensively considered, and the overall average or weighted calculation can be compared with the preset reasonable error range to finally decide whether to accept or reject the input signal.

[0067] In this embodiment of the invention, compared with the prior art's preliminary threshold decision method for voice activity detection, the disadvantage is that if the wind noise in the environment is too high, the wind noise signal can still easily exceed the threshold value, leading to the wind noise signal being misjudged as a voice signal, resulting in insufficient detection accuracy, easy waste of resources and increased power consumption. This embodiment of the invention calculates three wind noise decision coefficients for multiple sound signals collected by multiple microphones from the same input signal, and based on the comparison results of the wind noise decision coefficients with each preset threshold or preset difference, it can accurately determine whether there is wind noise (mainly wind noise signal) in the input signal, reduce unnecessary voice activity detection caused by excessive environmental wind noise, improve detection accuracy, reduce bandwidth resource waste, and reduce power consumption.

[0068] Reference Figure 2 , Figure 2 yes Figure 1The flowchart of a specific implementation of step S12 shows that determining the correlation coefficient between every two audio signals in the multi-channel audio signals may include steps S21 to S23, and each step is described below.

[0069] In step S21, for each audio signal, a second preset duration is used for frame processing to obtain multiple frames of audio sub-signals corresponding to different acquisition time periods.

[0070] In this embodiment of the invention, by performing frame-by-frame processing on each audio signal using a second preset duration and then calculating the correlation coefficient, the calculation efficiency and accuracy can be effectively improved.

[0071] In step S22, the initial correlation coefficient between the sound sub-signals corresponding to the same acquisition time period in each pair of sound signals is calculated.

[0072] Furthermore, the initial correlation coefficient between the sound sub-signals corresponding to the same acquisition time period in each pair of sound signals is calculated using the following formula:

[0073]

[0074] Where: corrcoef is used to indicate the initial correlation coefficient between the sound sub-signals corresponding to the same acquisition time period in two sound signals, x 1_frame x2 _frame These are used to indicate two frames of audio sub-signals corresponding to the same acquisition time period in two audio signals, respectively. cov() is used to indicate the covariance calculation function, and var() is used to indicate the variance calculation function.

[0075] As a non-limiting embodiment, the duration of each of the two acquired audio signals is 40 milliseconds. Using 4 milliseconds as the second preset duration, each audio signal is divided into several frames of audio sub-signals after frame processing. (It should be noted that, in frame division, in addition to using 1-4 milliseconds as the first frame, 5-8 milliseconds as the second frame, 9-12 milliseconds as the third frame, etc., another method can be used: 1-4 milliseconds as the first frame, 2-5 milliseconds as the second frame, 3-6 milliseconds as the third frame, etc.) Assuming that the acquisition time periods of 20 frames of audio sub-signals correspond one-to-one and are the same (20 pairs of audio sub-signals in the two audio signals belong to the same acquisition time period), then in the above steps, the initial correlation coefficient between these 20 pairs of audio sub-signals is calculated.

[0076] In step S23, the correlation coefficient between the two audio signals is determined based on the multiple initial correlation coefficients obtained.

[0077] In some non-limiting embodiments, the average or median of the multiple initial correlation coefficients obtained can be used as the correlation coefficient between the two audio signals.

[0078] For more detailed information on steps S21 to S23 in the specific implementation, please refer to the preceding text and... Figure 1 The steps described in the document will be executed, and will not be repeated here.

[0079] Reference Figure 3 , Figure 3 yes Figure 1 A flowchart of another specific implementation of step S12. Determining the cumulative error energy value between every two audio signals in the multi-channel audio signals may include steps S31 to S32, which are described below.

[0080] In step S31, for every two audio signals in the multi-channel audio signals, the audio signal difference between the sampling points belonging to the same acquisition time in the two audio signals is determined.

[0081] Each audio signal is acquired using a preset sampling frequency. As can be understood, sampling is a step in converting a continuous analog signal into a digital signal. By acquiring a continuous input signal (an analog signal) using the preset sampling frequency, a time-discrete digital signal can be obtained. From an effectiveness standpoint, the higher the sampling frequency, the closer the resulting discrete signal is to the original analog signal. However, excessively high sampling frequencies place higher demands on the actual circuitry and require significant computation and storage; conversely, excessively low sampling frequencies can lead to information loss, and in severe cases, information distortion, rendering the signal unusable. As a non-limiting embodiment, each audio signal can use an 8kHz sampling rate as the preset sampling frequency to obtain a number of sampling points. Each sampling point represents the instantaneous value of the original continuous analog signal at a certain moment, which can be represented by a binary number.

[0082] In step S32, the error energy between sampling points belonging to the same acquisition time is determined based on the difference in the sound signals, and the sum of the error energy between multiple pairs of sampling points belonging to the same acquisition time in the two signals is calculated as the cumulative error energy value between the two sound signals.

[0083] Furthermore, the following formula is used to determine the error energy between sampling points belonging to the same acquisition time based on the difference in the sound signals, and to calculate the sum of the error energy between multiple pairs of sampling points belonging to the same acquisition time in the two signals:

[0084] Delta = x1(t) - x2(t);

[0085] Delta_RSSI=[x1(t)-x2(t)] 2 ;

[0086]

[0087] Where t indicates different acquisition times, x1(t) and x2(t) indicate the audio signals of the sampling points belonging to the same acquisition time in the two audio signals respectively; Delta indicates the audio signal difference between each pair of sampling points belonging to the same acquisition time in the two audio signals; Delta_RSSI indicates the error energy between each pair of sampling points belonging to the same acquisition time in the two audio signals; S_RSSI indicates the sum of the error energy between n pairs of sampling points belonging to the same acquisition time in the two audio signals, where n indicates the number of sampling points belonging to the same acquisition time in the two audio signals.

[0088] As a non-limiting embodiment, each of the two acquired audio signals has 300 sampling points. If the acquisition times of 200 of these sampling points correspond one-to-one and are the same (200 pairs of sampling points in the two audio signals belong to the same acquisition time), then in the above steps, for the two audio signals, the audio signal difference between these 200 pairs of sampling points is determined, and then the error energy between these 200 pairs of sampling points is determined respectively. The sum of the error energy between these 200 pairs of sampling points is calculated as the cumulative error energy value between the two audio signals.

[0089] Reference Figure 4 , Figure 4 yes Figure 1 A flowchart of another specific implementation of step S12. Determining the cumulative energy difference between every two audio signals in the multi-channel audio signals may include steps S41 to S42, which are described below.

[0090] In step S41, for every two audio signals in the multi-channel audio signals, the energy value of the sampling points belonging to the same acquisition time in the two audio signals is determined.

[0091] Each audio signal can be acquired using a preset sampling frequency.

[0092] In step S42, the energy difference between the sampling points of the two audio signals at the same acquisition time is calculated, and then the sum of the energy differences between multiple pairs of sampling points of the two signals at the same acquisition time is calculated as the cumulative energy difference between the two audio signals.

[0093] Furthermore, the energy difference between sampling points belonging to the same acquisition time in the two audio signals is calculated using the following formula, and then the sum of the energy differences between multiple pairs of sampling points belonging to the same acquisition time in the two signals is calculated:

[0094] RSSI_x1(t)=x1(t) 2 ;

[0095] RSSI_x2(t)=x2(t) 2 ;

[0096] RSSI_Delta(t)=RSSI_x1(t)-RSSI_x2(t);

[0097]

[0098] Where t indicates different acquisition times, x1(t) and x2(t) indicate the audio signals of the sampling points belonging to the same acquisition time in the two audio signals, respectively; RSSI_x1(t) and RSSI_x2(t) indicate the energy values ​​of each pair of sampling points belonging to the same acquisition time in the two audio signals, respectively; RSSI_Delta(t) indicates the difference in energy values ​​between each pair of sampling points belonging to the same acquisition time in the two audio signals; RSSI_S indicates the sum of the differences in energy values ​​between n pairs of sampling points belonging to the same acquisition time in the two audio signals, where n indicates the number of pairs of sampling points belonging to the same acquisition time in the two audio signals.

[0099] As a non-limiting embodiment, each of the two acquired audio signals has 200 sampling points. If the acquisition times of 150 of these sampling points correspond one-to-one and are the same (150 pairs of sampling points in the two audio signals belong to the same acquisition time), then in the above steps, for the two audio signals, the difference in energy values ​​between these 150 pairs of sampling points is determined, and the sum of the differences in energy values ​​between these 150 pairs of sampling points is calculated as the cumulative energy difference between the two audio signals.

[0100] Reference Figure 5 , Figure 5 This is a flowchart of the second sound signal detection method in an embodiment of the present invention. The second sound signal detection method may include steps S51 to S55, which are described below.

[0101] In step S51, multiple audio signals are determined by multiple microphones acquiring the same input signal within a first preset time period.

[0102] In step S52, the correlation coefficient, cumulative error energy value, and cumulative energy difference value between every two audio signals in the multi-channel audio signals are determined.

[0103] In practical implementation, determining the correlation coefficient, cumulative error energy value, and cumulative energy difference value between every two audio signals in the multi-channel audio signal set can be done by converting the continuous analog signal into a discrete digital signal. The analog-to-digital conversion process can be found in [reference needed]. Figure 3 The relevant explanations regarding step S31 will not be repeated here.

[0104] In step S53, if one or more of the following conditions are met, it is determined that wind noise exists in the input signal: at least one of the correlation coefficients between any two audio signals in the multi-channel audio signals is less than a preset correlation threshold; at least one of the cumulative error energy values ​​between any two audio signals in the multi-channel audio signals is greater than a first preset difference; and at least one of the cumulative energy difference values ​​between any two audio signals in the multi-channel audio signals is greater than a second preset difference.

[0105] In step S54, it is determined that there is no wind noise in the input signal and that the extreme values ​​of the amplitude of the input signal exceed a preset threshold range.

[0106] In specific implementation, when none of the three conditions in step S53 are met, it can be determined that there is no wind noise in the input signal (in specific implementation, when the wind noise is weak and the voice signal is dominant, the three conditions in step S53 above will not be met either. In this case, the situation of weak wind noise can be classified as the situation of no wind noise), and then the threshold value judgment is performed to detect the real voice signal.

[0107] In step S55, the input signal is determined to be a voice signal.

[0108] In this embodiment of the invention, compared with the existing technology that adopts a two-step method of preliminary threshold decision and voice detection, in scenarios with excessive wind noise, the wind noise signal is likely to exceed the threshold value and trigger the second stage, which leads to the second stage performing voice detection on the wind noise signal, thereby reducing the detection accuracy and wasting resources; the embodiment of the invention adopts a three-step method: before making a threshold decision, the input signal is first detected for wind noise signal. When it is determined that there is no wind noise in the input (or the wind noise is very weak), the threshold decision is then made. Finally, the input signal that has no wind noise and whose extreme value of amplitude exceeds the preset threshold range is determined as the real voice signal, thereby effectively improving the accuracy of voice signal detection.

[0109] Furthermore, after determining that the input signal is a voice signal, various functions in different application scenarios can be triggered. For example, in the application of intelligent voice robots, keywords can be extracted from the input signal and searched, and then the search results can be fed back to the user (such as playing the searched music, reading the searched news or web pages, etc.).

[0110] Reference Figure 6 , Figure 6 This is a schematic diagram of a sound signal detection device according to an embodiment of the present invention. The sound signal detection device may include:

[0111] The multi-channel audio signal determination module 61 is used to determine the multi-channel audio signals acquired by multiple microphones from the same input signal within a first preset time period;

[0112] The wind noise decision coefficient determination module 62 is used to determine the correlation coefficient, error energy accumulation value, and energy difference accumulation value between every two audio signals in the multi-channel audio signals, wherein the correlation coefficient is used to indicate the similarity between audio signals;

[0113] The wind noise detection module 63 is used to determine that wind noise exists in the input signal when one or more of the following conditions are met: at least one of the correlation coefficients between any two audio signals in the multi-channel audio signals is less than a preset correlation threshold; at least one of the cumulative error energy values ​​between any two audio signals in the multi-channel audio signals is greater than a first preset difference; and at least one of the cumulative energy difference values ​​between any two audio signals in the multi-channel audio signals is greater than a second preset difference.

[0114] In specific implementation, the aforementioned sound signal detection device may correspond to a chip with sound signal detection function; or to a chip module with sound signal detection function in a terminal; or to a terminal.

[0115] In practical implementation, please refer to the preceding text for the principle, specific implementation, and beneficial effects of this sound signal detection device. Figures 1 to 5 The description of the sound signal detection method shown is not repeated here.

[0116] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the steps of the aforementioned sound signal detection method. The computer-readable storage medium may include non-volatile or non-transitory memory, and may also include optical discs, hard disk drives, solid-state drives, etc.

[0117] Specifically, in this embodiment of the invention, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0118] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0119] This invention also provides a terminal, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it performs the steps of the aforementioned sound signal detection method. The terminal may include, but is not limited to, mobile phones, computers, tablets, and other terminal devices, and may also be servers, cloud platforms, etc.

[0120] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or in a combination of hardware and software functional units. For example, for various devices or products applied to or integrated into a chip, each module / unit can be implemented using hardware such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware such as circuits; for various devices or products applied to or integrated into a chip module, each module / unit can be implemented using hardware such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0123] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0124] In the embodiments of this application, "multiple" refers to two or more.

[0125] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0126] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0127] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method of detecting a sound signal, characterized by, The method comprises: determining a plurality of sound signals collected by a plurality of microphones from the same input signal within a first preset time length; determining a correlation coefficient, an error energy cumulative value and an energy difference cumulative value between each two sound signals in the plurality of sound signals, wherein the correlation coefficient is used to indicate the similarity between the sound signals; if one or more of the following conditions are met, it is determined that wind noise exists in the input signal: at least one of the correlation coefficients between each two sound signals in the plurality of sound signals is less than a preset correlation threshold, at least one of the error energy cumulative values between each two sound signals in the plurality of sound signals is greater than a first preset difference value, and at least one of the energy difference cumulative values between each two sound signals in the plurality of sound signals is greater than a second preset difference value; wherein the spatial positions of the microphones are different when the sound signals are collected; wherein the method further comprises: when it is determined that wind noise does not exist in the input signal and the extreme value of the amplitude of the input signal exceeds a preset threshold range, determining that the input signal is a speech signal.

2. The voice signal detection method according to claim 1, wherein After determining that the input signal is a speech signal, the method further comprises: extracting keywords from the input signal and performing retrieval.

3. The voice signal detection method of claim 1, wherein, Determining the correlation coefficient between each two sound signals in the plurality of sound signals comprises: for each sound signal, performing frame processing using a second preset time length to obtain a plurality of sound sub-signals corresponding to different collection time periods; calculating the initial correlation coefficient between the sound sub-signals corresponding to the same collection time period in each two sound signals, respectively; determining the correlation coefficient between the two sound signals according to the plurality of obtained initial correlation coefficients.

4. The voice signal detection method according to claim 3, wherein The initial correlation coefficient between the sound sub-signals corresponding to the same collection time period in each two sound signals is calculated using the following formula: ; wherein: x for indicating an initial correlation coefficient between sound sub-signals corresponding to the same acquisition time period in the two-way sound signal, 1_frame x2 _frame for indicating two frames of sound sub-signals corresponding to the same acquisition time period in the two-way sound signal, respectively, for indicating a covariance calculation function, for indicating a variance calculation function.

5. The method of claim 3, wherein determining the correlation coefficient between the two sound signals according to the plurality of obtained initial correlation coefficients comprises: taking the average or median value of the plurality of obtained initial correlation coefficients as the correlation coefficient between the two sound signals.

6. The voice signal detection method of claim 1, wherein Each sound signal is collected using a preset sampling frequency. For each two sound signals in the plurality of sound signals, the sound signal difference between the sampling points belonging to the same collection time is determined; the error energy between the sampling points belonging to the same collection time is determined according to the sound signal difference, and the sum of the error energies between a plurality of pairs of sampling points belonging to the same collection time in the two sound signals is calculated as the error energy cumulative value between the two sound signals.

7. The voice signal detection method according to claim 6, wherein The error energy between the sampling points belonging to the same collection time is determined according to the sound signal difference using the following formula: ; ; ; wherein t is used to indicate different collection time instants, , are respectively used to indicate the sound signals of the sampling points belonging to the same collection time instant in the two sound signals; is used to indicate the sound signal difference between each pair of sampling points belonging to the same collection time instant in the two sound signals; is used to indicate the error energy between each pair of sampling points belonging to the same collection time instant in the two sound signals; is used to indicate the sum of the error energies between n pairs of sampling points belonging to the same collection time instant in the two sound signals, wherein n is used to indicate the number of pairs of sampling points belonging to the same collection time instant in the two sound signals.

8. The voice signal detection method of claim 1, wherein, Each sound signal is collected using a preset sampling frequency. Determining the energy difference cumulative value between each two sound signals in the plurality of sound signals comprises: For each two of the multi-channel sound signals, determine energy values of sampling points belonging to the same collection time instant in the two sound signals; Calculate the difference between the energy values of the sampling points belonging to the same collection time instant in the two sound signals, and further calculate the sum of the differences between the energy values of multiple pairs of sampling points belonging to the same collection time instant in the two sound signals as the energy difference accumulation value between the two sound signals.

9. The method of claim 8, wherein, The difference between the energy values of the sampling points belonging to the same collection time instant in the two sound signals is calculated by using the following formula, and the sum of the differences between the energy values of multiple pairs of sampling points belonging to the same collection time instant in the two sound signals is calculated: ; ; ; ; wherein t is used to indicate different collection time instants, , are respectively used to indicate the sound signals of the sampling points belonging to the same collection time instant in the two sound signals; , are respectively used to indicate the energy values of each pair of sampling points belonging to the same collection time instant in the two sound signals; is used to indicate the difference between the energy values of each pair of sampling points belonging to the same collection time instant in the two sound signals; is used to indicate the sum of the differences between the energy values of n pairs of sampling points belonging to the same collection time instant in the two sound signals, n is used to indicate the number of pairs of sampling points belonging to the same collection time instant in the two sound signals.

10. The method of claim 1, wherein, The preset correlation threshold, the first preset difference value and the second preset difference value are determined according to one or more of the following: The wind noise intensity in the collection space, the sound wave incidence angle, the spatial position of the microphone, and the frequency response of the microphone.

11. The voice signal detection method of claim 1, wherein, Before determining the correlation coefficient, the error energy accumulation value and the energy difference accumulation value between each two of the multi-channel sound signals, the method further comprises: Respectively performing downsampling processing on each sound signal.

12. A sound signal detection apparatus characterized by comprising: Comprise: A multi-channel sound signal determination module is configured to determine multi-channel sound signals collected by multiple microphones from the same input signal within a first preset time length; A wind noise judgment coefficient determination module is configured to determine the correlation coefficient, the error energy accumulation value and the energy difference accumulation value between each two of the multi-channel sound signals, wherein the correlation coefficient is used to indicate the similarity between sound signals; A wind noise detection module is configured to determine that wind noise exists in the input signal when one or more of the following conditions are met: at least one of the correlation coefficients between each two of the multi-channel sound signals is less than a preset correlation threshold, at least one of the error energy accumulation values between each two of the multi-channel sound signals is greater than a first preset difference value, and at least one of the energy difference accumulation values between each two of the multi-channel sound signals is greater than a second preset difference value; Further comprise: A module for determining that the input signal is a speech signal when it is determined that no wind noise exists in the input signal and the extreme value of the amplitude of the input signal exceeds a preset threshold range.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is run by the processor to perform the steps of the sound signal detection method of any one of claims 1 to 11.

14. A terminal comprising a memory and a processor, said memory having stored thereon a computer program capable of running on said processor, characterized in that, The processor runs the computer program to perform the steps of the sound signal detection method of any one of claims 1 to 11. The processor runs the computer program to perform the steps of the sound signal detection method of any one of claims 1 to 11.

Images