Adaptive threshold voice activity detection method and system

Abstract

The present application relates to the technical field of speech recognition, and especially relates to a speech endpoint detection method and system with adaptive threshold, comprising: setting an initial short-time energy threshold and an initial zero-crossing rate threshold, setting an analysis space L for storing analyzed speech data frames and a silence frame counter N with an initial value of 0; storing a current speech data frame into the analysis space, analyzing the current speech data frame to obtain a data frame type according to the short-time energy threshold and the zero-crossing rate threshold, and processing the silence frame counter according to the data frame type; if the data frame type is a silence frame, updating the short-time energy threshold and the zero-crossing rate threshold according to the number of frames of the silence frame counter; and further realizing a threshold adaptive scene and improving the technical effect of detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a speech endpoint detection method and system with adaptive threshold. Background Technology

[0002] Advances in technology have made life more convenient, with voice replacing text-based communication in many situations. Machines can now be controlled via voice. However, significant noise levels pose a major challenge in speech recognition; the solution is speech endpoint analysis. Speech endpoint analysis involves accurately locating the start and end points of speech within noisy audio, removing silent and noisy segments to extract the truly valid content. This is a crucial technique in speech recognition, providing accurate start and end points for subsequent speech processing.

[0003] Currently, there are two main types of speech endpoint detection methods: one is the algorithm-based model-based method. This method requires multi-dimensional feature parameters and a large amount of training data, making it difficult and costly to implement. Furthermore, the recognition results are too tightly coupled with the training data, making it difficult to accurately detect speech endpoints outside the training data. The other type is the threshold-based method, which uses speech feature indicators. This involves selecting a judgment indicator, such as the energy entropy ratio, and comparing the current frame's or a transformed indicator value with a threshold to obtain the speech judgment result. A problem with this method is that the threshold cannot be updated according to environmental changes, making it difficult to accurately detect speech endpoints outside the expected environment. In summary, existing speech endpoint detection technologies suffer from unstable accuracy due to insufficient adaptability. Summary of the Invention

[0004] The purpose of this invention is to provide an adaptive threshold voice endpoint detection method to address the shortcomings of existing technologies, aiming to solve the problem of unstable accuracy caused by insufficient adaptability in existing voice endpoint detection technologies.

[0005] The present invention achieves the above objectives through the following technical solutions:

[0006] An adaptive threshold-based speech endpoint detection method includes:

[0007] Setup steps: Set the initial short-time energy threshold and the initial zero-crossing rate threshold, set a parsing space L for buffering parsed voice data frames and a silence frame counter N with an initial value of 0;

[0008] Parsing steps: Store the current speech data frame into the parsing space, parse the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and process the silence frame counter according to the data frame type;

[0009] Processing steps: If the data frame type is a silent frame, then proceed with the first processing branch and the second processing branch sequentially.

[0010] The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space.

[0011] The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space.

[0012] Furthermore, the setup steps include:

[0013] Type determination sub-step: If the voice data to be parsed is existing voice data, then proceed to the first threshold setting sub-step; if the voice data to be parsed is real-time voice data, then proceed to the second threshold setting sub-step.

[0014] The first step in setting the threshold is to set the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data.

[0015] The second threshold setting sub-step involves calculating the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames preceding the voice data, which are then used as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

[0016] Furthermore, the parsing steps include:

[0017] Calculation sub-step: Calculate the short-time energy value e and the zero-crossing rate value z of the current speech data frame f;

[0018] Processing sub-step: Determine the type of the current speech data frame based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z. The process is as follows:

[0019] ;

[0020] ;

[0021] Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

[0022] Furthermore, in the second processing branch of the processing steps, the process of adjusting the short-time energy threshold and the zero-crossing rate threshold includes:

[0023] Judgment sub-step: Determine whether the silence frame counter has reached the adjustment threshold. If so, start the adjustment step.

[0024] Adjustment sub-step: Calculate the new short-time energy threshold and the new zero-crossing rate threshold based on the speech data frames in the parsing space.

[0025] Furthermore, the process of calculating the new short-time energy threshold and the new zero-crossing rate threshold in the adjustment sub-step includes:

[0026] Sorting step: Calculate the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sort them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows:

[0027] Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list;

[0028] The list of zero-crossing rate values ​​is as follows:

[0029] Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values;

[0030] Constructing the difference column: The first difference column is obtained by constructing the difference column corresponding to the short-time energy value list. The first difference column is as follows:

[0031] Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list;

[0032] Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being:

[0033] Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values;

[0034] The steps for obtaining outlier differences are as follows: draw box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot; obtain the first list of outlier differences from the first box plot; and obtain the second list of outlier differences from the second box plot.

[0035] The final calculation steps are as follows: Create a recursive counter, set the initial value to the first predetermined value, use De as the target list and the first abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new short-time energy threshold. Use Dz as the target list and the second abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new zero-crossing rate threshold.

[0036] The longest stable sublist calculation method includes: obtaining the first sequential index k corresponding to the partition value from the target list; if k does not exist, then performing a quartile calculation on the target list; if k exists, and If true, calculate the average of the first to the kth items in the target list as the result and reset the recursive counter to a first predetermined value. If there exists k, and If the condition is not met, determine whether the value of the recursion counter is the first predetermined value. If it is, take the (k+1)th to the nth as the new target list, recursively execute the longest stable sublist calculation method on the new target list, and set the value of the recursion counter to the second predetermined value. Otherwise, calculate the average value of the (k+1)th to the nth of the target list as the result, and reset the recursion counter to the first predetermined value.

[0037] The quartile calculation method includes: calculating the average of the upper and lower quartiles of the box plot as the result, and setting the value of the recursive counter to a first predetermined value.

[0038] An adaptive threshold speech endpoint detection system, applying the aforementioned adaptive threshold speech endpoint detection method, includes:

[0039] Setting module: Sets the initial short-time energy threshold and the initial zero-crossing rate threshold, sets a parsing space L for buffering parsed speech data frames and a silence frame counter N with an initial value of 0;

[0040] Parsing module: Stores the current speech data frame into the parsing space, parses the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and processes the silence frame counter according to the data frame type;

[0041] Processing module: If the data frame type is a silent frame, then the first processing branch and the second processing branch are performed sequentially.

[0042] The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space.

[0043] The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space.

[0044] Furthermore, the setting module includes:

[0045] Type determination submodule: If the voice data to be parsed is existing voice data, then the first threshold setting submodule is performed; if the voice data to be parsed is real-time voice data, then the second threshold setting submodule is performed.

[0046] The first threshold setting submodule sets the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data;

[0047] The second threshold setting submodule calculates the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames before the voice data, and uses them as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

[0048] Furthermore, the parsing module includes:

[0049] Calculation submodule: Calculates the short-time energy value e and the zero-crossing rate value z of the current speech data frame f;

[0050] Processing submodule: Based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z, the type of the current speech data frame is determined. The process is as follows:

[0051] ;

[0052] ;

[0053] Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

[0054] Furthermore, in the second processing branch of the processing module, the process of adjusting the short-time energy threshold and the zero-crossing rate threshold includes:

[0055] Judgment Submodule: Determines whether the silence frame counter has reached the adjustment threshold; if so, the adjustment module begins.

[0056] Adjustment submodule: Calculates a new short-time energy threshold and a new zero-crossing rate threshold based on the speech data frames in the parsing space.

[0057] Furthermore, in the adjustment submodule, the process of calculating the new short-time energy threshold and the new zero-crossing rate threshold includes:

[0058] The sorting module calculates the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sorts them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows:

[0059] Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list;

[0060] The list of zero-crossing rate values ​​is as follows:

[0061] Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values;

[0062] The module constructs a difference column: It constructs the difference column corresponding to the short-time energy value list to obtain the first difference column, which is:

[0063] Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list;

[0064] Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being:

[0065] Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values;

[0066] The module for obtaining abnormal differences: draws box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot, obtains the first abnormal difference list from the first box plot, and obtains the second abnormal difference list from the second box plot;

[0067] The final calculation module for the grandson is as follows: a recursive counter is created, with the initial value set to the first predetermined value. De is used as the target list and the first abnormal difference is used as the segmentation value. The longest stable sublist calculation method is used to obtain a new short-time energy threshold. Dz is used as the target list and the second abnormal difference is used as the segmentation value. The longest stable sublist calculation method is used to obtain a new zero-crossing rate threshold.

[0068] The longest stable sublist calculation method includes: obtaining the first sequential index k corresponding to the partition value from the target list; if k does not exist, then performing a quartile calculation on the target list; if k exists, and If true, calculate the average of the first to the kth items in the target list as the result and reset the recursive counter to a first predetermined value. If there exists k, and If the condition is not met, determine whether the value of the recursion counter is the first predetermined value. If it is, take the (k+1)th to the nth as the new target list, recursively execute the longest stable sublist calculation method on the new target list, and set the value of the recursion counter to the second predetermined value. Otherwise, calculate the average value of the (k+1)th to the nth of the target list as the result, and reset the recursion counter to the first predetermined value.

[0069] The quartile calculation method includes: calculating the average of the upper and lower quartiles of the box plot as the result, and setting the value of the recursive counter to a first predetermined value.

[0070] The beneficial effects of this invention are as follows: First, depending on whether the speech to be parsed comes from a speech file or is speech data recorded in real time from a microphone, different thresholds are set according to different rules, and then the speech is parsed, thereby achieving a technical effect that adapts to multiple scenarios; then, the speech endpoint is determined based on the number of consecutive silence frames, thereby achieving a technical effect of removing a large number of silence frames from the speech data to facilitate speech recognition; when the number of consecutive silence frames reaches a predetermined number, it indicates that the detection threshold of the detection endpoint needs to be adjusted, and a new threshold index is set based on the short-time energy value and zero-crossing rate value of the consecutive silence frames, thereby achieving a technical effect of improving the detection accuracy by adapting the detection threshold according to the scenario. Attached Figure Description

[0071] Figure 1 This is a flowchart of an adaptive threshold speech endpoint detection method proposed in this invention.

[0072] Figure 2 This is a schematic diagram illustrating the principle of adaptive threshold endpoint detection for existing speech files and real-time speech in this invention. Detailed Implementation

[0073] The present invention will now be described in detail with reference to the accompanying drawings.

[0074] like Figure 1 The diagram shows a flowchart of an adaptive threshold speech endpoint detection method proposed in this invention, including: a setting step, a parsing step, and a processing step. Different thresholds are set according to different rules depending on whether the speech to be parsed comes from a speech file or is real-time speech data recorded from a microphone. The speech is then parsed, achieving a technical effect that adapts to various scenarios. Speech endpoints are then determined based on the number of consecutive silence frames, thus achieving the technical effect of removing a large number of silence frames from the speech data to facilitate speech recognition. When the number of consecutive silence frames reaches a predetermined number, it indicates that the detection threshold for the endpoint needs to be adjusted. A new threshold index is set based on the short-time energy value and zero-crossing rate value of the consecutive silence frames, thereby achieving the technical effect of improving detection accuracy by adapting the detection threshold to the scenario.

[0075] Setup steps: Set the initial short-time energy threshold and the initial zero-crossing rate threshold, set a parsing space L for buffering parsed voice data frames and a silence frame counter N with an initial value of 0;

[0076] Parsing steps: Store the current speech data frame into the parsing space, parse the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and process the silence frame counter according to the data frame type;

[0077] Processing steps: If the data frame type is a silent frame, then proceed with the first processing branch and the second processing branch sequentially.

[0078] The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space.

[0079] The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space.

[0080] like Figure 2The diagram illustrates the principle of this invention. This invention proposes different methods for continuous speech files and microphone speech data to solve the problem of speech data endpoint identification. It blurs the distinction between the start and end points of speech in conventional endpoint identification. Here, speech endpoint identification distinguishes between speech and non-speech, and uses a certain judgment logic to trim out the effective speech segment. In the distinction between speech and non-speech, traditional indicators are used: short-time energy threshold and zero-crossing rate threshold. These two indicators directly represent the characteristics of speech. The two indicators are combined to form the speech threshold judgment condition. For the initial setting of the speech threshold, the speech file has a specified initial value, while microphone speech has a short silence period before the start of effective speech. The background noise during this silence period can be captured, so the initial threshold is obtained from the ambient background noise. The speech threshold is updated adaptively: the calculated values ​​of the two indicators from the previous segment of non-speech data are used as the threshold for the next stage of speech judgment. This allows for better adaptation to the background noise of the current speech data, making the judgment more accurate.

[0081] In the specific implementation of this scheme, the endpoint threshold is set to 30. If the number of consecutive silent frames reaches 30, it means that there are 30 consecutive frames of data that are not speech, which meets the segmentation standard for speech endpoints, and this frame is determined to be a speech endpoint. If the number of frames in the parsing space L is equal to 30, it means that all the data in L is not speech and is not stored. Otherwise, the speech data in L is stored as a single file. After processing the data in L, L is cleared and the next judgment and analysis are performed. By setting an endpoint threshold, the accurate detection of speech endpoints is thus completed.

[0082] In one embodiment, the setup step includes:

[0083] Type determination sub-step: If the voice data to be parsed is existing voice data, then proceed to the first threshold setting sub-step; if the voice data to be parsed is real-time voice data, then proceed to the second threshold setting sub-step.

[0084] The first step in setting the threshold is to set the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data.

[0085] The second threshold setting sub-step involves calculating the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames preceding the voice data, which are then used as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

[0086] Specifically, for existing audio files, the context in which the speech occurs is known. Therefore, the short-time energy threshold and zero-crossing rate threshold are set to values ​​most suitable for the corresponding scenario to improve accuracy. For example, in a noisy environment like a border inspection hall, the initial thresholds are set higher than usual. For audio recorded in real-time from a microphone, there is a brief silence period before the speech begins. The average short-time energy value and average zero-crossing rate value during this silence period are set as the initial short-time energy threshold and zero-crossing rate threshold.

[0087] In one embodiment, the parsing step includes:

[0088] Calculation sub-step: Calculate the short-time energy value e and the zero-crossing rate value z of the current speech data frame f;

[0089] Processing sub-step: Determine the type of the current speech data frame based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z. The process is as follows:

[0090] ;

[0091] ;

[0092] Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

[0093] Specifically, the coefficients k and m are determined by experimental adjustment. That is, when analyzing real speech files, different coefficient values ​​are used to segment the speech at the endpoints, and the coefficient value with the most accurate segmentation is selected. If the speech data frame being parsed is a speech frame, it means that there are no continuous silence frames and it cannot be determined as a speech endpoint, so the silence frame counter is reset to zero.

[0094] In another embodiment, the process of adjusting the short-time energy threshold and the zero-crossing rate threshold in the second processing branch of the processing steps includes:

[0095] Judgment sub-step: Determine whether the silence frame counter has reached the adjustment threshold. If so, start the adjustment step.

[0096] Adjustment sub-step: Calculate the new short-time energy threshold and the new zero-crossing rate threshold based on the speech data frames in the parsing space.

[0097] Specifically, this solution has several implementation methods. One method is to adjust the threshold to 90, meaning that when there are 90 consecutive silent frames, the short-time energy threshold and zero-crossing rate threshold should be adjusted. The short-time energy value and zero-crossing rate value of these 90 frames are discarded, with the largest and smallest 20% of the data being discarded, and the middle 60% of the data values ​​are taken. The average of the two indicators is then calculated as the new short-time energy threshold and zero-crossing rate threshold. The largest and smallest data are not included in the indicator calculation to avoid interference from extreme environmental fluctuations, making the 60% of data that are actually calculated more representative of the background noise situation. This makes the adjusted threshold more accurate in the process of detecting speech endpoints.

[0098] In another embodiment of this scheme, the process of calculating the new short-time energy threshold and the new zero-crossing rate threshold in the adjustment sub-step includes:

[0099] Sorting step: Calculate the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sort them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows:

[0100] Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list;

[0101] The list of zero-crossing rate values ​​is as follows:

[0102] Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values;

[0103] Constructing the difference column: The first difference column is obtained by constructing the difference column corresponding to the short-time energy value list. The first difference column is as follows:

[0104] Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list;

[0105] Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being:

[0106] Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values;

[0107] The steps for obtaining outlier differences are as follows: draw box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot; obtain the first list of outlier differences from the first box plot; and obtain the second list of outlier differences from the second box plot.

[0108] The final calculation steps are as follows: Create a recursive counter, set the initial value to the first predetermined value, use De as the target list and the first abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new short-time energy threshold. Use Dz as the target list and the second abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new zero-crossing rate threshold.

[0109] The longest stable sublist calculation method includes: obtaining the first sequential index k corresponding to the partition value from the target list; if k does not exist, then performing a quartile calculation on the target list; if k exists, and If true, calculate the average of the first to the kth items in the target list as the result and reset the recursive counter to a first predetermined value. If there exists k, and If the condition is not met, determine whether the value of the recursion counter is the first predetermined value. If it is, take the (k+1)th to the nth as the new target list, recursively execute the longest stable sublist calculation method on the new target list, and set the value of the recursion counter to the second predetermined value. Otherwise, calculate the average value of the (k+1)th to the nth of the target list as the result, and reset the recursion counter to the first predetermined value.

[0110] The quartile calculation method includes: calculating the average of the upper and lower quartiles of the box plot as the result, and setting the value of the recursive counter to a first predetermined value.

[0111] Specifically, when a large number of silent frames appear consecutively, meaning a large number of speech data frames are identified as silent frames, it indicates that the detection threshold may not be suitable for the current speech file and needs adjustment. This solution calculates the short-time energy value and zero-crossing rate value of each speech data frame in this stage, sorts the calculation results, and then calculates a difference series. Box plots are used to detect the dispersion of each element in the difference series, thereby finding data with abnormal short-time energy values ​​and zero-crossing rates. Based on the position of the abnormal data in the sorted list, it is determined whether the amount of data before or after the abnormal data is larger. If the amount of data before the abnormal data is larger, the average value of the first part is calculated as the result. If the amount of data after the abnormal data is larger, the first data corresponding to the abnormal difference value in the second part is identified, and the amount of data before and after is again determined. The average value of the data that can form the longest sublist is calculated as the result, and the short-time energy value and zero-crossing rate value are updated to adapt to the speech data frame being detected. If there is no abnormal data, the average value of the upper and lower quartiles of the box plot is calculated as the new short-time energy threshold and zero-crossing rate threshold.

[0112] An adaptive threshold speech endpoint detection system, applying the aforementioned adaptive threshold speech endpoint detection method, includes:

[0113] Setting module: Sets the initial short-time energy threshold and the initial zero-crossing rate threshold, sets a parsing space L for buffering parsed speech data frames and a silence frame counter N with an initial value of 0;

[0114] Parsing module: Stores the current speech data frame into the parsing space, parses the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and processes the silence frame counter according to the data frame type;

[0115] Processing module: If the data frame type is a silent frame, then the first processing branch and the second processing branch are performed sequentially.

[0116] The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space.

[0117] The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space.

[0118] This invention proposes different methods for continuous speech files and microphone speech data to solve the problem of speech data endpoint identification. It blurs the distinction between the start and end points of speech in conventional endpoint identification. Here, speech endpoint identification actually distinguishes between speech and non-speech, and uses a certain judgment logic to trim out the valid speech segments. In the distinction between speech and non-speech, traditional indicators are used: speech energy threshold and speech short-time zero-crossing rate. These two indicators directly represent the characteristics of speech. The two indicators are combined to form the speech threshold judgment condition. For the initial setting of the indicator threshold, the speech file has a specified initial value, while microphone speech has a short silence period before the start of effective speech, which can capture the background noise of the current environment. Therefore, the initial threshold is obtained from the background noise. The indicator threshold is updated adaptively: the calculated values ​​of the two indicators from the previous segment of non-speech data are used as the threshold for the next stage of speech judgment. This allows for better adaptation to the background noise of the current speech data, making the judgment more accurate.

[0119] In the specific implementation of this scheme, the endpoint threshold is set to 30. If the number of consecutive silent frames reaches 30, it means that there are 30 consecutive frames of data that are not speech, which meets the segmentation standard for speech endpoints, and this frame is determined to be a speech endpoint. If the number of frames in the parsing space L is equal to 30, it means that all the data in L is not speech and is not stored. Otherwise, the speech data in L is stored as a single file. After processing the data in L, L is cleared and the next judgment and analysis are performed. By setting an endpoint threshold, the accurate detection of speech endpoints is achieved.

[0120] In one embodiment, the setting module includes:

[0121] Type determination submodule: If the voice data to be parsed is existing voice data, then the first threshold setting submodule is performed; if the voice data to be parsed is real-time voice data, then the second threshold setting submodule is performed.

[0122] The first threshold setting submodule sets the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data;

[0123] The second threshold setting submodule calculates the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames before the voice data, and uses them as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

[0124] Specifically, for existing audio files, the context in which the speech occurs is known. Therefore, the short-time energy threshold and zero-crossing rate threshold are set to values ​​most suitable for the corresponding scenario to improve accuracy. For example, in a noisy environment like a border inspection hall, the initial thresholds are set higher than usual. For speech recorded in real-time from a microphone, there is a brief silence period before the speech begins. The average short-time energy value and average zero-crossing rate value during this silence period are set as the initial short-time energy threshold and zero-crossing rate threshold.

[0125] In specific implementation, the parsing module includes:

[0126] Calculation submodule: Calculates the short-time energy value e and the zero-crossing rate value z of the current speech data frame f;

[0127] Processing submodule: Based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z, the type of the current speech data frame is determined. The process is as follows:

[0128] ;

[0129] ;

[0130] Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

[0131] Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, and q represents a silence frame.

[0132] Specifically, the coefficients k and m are determined by experimental adjustment. That is, when analyzing real speech files, different coefficient values ​​are used to segment the speech at the endpoints, and the coefficient value with the most accurate segmentation is selected. If the speech data frame being parsed is a speech frame, it means that there are no continuous silence frames and it cannot be determined as a speech endpoint, so the silence frame counter is reset to zero.

[0133] In another embodiment, the process of adjusting the short-time energy threshold and the zero-crossing rate threshold in the second processing branch of the processing module includes:

[0134] Judgment Submodule: Determines whether the silence frame counter has reached the adjustment threshold; if so, the adjustment module begins.

[0135] Adjustment submodule: Calculates a new short-time energy threshold and a new zero-crossing rate threshold based on the speech data frames in the parsing space.

[0136] Specifically, in the implementation of this scheme, the threshold is adjusted to 90. That is, when there are 90 consecutive silent frames, the short-time energy threshold and zero-crossing rate threshold are adjusted. The short-time energy value and zero-crossing rate value of these 90 frames are discarded, and the middle 60% of the data are taken. The average of the two indicators is calculated as the new short-time energy threshold and zero-crossing rate threshold. The maximum and minimum data are not included in the indicator calculation to avoid interference from extreme environmental fluctuations, so that the 60% of data that are actually calculated can better represent the background noise situation. In this way, the adjusted threshold is more accurate in the process of detecting speech endpoints.

[0137] In one embodiment, the process of calculating the new short-time energy threshold and the new zero-crossing rate threshold in the adjustment submodule includes:

[0138] The sorting module calculates the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sorts them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows:

[0139] Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list;

[0140] The list of zero-crossing rate values ​​is as follows:

[0141] Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values;

[0142] The module constructs a difference column: It constructs the difference column corresponding to the short-time energy value list to obtain the first difference column, which is:

[0143] Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list;

[0144] Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being:

[0145] Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values;

[0146] The steps for obtaining outlier differences are as follows: draw box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot; obtain the first list of outlier differences from the first box plot; and obtain the second list of outlier differences from the second box plot.

[0147] The final calculation module: If the speech data frames in the parsing space are unstable, then

[0148] The first abnormal short-time energy value is obtained from the short-time energy value list based on the abnormal difference. Obtain the first abnormal zero-crossing rate value from the list of zero-crossing rate values. ,

[0149] like If true, then calculate. The average value of the short- and medium-time energy values ​​is used as the new short-time energy threshold; otherwise, the calculation is performed. The average value of the medium and short-time energy values ​​is used as the new short-time energy threshold.

[0150] like Established, calculated The average of the zero-crossing rate values ​​is used as the new zero-crossing rate threshold; otherwise, the following calculation is performed. The average of the zero-crossing rate values ​​is used as the new zero-crossing rate threshold.

[0151] If the speech data frames in the parsing space are stable, then

[0152] The average of the short-time energy values ​​of the upper and lower quartiles of the first box plot is used as a new short-time equalization threshold, and the average of the zero-crossing rate values ​​of the upper and lower quartiles of the second box plot is used as a new zero-crossing rate threshold.

[0153] Specifically, when a large number of silent frames appear consecutively, meaning a large number of speech data frames are identified as silent frames, it indicates that the detection threshold may not be suitable for the current speech file and needs to be adjusted. This solution calculates the short-time energy value and zero-crossing rate value of each speech data frame in this stage, sorts the results, and then calculates the difference column. Box plots are used to detect the dispersion of each element in the difference column, thereby finding data with abnormal short-time energy values ​​and zero-crossing rates. Based on the position of the abnormal data in the sorted list, it is determined whether the amount of data before or after the abnormal data is larger. A new detection threshold is calculated based on the larger portion, and the short-time energy value and zero-crossing rate value are updated to adapt to the speech data frame being detected.

[0154] In summary, the present invention possesses the excellent characteristics described above, which enhances its effectiveness in use compared to previous technologies, making it a highly practical product.

[0155] The above description is only a preferred embodiment of the present invention. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the present invention. The content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting speech endpoints with an adaptive threshold, characterized in that, include: Setup steps: Set the initial short-time energy threshold and the initial zero-crossing rate threshold, set a parsing space L for buffering parsed speech data frames and a silence frame counter N with an initial value of 0; Parsing steps: Store the current speech data frame into the parsing space, parse the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and process the silence frame counter according to the data frame type; Processing steps: If the data frame type is a silent frame, then proceed with the first processing branch and the second processing branch sequentially. The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space. The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space. The process of updating the short-time energy threshold and the zero-crossing rate threshold includes: Sorting step: Calculate the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sort them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows: Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list; The list of zero-crossing rate values ​​is as follows: Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values; Constructing the difference column: The first difference column is obtained by constructing the difference column corresponding to the short-time energy value list. The first difference column is as follows: Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list; Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being: Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values; The steps for obtaining outlier differences are as follows: draw box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot; obtain the first list of outlier differences from the first box plot; and obtain the second list of outlier differences from the second box plot. The final calculation steps are as follows: Create a recursive counter, set the initial value to the first predetermined value, use De as the target list and the first abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new short-time energy threshold. Use Dz as the target list and the second abnormal difference as the segmentation value, and use the longest stable sublist calculation method to obtain a new zero-crossing rate threshold. The longest stable sublist calculation method includes: obtaining the first sequential index k corresponding to the partition value from the target list; if k does not exist, then performing a quartile calculation on the target list; if k exists, and If true, calculate the average of the first to the kth items in the target list as the result and reset the recursive counter to a first predetermined value. If there exists k, and If the condition is not met, determine whether the value of the recursion counter is the first predetermined value. If it is, take the (k+1)th to the nth as the new target list, recursively execute the longest stable sublist calculation method on the new target list, and set the value of the recursion counter to the second predetermined value. Otherwise, calculate the average value of the (k+1)th to the nth of the target list as the result, and reset the recursion counter to the first predetermined value. The quartile calculation method includes: calculating the average of the upper and lower quartiles of the box plot as the result, and setting the value of the recursive counter to a first predetermined value.

2. The adaptive threshold speech endpoint detection method according to claim 1, characterized in that, The setup steps include: Type determination sub-step: If the voice data to be parsed is existing voice data, then proceed to the first threshold setting sub-step; if the voice data to be parsed is real-time voice data, then proceed to the second threshold setting sub-step. The first step in setting the threshold is to set the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data. The second threshold setting sub-step involves calculating the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames preceding the voice data, which are then used as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

3. The adaptive threshold speech endpoint detection method according to claim 1, characterized in that, The parsing steps include: Calculation sub-step: Calculate the short-time energy value e and the zero-crossing rate value z of the current speech data frame f; Processing sub-step: Determine the type of the current speech data frame based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z. The process is as follows: ； ； Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

4. An adaptive threshold speech endpoint detection system, employing the adaptive threshold speech endpoint detection method according to any one of claims 1-3, characterized in that, include: Setting module: Sets the initial short-time energy threshold and the initial zero-crossing rate threshold, sets a parsing space L for buffering parsed voice data frames and a silence frame counter N with an initial value of 0; Parsing module: Stores the current speech data frame into the parsing space, parses the current speech data frame according to the short-time energy threshold and the zero-crossing rate threshold to obtain the data frame type, and processes the silence frame counter according to the data frame type; Processing module: If the data frame type is a silent frame, then the first processing branch and the second processing branch are performed sequentially. The first processing branch includes: determining whether the silence frame counter has reached a predetermined endpoint threshold; if so, determining the current speech data frame as a speech endpoint and proceeding to the first processing branch sub-branch; the first processing branch sub-branch includes: determining whether the number of parsed speech data frames in the parsing space is equal to the endpoint threshold; if so, clearing the parsing space; otherwise, saving the parsed speech data frames in the parsing space as independent speech files, and then clearing the parsing space. The second processing branch includes: determining whether the silence frame counter has reached a predetermined adjustment threshold; if so, updating the short-time energy threshold and the zero-crossing rate threshold based on the parsed speech data frames in the parsing space.

5. The adaptive threshold speech endpoint detection system according to claim 4, characterized in that, The settings module includes: Type determination submodule: If the voice data to be parsed is existing voice data, then the first threshold setting submodule is performed; if the voice data to be parsed is real-time voice data, then the second threshold setting submodule is performed. The first threshold setting submodule sets the initial short-time energy threshold and zero-crossing rate threshold according to the scenario of the voice data; The second threshold setting submodule calculates the average short-time energy value and the average zero-crossing rate value of a predetermined number of voice data frames before the voice data, and uses them as the initial short-time energy threshold and zero-crossing rate threshold, respectively.

6. The adaptive threshold speech endpoint detection system according to claim 4, characterized in that, The parsing module includes: Calculation submodule: Calculates the short-time energy value e and the zero-crossing rate value z of the current speech data frame f; Processing submodule: Based on the relationship between the short-time energy value e and the short-time energy threshold E, and the relationship between the zero-crossing rate value z and the zero-crossing rate threshold Z, the type of the current speech data frame is determined. The process is as follows: ； ； Where k and m are coefficients, f.class represents the type of the current speech data frame, v represents a speech frame, q represents a silence frame, and N represents a silence frame counter.

7. The adaptive threshold speech endpoint detection system according to claim 4, characterized in that, In the second processing branch of the processing module, the process of adjusting the short-time energy threshold and the zero-crossing rate threshold includes: Judgment Submodule: Determines whether the silence frame counter has reached the adjustment threshold; if so, the adjustment module begins. Adjustment submodule: Calculates a new short-time energy threshold and a new zero-crossing rate threshold based on the speech data frames in the parsing space.

8. The adaptive threshold speech endpoint detection system according to claim 7, characterized in that, The process of calculating the new short-time energy threshold and the new zero-crossing rate threshold in the adjustment submodule includes: The sorting module calculates the short-time energy value and zero-crossing rate value of each speech data frame in the parsing space, and sorts them according to the calculation results to obtain a list of short-time energy values ​​and a list of zero-crossing rate values. The short-time energy value list is as follows: Where Le is a list of short-term energy values. It is the short-time energy value of the nth speech data frame in the short-time energy value list; The list of zero-crossing rate values ​​is as follows: Where Lz is a list of zero-crossing rate values, It is the zero-crossing rate value of the nth voice data frame in the list of zero-crossing rate values; The module constructs a difference column: It constructs the difference column corresponding to the short-time energy value list to obtain the first difference column, which is: Where De is the first difference sequence, It is the difference between the short-time energy values ​​of the nth and (n-1)th speech data frames in the short-time energy value list; Construct a second difference column corresponding to the list of zero-crossing rate values, the second difference column being: Where Dz is the second difference sequence, It is the difference between the zero-crossing rate values ​​of the nth and (n-1)th voice data frames in the list of zero-crossing rate values; The module for obtaining abnormal differences: draws box plots based on the first difference column and the second difference column to obtain the first box plot and the second box plot, obtains the first abnormal difference list from the first box plot, and obtains the second abnormal difference list from the second box plot; The final calculation module for the grandson is as follows: a recursive counter is created, with the initial value set to the first predetermined value. De is used as the target list and the first abnormal difference is used as the segmentation value. The longest stable sublist calculation method is used to obtain a new short-time energy threshold. Dz is used as the target list and the second abnormal difference is used as the segmentation value. The longest stable sublist calculation method is used to obtain a new zero-crossing rate threshold. The longest stable sublist calculation method includes: obtaining the first sequential index k corresponding to the partition value from the target list; if k does not exist, then performing a quartile calculation on the target list; if k exists, and If true, calculate the average of the first to the kth items in the target list as the result and reset the recursive counter to a first predetermined value. If there exists k, and If the condition is not met, determine whether the value of the recursion counter is the first predetermined value. If it is, take the (k+1)th to the nth as the new target list, recursively execute the longest stable sublist calculation method on the new target list, and set the value of the recursion counter to the second predetermined value. Otherwise, calculate the average value of the (k+1)th to the nth of the target list as the result, and reset the recursion counter to the first predetermined value. The quartile calculation method includes: calculating the average of the upper and lower quartiles of the box plot as the result, and setting the value of the recursive counter to a first predetermined value.

Images