Voice segmented sentence detection method and device, equipment, storage medium and program product

Abstract

The invention discloses a voice sentence segmentation detection method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring first voice basic data of a first voice and second voice basic data of a second voice; determining a second mute duration threshold value according to the first voice basic data and the second voice basic data; and determining a first sentence segmentation detection result of the second voice according to the mute duration of the second voice and the second mute duration threshold, wherein the first sentence segmentation detection result is used for indicating whether the second voice needs sentence segmentation or not. A mute duration threshold is dynamically adjusted according to second voice basic data of voice needing sentence segmentation detection and first voice basic data of voice already subjected to sentence segmentation detection, then voice sentence segmentation detection is performed according to the second voice basic data and the adjusted mute duration threshold, and a voice sentence segmentation detection result is obtained. And the accuracy of voice sentence segmentation detection can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of sentence segmentation detection, and in particular to a method, device, equipment, storage medium and program product for speech sentence segmentation detection. Background Art

[0002] In the process of recognizing user speech, the traditional segmentation algorithm determines the segmentation of speech by setting a fixed silence duration threshold. When the pause duration of speech is greater than the silence duration threshold, it is judged as a segmentation, and when it is less than the silence duration threshold, it is not segmented. Studies have found that different users have different speaking speeds, which are related to the user's environment, emotional state, and the content described. The fixed silence duration threshold does not take into account the speaker's speaking speed and speaking duration, which will result in the fixed silence duration threshold being inapplicable in certain situations. For example, in some cases where the duration is less than the silence duration threshold, the sentence should be segmented but is not, which may cause the accuracy of speech segmentation detection to decrease. Summary of the invention

[0003] The embodiments of the present application provide a speech segmentation detection method, apparatus, device, storage medium and program product, which dynamically adjusts the silence duration threshold according to the speech rate data and duration data of the speech to be segmented and the speech rate data and duration data of the speech that has been segmented, as well as the first silence duration threshold, to obtain a second silence duration threshold, and then obtains the speech segmentation detection result according to the silence duration of the speech to be segmented and the second silence duration threshold, which is beneficial to improving the accuracy of speech segmentation detection.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting speech segmentation, comprising:

[0005] Acquire first voice basic data of a first voice and second voice basic data of a second voice, wherein the first voice basic data includes first voice duration data, first voice speech rate data, and a first silence duration threshold, and the second voice basic data includes second voice duration data and second voice speech rate data;

[0006] Determine a second silence duration threshold according to the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data, and the second voice speed data;

[0007] A first sentence segmentation detection result of the second speech is determined according to the silence duration of the second speech and the second silence duration threshold, wherein the first sentence segmentation detection result is used to indicate whether the second speech needs to be segmented.

[0008] In a second aspect, an embodiment of the present application provides a speech segmentation detection device, comprising:

[0009] An acquisition module, used to acquire first voice basic data of a first voice and second voice basic data of a second voice;

[0010] A processing module is used to determine a second silence duration threshold based on the first voice basic data and the second voice basic data; and to determine a first sentence segmentation detection result of the second voice based on the silence duration of the second voice and the second silence duration threshold, wherein the first sentence segmentation detection result is used to indicate whether the second voice requires a sentence segmentation generation module.

[0011] In a third aspect, an embodiment of the present application provides a speech segmentation detection device, the device comprising:

[0012] A memory, a processor, and a speech segmentation detection program stored in the memory and executable on the processor, wherein the speech segmentation detection program is configured to implement part or all of the steps described in any method in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a speech segmentation detection program is stored. When the speech segmentation detection program is executed by a processor, some or all of the steps described in any method in the first aspect are implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product can be a software installation package.

[0015] By implementing the embodiment of the present application, the server of the speech segmentation detection system first obtains the first speech basic data of the first speech and the second speech basic data of the second speech, the first speech basic data includes the first speech duration data, the first speech speed data and the first silence duration threshold, the second speech basic data includes the second speech duration data and the second speech speed data, and then determines the second silence duration threshold according to the first speech duration data, the first speech speed data, the first silence duration threshold, the second speech duration data and the second speech speed data, and finally determines the first segmentation detection result of the second speech according to the silence duration and the second silence duration threshold of the second speech, and the first segmentation detection result is used to indicate whether the second speech needs to be segmented. By dynamically adjusting the silence duration threshold according to the speech speed data, duration data of the speech to be segmented and the speech speed data, duration data and the first silence duration threshold of the speech that has been segmented, the second silence duration threshold is obtained, and then the speech segmentation detection result is obtained according to the silence duration and the second silence duration threshold of the speech to be segmented, which is conducive to improving the accuracy of speech segmentation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0017] Figure 1 It is a schematic diagram of the architecture of a speech segmentation detection system provided in an embodiment of the present application;

[0018] Figure 2 It is a flow chart of a method for detecting speech segmentation provided by an embodiment of the present application;

[0019] Figure 3 is a flow chart of another speech segmentation detection method provided in an embodiment of the present application;

[0020] Figure 4 It is a structural schematic diagram of a speech segmentation detection device provided in an embodiment of the present application;

[0021] Figure 5 It is a structural schematic diagram of a speech segmentation detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0023] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0024] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] In the process of recognizing user speech, the traditional segmentation algorithm determines the segmentation of speech by setting a fixed silence duration threshold. When the pause duration of speech is greater than the silence duration threshold, it is judged as a segmentation, and when it is less than the silence duration threshold, it is not segmented. Studies have found that different users have different speaking speeds, which are related to the user's environment, emotional state, and the content described. The fixed silence duration threshold does not take into account the speaker's speaking speed and speaking duration, which will result in the fixed silence duration threshold being inapplicable in certain situations. For example, in some cases where the duration is less than the silence duration threshold, the sentence should be segmented but is not, which may cause the accuracy of speech segmentation detection to decrease.

[0026] There are several main methods for speech segmentation detection:

[0027] Silence duration counting: During the voice input process, the length of continuous silence is counted and compared with the preset fixed threshold. When the continuous silence exceeds the preset fixed threshold, it is judged as a voice interruption.

[0028] Energy or power-based detection: Determine whether to perform speech segmentation by analyzing the energy or power changes of the speech signal. When the energy or power drops below a preset fixed threshold for a period of time, it is considered that speech segmentation is necessary.

[0029] SNR detection: The signal-to-noise ratio between ambient noise and speech signals is used to determine the segmentation. When the signal-to-noise ratio is lower than the preset signal-to-noise ratio threshold for a period of time, it is considered that speech segmentation is required.

[0030] It can be seen that the current methods for speech segmentation detection all need to compare the speech data to be detected with a preset threshold to obtain the speech segmentation detection result. However, the thresholds used in the current detection are all preset fixed thresholds, which cannot be adjusted according to actual application scenarios (such as the speaking speed and speaking time of different speakers), resulting in a decrease in the accuracy of speech segmentation detection.

[0031] In response to the above problems, the embodiments of the present application provide a speech segmentation detection method, apparatus, device, storage medium and program product. The server of the speech segmentation detection system first obtains first speech basic data of a first speech and second speech basic data of a second speech, the first speech basic data including first speech duration data, first speech speed data and first silence duration threshold, the second speech basic data including second speech duration data and second speech speed data, then determines the second silence duration threshold according to the first speech duration data, the first speech speed data, the first silence duration threshold, the second speech duration data and the second speech speed data, finally determines the first segmentation detection result of the second speech according to the silence duration and the second silence duration threshold of the second speech, the first segmentation detection result is used to indicate whether the second speech needs to be segmented. By dynamically adjusting the silence duration threshold according to the speech rate data and duration data of the speech to be segmented and the speech rate data and duration data of the speech that has been segmented, as well as the first silence duration threshold, a second silence duration threshold is obtained, and then a speech segmentation detection result is obtained according to the silence duration of the speech to be segmented and the second silence duration threshold, which is beneficial to improving the accuracy of speech segmentation detection.

[0032] The speech segmentation detection method, apparatus, device, storage medium and program product provided in the embodiments of the present application can be applied to Figure 1 In the speech segmentation detection system shown in FIG. Figure 1 , Figure 1 It is a schematic diagram of the architecture of a speech segmentation detection system provided by an embodiment of the present application. The speech segmentation detection system 100 includes a terminal 101 and a server 102. The terminal 101 can communicate with the server 102 through a network. The terminal 101 refers to a device used by a user, such as a smart phone, a computer, etc. In this solution, the terminal 101 is mainly responsible for collecting an audio data set to be detected by speech segmentation and sending it to the server 102 for processing. The user can interact with the system through the terminal 101, set the initial silence duration threshold, upload voice, select the text that needs to be detected by speech segmentation, etc.; the server 102 refers to a remote computer used to process a large number of computing tasks and store data. In this solution, the server 102 receives the audio data set from the terminal, and calculates the second silence duration threshold in real time according to the basic voice data in the audio data set, and then performs speech segmentation detection on the voice to be detected in the audio data set, obtains the speech segmentation detection result, and returns the speech segmentation detection result to the terminal 101.

[0033] Based on this, the present application provides a speech segmentation detection method, device, equipment, storage medium and program product, and the present application is described in detail below in conjunction with the accompanying drawings.

[0034] See also Figure 2 , Figure 2is a flow chart of a speech segmentation detection method provided by an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0035] S201, obtaining first basic voice data of a first voice and second basic voice data of a second voice.

[0036] The first basic voice data includes first voice duration data, first voice speech rate data and first silence duration threshold, and the second basic voice data includes second voice duration data and second voice speech rate data.

[0037] Among them, the first voice duration data and the second voice duration data both represent the duration of the voice, the unit of the first voice duration data and the second voice duration data is second (s), the first voice speed data and the second voice speed data both represent the speed of the voice, the unit of the first voice speed data and the second voice speed data is words per second (words / s), for example, the first voice duration data is 2s, the second voice duration data is 4s, the first voice speed data is 2.5 words / s, and the second voice speed data is 3.25 words / s.

[0038] Among them, the first voice basic data of the first voice can be obtained from the previous voice of the second voice. For example, the voice duration of the previous voice of the second voice is 3s, and the voice speed is 3 words / s. At this time, it can be known that the first voice duration data is 3s, and the first voice speed data is 3 words / s. Alternatively, the first voice basic data of the first voice can be obtained from the voice basic data of multiple voices before the second voice. For example, there are 10 voices that have been segmented before the second voice, then the voice basic data of these 10 voices are obtained respectively, including voice duration data, voice speed data and silence duration threshold, and then the first voice basic data is obtained by calculating the average of all data.

[0039] Among them, when performing speech segmentation detection at the beginning, the silence duration threshold can be set by the user on the terminal 101. For example, the user directly sets the silence duration threshold to 200ms on the terminal 101. Alternatively, the silence duration threshold can be based on a large amount of experimental data and take the average of the pause durations during speech segmentation as the silence duration threshold for the first sentence. The remaining adaptive silence duration thresholds are calculated based on the initial silence duration threshold.

[0040] S202: Determine a second silence duration threshold according to the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data, and the second voice speed data.

[0041] The second silence duration threshold may be determined by at least two of the first voice duration data, the first voice speed data, the second voice duration data and the second voice speed data, and the first silence duration threshold.

[0042] Among them, when only considering the influence of speech duration on speech segmentation detection, the second silence duration threshold can be determined by the first speech duration data, the second speech duration data and the first silence duration threshold. Specifically, the calculation formula of the second silence duration threshold is as follows:

[0043]

[0044] Among them, T1 is the second silence duration threshold, T0 is the first silence duration threshold, α is the weight factor, t1 is the second voice duration data, and t0 is the first voice duration data.

[0045] Among them, when only considering the influence of speech speed on speech segmentation detection, the second silence duration threshold can be determined by the first speech speed data, the second speech speed data and the first silence duration threshold. Specifically, the calculation formula of the second silence duration threshold is as follows:

[0046]

[0047] Among them, T1 is the second silence duration threshold, T0 is the first silence duration threshold, β is the weight factor, v1 is the second voice speed data, and v0 is the first voice speed data.

[0048] In a possible implementation, determining the second silence duration threshold according to the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data, and the second voice speed data includes:

[0049] Determine a first weight based on a first weight balancing factor, the second voice speed data and the first voice speed data; determine a second weight based on the first weight balancing factor, the second voice duration data and the first voice duration data; determine a target weight based on a second weight balancing factor, the first weight and the second weight; determine a second silence duration threshold based on the target weight and the first silence duration threshold.

[0050] The first weight balancing factor and the second weight balancing factor are both set by the user at the terminal 101 . The first weight balancing factor and the second weight balancing factor are constants, and their specific values ​​are not limited herein.

[0051] The second silence duration threshold can be determined by the first voice duration data, the first voice speed data, the second voice duration data, the second voice speed data and the first silence duration threshold.

[0052] Specifically, the first weight can be calculated by a first weight balancing factor, the second voice speed data and the first voice speed data through simple operations (combination of addition, subtraction, multiplication and division) or linear weighting, exponential weighting and other methods, and the second weight can be calculated by a first weight balancing factor, the second voice duration data and the first voice duration data through simple operations or linear weighting, exponential weighting and other methods. The target weight can be the sum of the second weight balancing factor, the first weight and the second weight, and the second silence duration threshold can be the product of the target weight and the first silence duration threshold.

[0053] In another example, the first weight is determined by a first weight balancing factor, the second speech rate data, and the first speech rate data; the second weight is determined by a second weight balancing factor, the second speech duration data, and the first speech duration data; and the target weight is determined based on a third weight balancing factor, the first weight, and the second weight.

[0054] Among them, the first weight balancing factor, the second weight balancing factor and the third weight balancing factor can be set by the user in the terminal, and the values ​​of the first weight balancing factor, the second weight balancing factor and the third weight balancing factor are not limited here.

[0055] It can be seen that in this example, by setting multiple balancing factors, the target weight can be determined more accurately by combining the duration data and speech rate data of the speech that has been segmented and the speech to be segmented. Then, the second silence duration threshold is obtained by combining the target weight with the first silence duration threshold, which is conducive to providing a more accurate silence duration threshold for subsequent speech segmentation detection.

[0056] In a possible implementation manner, determining the first weight according to the first weight balancing factor, the second speech speed data, and the first speech speed data includes:

[0057] Obtain a first difference between the first voice speed data and the second voice speed data; obtain a first quotient of the first difference and the first voice speed data; obtain a second product of the first quotient and the first weight balance factor, wherein the second product is a first weight.

[0058] Specifically, the calculation formula of the first weight is as follows:

[0059]

[0060] Among them, α is the first weight balancing factor, v0 is the first voice speed data, v1 is the second voice speed data, and γ1 is the first weight.

[0061] Among them, research has found that different users have different speaking speeds. Judging whether a sentence with a pause should be interrupted has a lot to do with the user's speaking speed and the duration of speaking the sentence. The faster the speaking speed and the longer the duration, the higher the possibility of interrupting the sentence at the pause.

[0062] It can be seen that in this example, by obtaining the first difference between the first speech speed data and the second speech speed data, and combining the first difference, the first speech speed data and the first weight balancing factor, a dynamically changing first weight can be obtained. This is conducive to automatically adjusting the weight according to the actual speech situation to adapt to different speech segmentation detection scenarios.

[0063] In a possible implementation manner, determining the second weight according to the first weight balancing factor, the second voice duration data, and the first voice duration data includes:

[0064] Obtain a second difference between the first voice duration data and the second voice duration data; obtain a second quotient of the second difference and the first voice duration data; obtain a third difference between a first constant and the first weight balancing factor; obtain a third product of the second quotient and the third difference, wherein the third product is a second weight.

[0065] The first constant may be set by a user in the terminal, and the first constant may be 1. The value of the first constant is not restricted herein.

[0066] Specifically, the calculation formula of the first weight is as follows:

[0067]

[0068] Among them, α is the first weight balancing factor, t0 is the first speech duration data, t1 is the second speech duration data, γ2 is the first weight, and C is the first constant.

[0069] Among them, research has found that different users have different speaking speeds. Judging whether a sentence with a pause should be interrupted has a lot to do with the user's speaking speed and the duration of speaking the sentence. The faster the speaking speed and the longer the duration, the higher the possibility of interrupting the sentence at the pause.

[0070] It can be seen that in this example, by obtaining the first difference between the first speech speed data and the second speech speed data, and combining the first difference, the first speech speed data and the first weight balancing factor, a dynamically changing first weight can be obtained. This is conducive to automatically adjusting the weight according to the actual speech situation to adapt to different speech segmentation detection scenarios.

[0071] S203: Determine a first sentence segmentation detection result of the second speech according to the silence duration of the second speech and the second silence duration threshold.

[0072] The first sentence segmentation detection result of the second speech is determined by judging the magnitude relationship between the silence duration of the second speech and the second silence duration threshold.

[0073] In a possible implementation, determining a first sentence segmentation detection result of the second speech according to the silence duration of the second speech and the second silence duration threshold includes:

[0074] Detecting that the silence duration of the second speech is greater than the second silence duration threshold, determining that the first sentence segmentation detection result of the second speech is that the second speech needs to be segmented;

[0075] It is detected that the silence duration of the second speech is not greater than the second silence duration threshold, and it is determined that the first sentence segmentation detection result of the second speech is that the second speech does not need to be segmented.

[0076] For example, the second silence duration threshold of a second speech "The weather is good today (pause) Let's go for a spring outing" is set to 1 second, and the actual detected silence duration is used to determine whether it is necessary to perform a sentence break, and the silence duration (2 seconds) is detected to be greater than the target threshold (1 second). According to the sentence break rule, the first sentence break detection result of the second speech is determined to be "needs to perform a sentence break", and the subsequent system will make a sentence break here, dividing it into two parts, "The weather is good today." (pause) "Let's go to the park".

[0077] It can be seen that in this example, sentence segmentation is performed by judging the silence duration and the size of the silence duration threshold. The silence duration threshold can be dynamically adjusted according to different scenarios or user needs, which is conducive to improving the accuracy of speech segmentation detection.

[0078] In a possible implementation manner, after determining that the first sentence segmentation detection result of the second speech is that the second speech is not a sentence segmentation, the method further includes:

[0079] A first speech frame set is obtained based on the target number of speech frames before the pause of the second speech; a first total power is obtained based on the sum of the power of each speech frame in the first speech frame set; a second speech frame set is obtained based on the target number of speech frames after the pause of the second speech; a second total power is obtained based on the sum of the power of each speech frame in the second speech frame set; a second sentence segmentation detection result is obtained based on the first total power, the second total power and a power difference threshold, and the second sentence segmentation detection result is used to indicate whether the second speech needs to be subjected to a second sentence segmentation detection.

[0080] Specifically, the following method may be used to determine the pause of the second speech:

[0081] Energy threshold detection: The speech signal is divided into frames and the energy of each frame is calculated. If the energy of a continuous frame is lower than the set threshold, it can be considered as a potential pause point.

[0082] Short-time zero-crossing rate detection: The zero-crossing rate indicates the number of times the signal crosses the 0 point from the positive direction or the negative direction. If the zero-crossing rate between consecutive frames is high and lasts for a long time, there may be a pause.

[0083] Short-term average amplitude difference (SAD) detection: Determine whether there is a pause by calculating the average value of the amplitude difference between adjacent speech frames. If the amplitude difference exceeds a certain threshold and lasts for a long time, there may be a pause.

[0084] Voice activity detection based on acoustic models: Use machine learning and deep learning technologies to build acoustic models, classify and distinguish different voice segments, and then identify voice activities and non-voice activities (silence, noise, etc.), and determine whether there are pauses based on the acoustic model.

[0085] Among them, the target number can be set by the user at the terminal, and there is no limit on the target number here. Among them, the target number of frames is not greater than the smallest number of speech frames in the two sentences before and after the pause of the second voice. For example, the speech frames of the first voice before the pause of the second voice are 20 frames, and the speech frames of the second voice after the pause of the second voice are 14 frames, then the target number is not greater than 14 frames.

[0086] Among them, when obtaining the first total power and the second total power for comparison, the first total power can be the sum of the powers of the target number of speech frames of the first speech, or the average value of the powers of the target number of speech frames of the first speech, and the second total power can be the sum of the powers of the target number of speech frames of the second speech, or the average value of the powers of the target number of speech frames of the second speech. The calculation method of the first total power and the second total power is not restricted here.

[0087] Specifically, a second sentence segmentation detection result is obtained based on the first total power, the second total power and the power difference threshold, including: obtaining the absolute value of the fourth difference between the first total power and the second total power; detecting that the absolute value of the fourth difference is not greater than the power difference threshold, and determining that the second sentence segmentation detection result is that the second speech does not need a second sentence segmentation detection; detecting that the absolute value of the fourth difference is greater than the power difference threshold, and determining that the second sentence segmentation detection result is that the second speech needs a second sentence segmentation detection.

[0088] The power difference threshold may be set by the user in the terminal and is not limited here.

[0089] In another example, a second sentence segmentation detection result is obtained based on the first total power, the second total power and the power difference threshold, including: obtaining the absolute value of the fourth difference between the first total power and the second total power; obtaining a first ratio of the absolute value of the fourth difference to the first total power, detecting that the first ratio is not greater than the ratio threshold, and determining that the second sentence segmentation detection result is that the second speech does not need a second sentence segmentation detection; detecting that the first ratio is greater than the ratio threshold, and determining that the second sentence segmentation detection result is that the second speech needs a second sentence segmentation detection.

[0090] The ratio threshold may be 5%, 10% or 15%, and the ratio threshold may be set by a user in the terminal, and is not limited here.

[0091] Exemplarily, the second speech contains multiple pauses, and it is necessary to determine whether a second sentence segmentation detection is required based on the energy difference between the speech frames before and after the pause. Set the target number to 5 frames and the power difference threshold to 10 units. At a specific pause, select 5 frames of the first speech before the pause and 5 frames of the second speech after the pause; obtain the first total power (before the pause): the energy of the 5 frames is respectively: [15, 12, 14, 13, 16], the first total power calculation result = (15 + 12 + 14 + 13 + 16) / 5 = 14, obtain the second total power (after the pause): the energy of the 5 frames is respectively: [20, 18, 22, 19, 21], the second total power calculation result = (20 + 18 + 22 + 19 + 21) / 5 = 20, calculate the energy difference: energy difference = second total power - first total power = 20-14 = 6. The energy difference is less than the power difference threshold, and it is determined that the second speech does not need a second sentence segmentation detection.

[0092] It can be seen that in this example, by analyzing the energy difference between the speech frames before and after the pause, it can be more accurately determined whether further sentence segmentation is required, thereby improving the accuracy of sentence segmentation.

[0093] In a possible implementation manner, after determining that the second sentence segmentation detection result indicates that the second speech needs to be subjected to a second sentence segmentation detection, the method further includes:

[0094] Perform speech recognition on the second speech to obtain speech-to-text data of the second speech; detect that the speech-to-text data is in the target text set, and determine that the second sentence segmentation detection result of the second speech is that the second speech needs to be segmented; detect that the speech-to-text data is not in the target text set, and determine that the second sentence segmentation detection result of the second speech is that the second speech does not need to be segmented.

[0095] Among them, after determining that the second sentence segmentation detection result is that the second speech needs to be subjected to a second sentence segmentation detection, the method for performing speech recognition on the second speech can be a traditional speech recognition including a hidden Markov model (HMM), dynamic time warping (DTW), or a feature-based method including Mel-frequency cepstral coefficients (MFCC) and linear predictive coding (LPC), or a deep learning method including a convolutional neural network (CNN) and a recurrent neural network (RNN).

[0096] The target text set is set by the user in the terminal, and the target text set may include common modal particles and words of summary nature. For example, the target text set may be [in short, in summary, therefore].

[0097] In another example, the method for obtaining the target text set is one of the following methods:

[0098] Corpus-based construction: Use existing text corpora (such as books, articles, conversation data, etc.) for analysis, extract common words and phrases, and thus generate a target text set suitable for a specific scenario or field.

[0099] Machine learning model training: Through machine learning algorithms, speech recognition is performed on a large number of speech segments, and training is performed based on the obtained text data to identify important keywords and phrases in specific fields or topics, and automatically generate target text sets.

[0100] It can be seen that in this example, determining whether speech segmentation is needed based on the text obtained by speech recognition and the user-defined target text set can avoid the arbitrary judgment of speech segmentation based solely on the length of silence and improve the accuracy of speech segmentation detection.

[0101] In one possible implementation, see Figure 3 , Figure 3 is a flow chart of another speech segmentation detection method provided by an embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps:

[0102] S301. Obtain the target voice.

[0103] When obtaining the target voice, obtain the voice basic data of the target voice, where the voice basic data includes voice duration data and voice speed data.

[0104] The voice duration data represents the duration of the voice, and the unit of the voice duration data is second (s). The voice speed data represents the speed of the voice, and the unit of the voice speed data is words per second (words / s). For example, the voice duration data is 4s and the voice speed data is 3.25 words / s.

[0105] S302. Determine the adaptive silence duration threshold c according to the target voice and the speech speed and voice duration of the previous sentence.

[0106] The calculation formula of the silence duration threshold c is as follows:

[0107]

[0108] where T0 is the silence duration threshold of the previous sentence, v is the speech speed of this sentence, v′ is the speech speed of the previous sentence, t is the voice duration of this sentence, t′ is the voice duration of the previous sentence, and α∈[0, 1] is a weight balance factor used to adjust the weight between the speech speed and the duration. α can be selected according to the experimental results in specific scenarios. For example, for a speaker with a忽快忽慢 speech speed, the value of α should not be selected too large to reduce the impact of uneven speech speed on the result.

[0109] For the judgment of the first sentence break, it is judged through a custom silence duration threshold. For example, this threshold can take the average pause duration at the speech break as the silence duration threshold of the first sentence according to the experimental data, that is, determine the break of the first sentence by judging whether the silence duration at the pause is greater than. The remaining subsequent adaptive silence duration thresholds are obtained according to the above calculation formula of the silence duration threshold c.

[0110] Exemplarily, the speech speed of the first sentence is obtained as 2.5 words / s, the voice duration is 2s, and the silence duration threshold is 200ms. The speech speed of the second sentence is 3.25 words / s, and the voice duration is 4s. If the parameter a is taken as 0.9, the silence duration threshold of the second sentence can be obtained as 120ms.

[0111] S303. Compare the silence duration t of the target voice with the adaptive silence duration threshold c.

[0112] If it is detected that the silence duration t of the target voice ≥ c, go to step S307; if it is detected that the silence duration t of the target voice < c, go to step S304.

[0113] Exemplarily, the silence duration threshold is 120 ms, the silence duration t of the target speech is 200 ms, and t>c, then step S307 is performed.

[0114] S304, comparing the energy a of the first number of speech frames of the target speech and the energy b of the first number of speech frames of the subsequent speech.

[0115] Among them, the energy a of the first number of speech frames before the target speech pause and the energy b of the first number of speech frames after the target speech pause are compared. When it is detected that the absolute value of the difference between the energy a and the energy b is greater than the power difference threshold, step S305 is performed. When it is detected that the absolute value of the difference between the energy a and the energy b is not greater than the power difference threshold, step S308 is performed.

[0116] Among them, the first number can be set by the user at the terminal, and the target number is not limited here. Among them, the number of frames of the first number is not greater than the smallest number of speech frames in the two sentences before and after the pause of the target speech. For example, the speech frames of the first speech before the pause of the target speech are 20 frames, and the speech frames of the second speech after the pause of the target speech are 14 frames, then the first number is not greater than 14 frames.

[0117] Among them, when obtaining the first total power and the second total power for comparison, the first total power can be the sum of the powers of the first number of speech frames of the first speech, or the average value of the powers of the first number of speech frames of the first speech, and the second total power can be the sum of the powers of the first number of speech frames of the second speech, or the average value of the powers of the first number of speech frames of the second speech. The calculation method of the first total power and the second total power is not restricted here.

[0118] Exemplarily, 20 frames of speech signals before and after the target speech pause are taken for energy calculation. It is assumed that after calculation, the energy a of the previous sentence is 120 and the energy b of the next sentence is 130.

[0119] S305, performing speech recognition on the target speech to obtain text data.

[0120] Among them, the speech recognition method can be traditional speech recognition including hidden Markov model (HMM), dynamic time warping (DTW), or feature-based methods including Mel-frequency cepstral coefficients (MFCC) and linear predictive coding (LPC), or deep learning methods including convolutional neural network (CNN) and recurrent neural network (RNN).

[0121] S306, determining whether the text data is special text.

[0122] If it is detected that the text data is a special text, then step S307 is performed; if it is detected that the text data is not a special text, then step S308 is performed.

[0123] Among them, the user customizes a special character set in the terminal. If the text data is in the special character set, the text data is considered to be special characters. For example, the text data is "in a word", and the special character set is [in a word, in summary, therefore]. At this time, it is detected that the text data is special characters, and step S307 is performed.

[0124] S307, segmenting the target speech.

[0125] S308: No sentence segmentation is performed, but the target speech is merged with the next speech to be detected, and the speech sentence segmentation detection is performed again.

[0126] It can be seen that in this example, the silence duration threshold is dynamically adjusted according to the speech rate data and duration data of the speech to be detected and the speech rate data and duration data of the previous sentence, as well as the silence duration threshold of the previous sentence, to obtain an adaptive silence duration threshold, and then the first speech segmentation detection result is obtained according to the silence duration of the speech to be detected and the adaptive silence duration threshold. When the first speech segmentation detection result is that no segmentation is performed, a second speech segmentation detection is continued according to the energy of the speech frame and the text recognition result, which is beneficial to improving the accuracy of speech segmentation detection.

[0127] See also Figure 4 , Figure 4 is a structural diagram of a speech segmentation detection device provided in an embodiment of the present application, such as Figure 4 As shown, the speech segmentation detection device 400 includes:

[0128] An acquisition module 401 is used to acquire first voice basic data of a first voice and second voice basic data of a second voice; a processing module 402 is used to determine a second silence duration threshold value based on the first voice basic data and the second voice basic data; and is used to determine a first sentence segmentation detection result of the second voice based on the silence duration of the second voice and the second silence duration threshold value, wherein the first sentence segmentation detection result is used to indicate whether the second voice requires a sentence segmentation generation module.

[0129] In a possible implementation, in terms of determining the second silence duration threshold based on the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data and the second voice speed data, the processing module 402 is specifically used to: determine the first weight based on the first weight balancing factor, the second voice speed data and the first voice speed data; determine the second weight based on the first weight balancing factor, the second voice duration data and the first voice duration data; determine the target weight based on the second weight balancing factor, the first weight and the second weight; determine the second silence duration threshold based on the target weight and the first silence duration threshold, the second silence duration threshold being the first product of the target weight and the first silence duration threshold.

[0130] In one possible implementation, in determining the first weight based on the first weight balancing factor, the second voice speed data and the first voice speed data, the processing module 402 is specifically used to: obtain a first difference between the first voice speed data and the second voice speed data; obtain a first quotient of the first difference and the first voice speed data; obtain a second product of the first quotient and the first weight balancing factor, the second product being the first weight.

[0131] In one possible implementation, in determining the second weight based on the first weight balancing factor, the second voice duration data and the first voice duration data, the processing module 402 is specifically used to: obtain a second difference between the first voice duration data and the second voice duration data; obtain a second quotient of the second difference and the first voice duration data; obtain a third difference between a first constant and the first weight balancing factor; obtain a third product of the second quotient and the third difference, wherein the third product is the second weight.

[0132] In one possible implementation, in terms of determining the first sentence segmentation detection result of the second speech based on the silence duration of the second speech and the second silence duration threshold, the processing module 402 is specifically used to: if it is detected that the silence duration of the second speech is greater than the second silence duration threshold, determine that the first sentence segmentation detection result of the second speech is that the second speech needs to be segmented; if it is detected that the silence duration of the second speech is not greater than the second silence duration threshold, determine that the first sentence segmentation detection result of the second speech is that the second speech does not need to be segmented.

[0133] In a possible implementation, after determining that the first sentence segmentation detection result of the second speech is that the second speech is not a sentence segmentation, the processing module 402 is further used to: obtain a first speech frame set based on a target number of speech frames before a pause of the second speech; obtain a first total power based on the sum of the powers of each speech frame in the first speech frame set; obtain a second speech frame set based on a target number of speech frames after the pause of the second speech; obtain a second total power based on the sum of the powers of each speech frame in the second speech frame set; obtain a second sentence segmentation detection result based on the first total power, the second total power and a power difference threshold, and the second sentence segmentation detection result is used to indicate whether the second speech needs to be subjected to a second sentence segmentation detection.

[0134] In one possible implementation, in terms of obtaining the second sentence segmentation detection result based on the first total power, the second total power and the power difference threshold, the processing module 402 is specifically used to: obtain the absolute value of the fourth difference between the first total power and the second total power; detect that the absolute value of the fourth difference is not greater than the power difference threshold, and determine that the second sentence segmentation detection result is that the second speech does not need a second sentence segmentation detection; detect that the absolute value of the fourth difference is greater than the power difference threshold, and determine that the second sentence segmentation detection result is that the second speech needs a second sentence segmentation detection.

[0135] In a possible implementation, after determining that the second sentence segmentation detection result is that the second speech needs to be subjected to a second sentence segmentation detection, the processing module is also used to: perform text recognition on the second speech to obtain speech and text data of the second speech; detect that the speech and text data are located in a target text set, and determine that the second sentence segmentation detection result of the second speech is that the second speech needs to be subjected to sentence segmentation; detect that the speech and text data are not located in the target text set, and determine that the second sentence segmentation detection result of the second speech is that the second speech does not need to be subjected to sentence segmentation.

[0136] It is worth noting that the specific functional implementation of the speech segmentation detection device 400 is shown in the above Figure 2The description of the speech segmentation detection method shown in the figure, such as the acquisition module 401 is used to implement the relevant content of executing S201, and the processing module 402 is used to implement the relevant content of executing S202-S203. Each unit or module in the speech segmentation detection device 400 can be respectively or completely merged into one or more other units or modules to constitute, or one (some) of the units or modules can also be split into multiple smaller units or modules in function to constitute, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the present invention. The above-mentioned units or modules are divided according to logical functions. In practical applications, the function of a unit (or module) is realized by multiple units (or modules), or the function of multiple units (or modules) is realized by one unit (or module).

[0137] According to the description of the above method embodiment and related device embodiment, please refer to Figure 5 , Figure 5 is a structural diagram of a speech segmentation detection device provided in an embodiment of the present application, Figure 5 The speech segmentation detection device 500 shown includes a processor 501, a memory 502, a communication interface 503 and a bus 504. The processor 501, the memory 502 and the communication interface 503 are connected to each other through the bus 504.

[0138] Optionally, the memory 502 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0139] The memory 502 can store executable program codes. When the executable program codes stored in the memory 502 are executed by the processor 501, the processor 501 and the communication interface 503 are used to execute Figure 2 The various steps of the speech segmentation detection method of the illustrated embodiment.

[0140] The processor 501 adopts a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to execute the speech segmentation detection method of the method embodiment of the present application.

[0141] The processor 501 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the speech segmentation detection method of the present application can be completed by the hardware integrated logic circuit or software instructions in the processor 501. Optionally, the processor 501 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor is a microprocessor or the processor is any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The optional software module is located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage media in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and combines its hardware to complete the functions required to be executed by the modules included in a speech segmentation detection device 400 in an embodiment of the present application, or executes the speech segmentation detection method in the method embodiment of the present application.

[0142] The communication interface 503 uses a transceiver or other related device such as, but not limited to, a transceiver.

[0143] The bus 504 may include a path for transmitting information between various components of the speech segmentation detection device 500 (eg, the memory 502 , the processor 501 , and the communication interface 503 ).

[0144] It should be noted that although Figure 5 The speech segmentation detection device 500 shown in the figure only shows a memory, a processor, and a communication interface. However, in the specific implementation process, those skilled in the art should understand that the speech segmentation detection device 500 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the speech segmentation detection device 500 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the speech segmentation detection device 500 may also only include the devices necessary for implementing the embodiments of the present application, and does not necessarily include Figure 5 All devices shown in .

[0145] An embodiment of the present application provides a computer-readable storage medium, in which a computer program for electronic data exchange is stored. The computer program includes execution instructions, and the execution instructions are used to execute part or all of the steps of any one of the speech segmentation detection methods recorded in the above-mentioned speech segmentation detection method embodiments. The above-mentioned computer includes an electronic terminal device.

[0146] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program can be operated to enable a computer to perform part or all of the steps of any speech segmentation detection method recorded in the above method embodiments. The computer program product can be a software installation package.

[0147] It should be noted that, for the sake of simplicity, the embodiments of any of the aforementioned speech segmentation detection methods are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0148] The above is a detailed introduction to the embodiments of the present application. In this article, specific examples are used to illustrate the principles and implementation methods of a speech segmentation detection method, device, equipment, storage medium and program product of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the idea of ​​a speech segmentation detection method, device, equipment, storage medium and program product of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0149] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0150] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1The memory may include: a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0151] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude multiple situations. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0152] A person skilled in the art can understand that all or part of the steps in the various methods of the method embodiments of any of the above-mentioned speech segmentation detection methods can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a disk or an optical disk, etc.

[0153] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in an embodiment of a speech segmentation detection method of the present application, such as the device and computer program product of the above flowchart, falls within the scope of the related products described in the present application.

[0154] Obviously, those skilled in the art can make various changes and modifications to the speech segmentation detection method, device, equipment, storage medium and program product provided by the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting speech segmentation, characterized in that: The method comprises: Acquire first voice basic data of a first voice and second voice basic data of a second voice, wherein the first voice basic data includes first voice duration data, first voice speech rate data, and a first silence duration threshold, and the second voice basic data includes second voice duration data and second voice speech rate data; Determine a second silence duration threshold according to the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data, and the second voice speed data; A first sentence segmentation detection result of the second speech is determined according to the silence duration of the second speech and the second silence duration threshold, wherein the first sentence segmentation detection result is used to indicate whether the second speech needs to be segmented.

2. The method according to claim 1, characterized in that The determining the second silence duration threshold according to the first voice duration data, the first voice speed data, the first silence duration threshold, the second voice duration data and the second voice speed data comprises: Determine a first weight according to a first weight balancing factor, the second speech speed data, and the first speech speed data; Determine a second weight according to the first weight balancing factor, the second voice duration data and the first voice duration data; determining a target weight according to a second weight balancing factor, the first weight, and the second weight; A second silence duration threshold is determined according to the target weight and the first silence duration threshold, where the second silence duration threshold is a first product of the target weight and the first silence duration threshold.

3. The method according to claim 2, characterized in that The determining the first weight according to the first weight balancing factor, the second speech speed data and the first speech speed data comprises: Obtaining a first difference between the first speech speed data and the second speech speed data; Obtaining a first quotient value of the first difference and the first speech rate data; A second product of the first quotient and the first weight balancing factor is obtained, where the second product is the first weight.

4. The method according to claim 2, characterized in that The determining the second weight according to the first weight balancing factor, the second voice duration data and the first voice duration data comprises: Obtaining a second difference between the first voice duration data and the second voice duration data; Obtaining a second quotient of the second difference and the first speech duration data; Obtaining a third difference between the first constant and the first weight balancing factor; A third product of the second quotient and the third difference is obtained, where the third product is a second weight.

5. The method according to claim 1, characterized in that The determining the first sentence segmentation detection result of the second speech according to the silence duration of the second speech and the second silence duration threshold includes: Detecting that the silence duration of the second speech is greater than the second silence duration threshold, determining that the first sentence segmentation detection result of the second speech is that the second speech needs to be segmented; It is detected that the silence duration of the second speech is not greater than the second silence duration threshold, and it is determined that the first sentence segmentation detection result of the second speech is that the second speech does not need to be segmented.

6. The method according to any one of claims 1 to 5, characterized in that: After determining that the first sentence segmentation detection result of the second speech is that the second speech is not a sentence segmentation, the method further includes: Obtaining a first speech frame set according to a target number of speech frames preceding the pause of the second speech; Obtaining a first total power according to the sum of the powers of each speech frame in the first speech frame set; Obtaining a second speech frame set according to a target number of speech frames following the pause of the second speech; Obtaining a second total power according to the sum of the powers of each speech frame in the second speech frame set; A second sentence segmentation detection result is obtained according to the first total power, the second total power and the power difference threshold, and the second sentence segmentation detection result is used to indicate whether the second speech needs to be subjected to a second sentence segmentation detection.

7. The method according to claim 6, characterized in that The obtaining a second sentence segmentation detection result according to the first total power, the second total power and the power difference threshold includes: Obtaining an absolute value of a fourth difference between the first total power and the second total power; Detecting that the absolute value of the fourth difference is not greater than the power difference threshold, determining that the second sentence segmentation detection result is that the second speech does not need a second sentence segmentation detection; It is detected that the absolute value of the fourth difference is greater than the power difference threshold, and it is determined that the second sentence segmentation detection result is that the second speech needs to be subjected to a second sentence segmentation detection.

8. The method according to claim 7, characterized in that After determining that the second sentence segmentation detection result indicates that the second speech needs to be subjected to a second sentence segmentation detection, the method further includes: Performing speech recognition on the second speech to obtain speech text data of the second speech; It is detected that the speech and text data are in the target text set, and it is determined that the second sentence segmentation detection result of the second speech is that the second speech needs to be segmented; It is detected that the voice and text data is not located in the target text set, and it is determined that the second sentence segmentation detection result of the second voice is that the second voice does not need to be segmented.

9. A speech segmentation detection device, characterized in that: The device comprises: An acquisition module, configured to acquire first voice basic data of a first voice and second voice basic data of a second voice, wherein the first voice basic data includes first voice duration data, first voice speech rate data, and a first silence duration threshold, and the second voice basic data includes second voice duration data and second voice speech rate data; A processing module, used to determine a second silence duration threshold value based on the first speech duration data, the first speech speed data, the first silence duration threshold value, the second speech duration data and the second speech speed data; and used to determine a first sentence segmentation detection result of the second speech based on the silence duration of the second speech and the second silence duration threshold value, wherein the first sentence segmentation detection result is used to indicate whether the second speech needs to be segmented.

10. A speech segmentation detection device, characterized in that: The device comprises: A memory, a processor, and an executable program code stored in the memory and executable on the processor, wherein the processor executes the steps of the speech segmentation detection method according to any one of claims 1 to 8 when executing the executable program code.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores executable program code, wherein the executable program code includes execution instructions, and the execution instructions are used to execute the steps of the speech segmentation detection method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used to enable a computer to execute the steps of the speech segmentation detection method according to any one of claims 1 to 8.

Images