Voice activity end prediction method in voice interaction process, computer program product and voice interaction system

Abstract

The invention provides a voice activity end prediction method in a voice interaction process, a computer program product and a voice interaction system. The method is applied to the voice interaction system of a telephone voice assistant, performs voice interaction with an incoming call user in a call process, identifies a fraud risk according to interaction content, and prevents the user from being fraud. The method comprises the following steps: detecting voice activity input in real time, acquiring voice activity input frequency before detecting that the voice activity input is ended, judging the probability that an incoming call user is speaking according to the voice activity input frequency, and if the probability that the incoming call user is speaking is lower than a first threshold value or the decreasing rate of the probability is faster than a second threshold value, judging that the incoming call user is speaking. And if yes, pre-judging that the voice activity input is ended and outputting the voice activity detection result, so that the condition that the incoming call user prepares to end speaking can be pre-judged without waiting for sending a speaking end signal, the time for waiting for sending the speaking end signal can be saved, and the response speed of the voice interaction system can be accelerated.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for predicting the end of a voice activity in a voice interaction process, a computer program product, and a voice interaction system. Background Art

[0002] With the rapid development of artificial intelligence technology, voice interaction systems based on artificial intelligence have emerged. Telephone voice assistants are a common scenario for voice interaction systems. After being turned on, telephone voice assistants can answer calls on behalf of users, interact with callers during calls, and identify fraud risks based on the content of the interaction to prevent users from being defrauded.

[0003] The voice interaction system usually includes a voice activity detector, a voice recognition engine, a large language model engine, and a text-to-speech engine. During the voice interaction process, the traditional voice interaction system has to wait until the voice activity detector detects the end of the speech and sends an end signal before connecting the voice activity detector to the voice recognition engine, and then sending the detected voice to the voice recognition engine to convert it into voice text. Then the large language model engine generates response text based on the voice text, and the text-to-speech engine converts the response text into response voice. In this process, it takes a certain amount of time to wait for the voice activity detector to send an end signal, resulting in a slow response speed of the voice interaction system. Summary of the invention

[0004] The technical problem to be solved by the present invention is how to increase the response speed of the voice interaction system.

[0005] To solve the above technical problems, the present invention provides a method for predicting the end of voice activity in a voice interaction process. The method is applied to a voice interaction system of a telephone voice assistant, and comprises the following steps: detecting voice activity input in real time, obtaining the voice activity input frequency before detecting the end of the voice activity input, and judging the probability that the incoming caller is speaking based on the voice activity input frequency; if the probability that the incoming caller is speaking is lower than a first threshold or the probability decrease rate is faster than a second threshold, predicting the end of the voice activity input and outputting the voice activity detection result.

[0006] Further, if the probability that the incoming caller is speaking is not lower than a first threshold and the probability decrease rate is not faster than a second threshold, the detection of voice activity input is maintained.

[0007] Furthermore, the higher the voice activity input frequency, the higher the probability that the caller is speaking.

[0008] Furthermore, the voice activity input frequency is obtained specifically as follows: within each time range, whether voice activity is being input is detected every short period of time, and then the number of time periods in which voice activity is detected to be input is divided by the total number of time periods to obtain the voice activity input frequency.

[0009] Furthermore, after the voice activity detection result is output, the voice activity detection result is converted into voice text, and then a response text is generated according to the voice text, and then the response text is converted into a response voice and played.

[0010] The present invention also provides a computer program product, comprising a computer program, which implements the steps in the above method when executed by a processor.

[0011] The present invention also provides a voice interaction system, which is applied to a telephone voice assistant, comprising a voice activity detector, a voice recognition engine, a large language model engine, a text-to-speech engine and a main control module, wherein the main control module is respectively connected to the voice activity detector, the voice recognition engine, the large language model engine and the text-to-speech engine, and the main control module comprises a memory and a processor connected to each other, wherein the memory stores the computer program product as described above; the main control module uses the voice activity detector to detect whether the voice activity input is completed.

[0012] The present invention has the following beneficial effects: before detecting the end of voice activity input, if it is determined that the probability that the caller is speaking is lower than a first threshold or the probability decrease rate is faster than a second threshold, it means that the caller is ready to finish speaking, so the end of the voice activity input can be predicted and the voice activity detection result can be output. In this way, it is possible to predict that the caller is ready to finish speaking without waiting for the speech end signal to be sent, which can save the time of waiting for the speech end signal to be sent and speed up the response speed of the voice interaction system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flowchart of a method for predicting the end of speech activity during speech interaction. DETAILED DESCRIPTION

[0014] The present invention is further described in detail below in conjunction with specific implementation methods.

[0015] The present embodiment provides a voice interaction system, which is applied to a telephone voice assistant, including a voice activity detection (VAD), a speech recognition engine (Automatic Speech Recognition, ASR), a large language model engine (LLM), a text-to-speech engine (TTS) and a main control module, wherein the voice recognition engine, the large language model engine and the text-to-speech engine are connected to each other with hot-swappable characteristics, and the main control module is respectively connected to the voice activity detector, the speech recognition engine, the large language model engine and the text-to-speech engine, and the main control module includes a memory and a processor connected to each other, and a computer program product is stored in the memory, and the computer program product includes a computer program. After the telephone voice assistant is turned on, it can answer the phone on behalf of the user. During the call, the telephone voice assistant conducts voice interaction with the caller through the voice interaction system, identifies fraud risks according to the interaction content, and prevents the user from being defrauded. During the voice interaction process, the processor of the main control module executes the computer program in the memory, and the computer program is implemented when the processor executes the following Figure 1 The method for predicting the end of speech activity during speech interaction is shown, thereby speeding up the response speed of the speech interaction system.

[0016] The method for predicting the end of voice activity in the voice interaction process includes the following steps implemented by the main control module: connecting the voice activity detector to the voice recognition engine, and then using the voice activity detector to detect the voice activity input in real time. It should be noted that after the caller pauses for a certain period of time (for example, 1 second), it means that the voice sentence input is finished. Therefore, if the voice activity detector cannot detect the voice input within a certain period of time (for example, 1 second), it will send a signal that the voice sentence input is finished. The main control module detects the end of the voice sentence input when it receives the end signal. Before the end of the voice activity input is detected, the voice activity input frequency is obtained by the voice activity detector. Specifically, within each time range, it is detected every small period of time whether the voice activity is being input, and then the number of time periods in which the voice activity is detected is divided by the total number of time periods to obtain the voice activity input frequency. For example, within each second, it is detected every 0.2 seconds whether the voice activity is being input, and the total number of time periods is 5. Then the number of time periods in which the voice activity is detected is divided by the total number of time periods (5 periods) to obtain the voice activity input frequency. Then, the probability that the caller is speaking is determined according to the voice activity input frequency, wherein the higher the voice activity input frequency, the higher the probability that the caller is speaking. If the probability that the incoming caller is speaking is not lower than the first threshold (e.g., 0.4) and the probability decrease rate is not faster than the second threshold (e.g., 80%), it means that the incoming caller is not ready to finish speaking, so the voice activity input detection is maintained; if the probability that the incoming caller is speaking is lower than the first threshold or the probability decrease rate is faster than the second threshold, it means that the incoming caller is ready to finish speaking, so the voice activity input can be predicted to end and the voice activity detection result can be output, so that the incoming caller can be predicted to end speaking without waiting for the voice activity detector to send a speech end signal, which can save the time of waiting for the speech end signal to be sent, and can speed up the response speed of the voice interaction system. Among them, judging whether the probability decrease rate is faster than the second threshold, specifically, first calculate the difference in the probability that the user is speaking two seconds before and after, and then divide the probability difference by the probability that the user is speaking in the first second to obtain the probability decrease rate, and compare the probability decrease rate with the second threshold to know whether the probability decrease rate is faster than the second threshold.

[0017] The voice activity detector outputs the voice activity detection result specifically to the voice recognition engine. The voice recognition engine converts the voice activity detection result into voice text, and then sends the voice text to the large language model engine. The large language model engine generates a response text based on the voice text, and then sends the response text to the text-to-speech engine. The text-to-speech engine converts the response text into a response voice and plays it, thereby realizing voice interaction with the incoming caller.

[0018] The above is only an implementation method of the invention, and does not limit the scope of patent protection. Those skilled in the art can make non-substantial changes or substitutions based on the invention, which still fall within the scope of patent protection.

Claims

1. A method for predicting the end of a voice activity during a voice interaction process, the method being applied to a voice interaction system of a telephone voice assistant, characterized in that: The method comprises the following steps: detecting voice activity input in real time, obtaining the voice activity input frequency before detecting the end of the voice activity input, judging the probability that the caller is speaking according to the voice activity input frequency, and if the probability that the caller is speaking is lower than a first threshold or the probability decrease rate is faster than a second threshold, predicting the end of the voice activity input and outputting the voice activity detection result.

2. The method for predicting the end of speech activity according to claim 1, characterized in that: If the probability that the incoming caller is speaking is not lower than the first threshold and the probability decrease rate is not faster than the second threshold, then the detection of the voice activity input is maintained.

3. The method for predicting the end of speech activity according to claim 1, characterized in that: The higher the voice activity input frequency, the higher the probability that the caller is speaking.

4. The method for predicting the end of speech activity according to claim 1, characterized in that: The voice activity input frequency is obtained specifically as follows: within each time range, whether voice activity is being input is detected every short period of time, and then the number of time periods in which voice activity is detected to be input is divided by the total number of time periods to obtain the voice activity input frequency.

5. The method for predicting the end of speech activity according to claim 1, characterized in that: After the voice activity detection result is output, the voice activity detection result is converted into voice text, and then a response text is generated according to the voice text, and then the response text is converted into a response voice and played.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 5 are implemented.

7. A voice interaction system, which is applied to a telephone voice assistant, characterized in that: The invention comprises a voice activity detector, a voice recognition engine, a large language model engine, a text-to-speech engine and a main control module, wherein the main control module is respectively connected to the voice activity detector, the voice recognition engine, the large language model engine and the text-to-speech engine, and the main control module comprises a memory and a processor connected to each other, wherein the memory stores the computer program product according to claim 6; the main control module uses the voice activity detector to detect whether the voice activity input is finished.

Images