Method for accelerating voice recognition in voice interaction process, computer program product and voice interaction system
By identifying and generating responsive speech in advance in the voice interaction system and analyzing the semantic similarity of the speech sentence update part, the problem of slow response speed in the traditional system is solved and faster speech interaction response is achieved.
Patent Information
- Application Number
- CN202411317981.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-27
AI Technical Summary
The traditional voice interaction system responds slowly when waiting for the voice activity detector to send an end signal, resulting in a slower response speed of the voice interaction system.
Before detecting that the voice sentence input is input, convert the received voice part into voice text in advance and generate a response voice. If the updated part is received, the semantic similarity is analyzed. If it is less than the preset value, the response voice will be updated; if it is not If the updated part is received or the semantic similarity is not less than the preset value, the original response voice will remain unchanged until the voice sentence is entered is played directly.
Save time waiting for the end signal, significantly speeding up the response speed of the voice interaction system, especially when there is an update part of the voice sentence but the update part is not very meaningful, the response voice is generated in advance, further improving the response speed.
Smart Images
Figure CN119207420A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for accelerating speech recognition during a speech interaction process, a computer program product, and a speech interaction system. Background Art
[0002] With the rapid development of artificial intelligence technology, voice interaction systems based on artificial intelligence have emerged. Telephone voice assistants are a common scenario for voice interaction systems. After being turned on, telephone voice assistants can answer calls on behalf of users, interact with callers during calls, and identify fraud risks based on the content of the interaction to prevent users from being defrauded.
[0003] The voice interaction system usually includes a voice activity detector, a voice recognition engine, a large language model engine, and a text-to-speech engine. During the voice interaction process, the traditional voice interaction system will wait until the voice activity detector detects the end of the speech and sends an end signal before connecting the voice activity detector to the voice recognition engine, and then sending the detected voice to the voice recognition engine to convert it into voice text. The large language model engine then generates a response text based on the voice text, and the text-to-speech engine converts the response text into a response voice. In this process, it takes a certain amount of time to wait for the voice activity detector to send an end signal, resulting in a slow response speed of the voice interaction system. Summary of the invention
[0004] The technical problem to be solved by the present invention is how to increase the response speed of the voice interaction system.
[0005] In order to solve the above technical problems, the present invention provides a method for accelerating speech recognition during speech interaction, which is applied to a speech interaction system of a telephone voice assistant and comprises the following steps:
[0006] Before detecting the end of the voice sentence input, convert the received part of the voice sentence into a first voice text, generate a response voice according to the first voice text, and if an updated part of the voice sentence is received, analyze the semantic similarity between the second voice text containing the updated part and the first voice text before the updated part, and if the semantic similarity is less than a preset value, generate an updated voice according to the second voice text, and use the updated voice as the response voice instead;
[0007] After detecting that the voice sentence input is completed, play the response voice.
[0008] Furthermore, if the updated part of the voice sentence is not received, the response voice is maintained unchanged.
[0009] Furthermore, if it is analyzed that the semantic similarity is not less than a preset value, the response voice is maintained unchanged.
[0010] Furthermore, the received part of the voice sentence is converted into the first voice text. Specifically, the received part of the voice sentence is sent to the voice recognition engine, and the voice recognition engine is used to convert the received part of the voice sentence into the first voice text.
[0011] Furthermore, generating a response voice according to the first voice text specifically includes: sending the first voice text to a large language model engine, using the large language model engine to generate a response text according to the first voice text, and then sending the response text to a text-to-speech engine, and using the text-to-speech engine to convert the response text into a response voice.
[0012] Furthermore, the updated voice is generated according to the second voice text specifically: the second voice text is sent to the large language model engine, the large language model engine is used to generate an updated text according to the second voice text, and then the updated text is sent to the text-to-speech engine, and the text-to-speech engine is used to convert the updated text into an updated voice.
[0013] Furthermore, the semantic similarity between the second voice text and the first voice text is analyzed specifically: the updated part of the voice sentence is sent to the voice recognition engine, the voice recognition engine is used to combine the previously received part and the updated part of the voice sentence into the second voice text, and then the semantic similarity between the second voice text containing the updated part and the first voice text before the updated part is analyzed.
[0014] Furthermore, if no voice input is detected within a certain period of time, the end of the voice sentence input is detected.
[0015] The present invention also provides a computer program product, comprising a computer program, which implements the steps in the above method when executed by a processor.
[0016] The present invention also provides a voice interaction system, which is applied to a telephone voice assistant, comprising a voice activity detector, a voice recognition engine, a large language model engine, a text-to-speech engine and a main control module, wherein the main control module is respectively connected to the voice activity detector, the voice recognition engine, the large language model engine and the text-to-speech engine, and the main control module comprises a memory and a processor connected to each other, wherein the memory stores the computer program product as described above; the main control module uses the voice activity detector to detect whether the input of the voice sentence is completed, uses the voice recognition engine to convert the received part of the voice sentence into a first voice text, uses the large language model engine and the text-to-speech engine to generate a response voice according to the first voice text, and uses the large language model engine and the text-to-speech engine to generate an update voice according to the second voice text.
[0017] The present invention has the following beneficial effects: the present invention recognizes the voice sentence and generates the response voice before detecting the end of the voice sentence input, and directly plays the response voice after detecting the end of the voice sentence input, which can save the time of waiting for the speech end signal and speed up the response speed of the voice interaction system. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flowchart of a method for accelerating speech recognition during speech interaction. DETAILED DESCRIPTION
[0019] The present invention is further described in detail below in conjunction with specific implementation methods.
[0020] The present embodiment provides a voice interaction system, which is applied to a telephone voice assistant, including a voice activity detection (VAD), a speech recognition engine (Automatic Speech Recognition, ASR), a large language model engine (LLM), a text-to-speech engine (TTS) and a main control module, wherein the voice recognition engine, the large language model engine and the text-to-speech engine are connected to each other with hot-swappable characteristics, and the main control module is respectively connected to the voice activity detector, the speech recognition engine, the large language model engine and the text-to-speech engine, and the main control module includes a memory and a processor connected to each other, and a computer program product is stored in the memory, and the computer program product includes a computer program. After the telephone voice assistant is turned on, it can answer the phone on behalf of the user. During the call, the telephone voice assistant conducts voice interaction with the caller through the voice interaction system, identifies fraud risks according to the interaction content, and prevents the user from being defrauded. During the voice interaction process, the processor of the main control module executes the computer program in the memory, and the computer program is implemented when the processor executes the following Figure 1 The method shown is for accelerating speech recognition during speech interaction, thereby speeding up the response speed of the speech interaction system.
[0021] The method for accelerating speech recognition in the speech interaction process includes the following steps implemented by the main control module: connecting the speech activity detector to the speech recognition engine, and then using the speech activity detector to detect the speech sentence input in real time. It should be noted that after the caller pauses for a certain period of time (for example, 1 second), it means that the speech sentence input is finished. Therefore, if the speech activity detector cannot detect the speech input within a certain period of time (for example, 1 second), it will send a signal that the speech sentence input is finished. The main control module detects the end of the speech sentence input when it receives the end signal. Before detecting the end of the speech sentence input, the received part of the speech sentence is converted into a first speech text, specifically, the received part of the speech sentence is sent to the speech recognition engine, and the speech recognition engine is used to convert the received part of the speech sentence into the first speech text. Then, a response speech is generated and cached according to the first speech text, specifically, the first speech text is sent to the large language model engine, and the large language model engine is used to generate a response text according to the first speech text, and then the response text is sent to the text-to-speech engine, and the text-to-speech engine is used to convert the response text into a response speech and then cached. During this process, a voice activity detector is continuously used to detect voice sentence input in real time, specifically to detect whether an updated part of the voice sentence is received. If the updated part of the voice sentence is not received, the response voice is maintained unchanged. If the updated part of the voice sentence is received, the previously received part of the voice sentence and the updated part are converted into a second voice text together. Specifically, the updated part of the voice sentence is sent to a voice recognition engine, and the voice recognition engine is used to combine the previously received part of the voice sentence and the updated part to convert them into the second voice text. Then, the semantic similarity between the second voice text containing the updated part and the first voice text before the updated part is analyzed. If the semantic similarity is not less than the preset value (for example, 90%), it means that the updated part is not very meaningful. The updated part can be ignored when responding to the voice sentence, so the response voice is maintained unchanged; if the semantic similarity is less than the preset value, it means that the updated part is meaningful. The updated part needs to be considered when responding to the voice sentence, so the updated voice is generated according to the second voice text. Specifically, the second voice text is sent to the large language model engine, and the large language model engine is used to generate an updated text according to the second voice text, and then the updated text is sent to the text-to-speech engine, and the text-to-speech engine is used to convert the updated text into an updated voice. After the updated voice is generated, the updated voice is used as the new response voice, specifically, the originally cached response voice is overwritten with the updated voice, so that the updated voice cache becomes the new response voice.After detecting that the voice sentence input has ended, the cached response voice is played directly. Specifically: if the voice sentence has no updated part, the response voice will be maintained unchanged, so the text-to-speech engine will directly play the originally cached response voice after detecting that the voice sentence input has ended; if the voice sentence has an updated part but the updated part is not meaningful, the response voice will be maintained unchanged, so the text-to-speech engine will directly play the originally cached response voice after detecting that the voice sentence input has ended; if the voice sentence has an updated part and the updated part is meaningful, an updated voice will be generated and used as the new response voice, so the text-to-speech engine will play the newly cached response voice after detecting that the voice sentence input has ended.
[0022] As can be seen from the above, whether the response voice is maintained unchanged or an updated voice is generated and used as a new response voice, the voice sentence is recognized and the response voice is generated before the end of the voice sentence input is detected, and the response voice is played directly after the end of the voice sentence input is detected. This can save the time of waiting for the voice activity detector to send an end signal, and can speed up the response speed of the voice interaction system. In addition, when the voice sentence has an updated part but the updated part is not meaningful, the response voice is maintained unchanged, and the originally cached response voice is played directly after the end of the voice sentence input is detected, without generating an updated voice based on the updated part. In this way, the response voice is generated before the caller says the updated part of the voice sentence, saving the time of the caller saying the updated part of the voice sentence, and can further speed up the response speed of the voice interaction system.
[0023] For example, a caller is about to say a voice sentence "The weather is so nice". When the caller says "today", the voice activity detector has received this part of the voice and sent it to the voice recognition engine to convert it into a first voice text. Then the large language model engine generates a response text based on the first voice text, and the text-to-speech engine generates a response voice based on the response text and caches it. When the user says "It's so nice", "It's so nice" is the updated part of the voice sentence, so the voice recognition engine converts "The weather is so nice" into a second voice text, and then analyzes the semantic similarity between the second voice text "The weather is so nice" and the first voice text "The weather". Since the semantics of the two are very different, the analysis result will be that the semantic similarity is less than the preset value. Therefore, the large language model engine generates an updated text based on the second voice text, and the text-to-speech engine generates an updated voice based on the updated text. Then, the updated voice is used to overwrite the originally cached response voice, so that the updated voice cache becomes the new response voice. When the user says "ah", "ah" is the updated part of the voice sentence. At this time, the speech recognition engine converts "The weather is so good" into a second voice text. The second voice text becomes "The weather is so good" containing the updated part "ah", and the first text becomes "The weather is so good" before the updated part "ah". Then, the semantic similarity between the second voice text "The weather is so good" and the first voice text "The weather is so good" is analyzed. Since the word "ah" is an interjection and has little meaning, the semantics of the second voice text and the first voice text are not much different. Therefore, the analysis result will be that the semantic similarity is not less than the preset value, so the response voice is maintained unchanged. When the caller finishes saying "The weather is so good" and pauses for 1 second, the voice activity detector will send a signal that the voice sentence input has ended, and then the text-to-speech engine will directly play the response voice corresponding to "The weather is so good". It can be seen that the voice sentence has been recognized and the corresponding response voice "The weather is so nice" has been generated before the end of the voice sentence input is detected. The response voice is played directly after the end of the voice sentence input is detected. This can save the time of waiting for the voice activity detector to send an end signal, and can speed up the response speed of the voice interaction system.
[0024] The above is only an implementation method of the invention, and does not limit the scope of patent protection. Those skilled in the art can make non-substantial changes or substitutions based on the invention, which still fall within the scope of patent protection.
Claims
1. A method for accelerating speech recognition during speech interaction, the method being applied to a speech interaction system of a telephone voice assistant, characterized in that: The steps include: Before detecting the end of the voice sentence input, convert the received part of the voice sentence into a first voice text, generate a response voice according to the first voice text, and if an updated part of the voice sentence is received, analyze the semantic similarity between the second voice text containing the updated part and the first voice text before the updated part, and if the semantic similarity is less than a preset value, generate an updated voice according to the second voice text, and use the updated voice as the response voice instead; After detecting that the voice sentence input is completed, play the response voice.
2. The method according to claim 1, characterized in that: If the updated part of the voice sentence is not received, the response voice remains unchanged.
3. The method according to claim 1, characterized in that: If the semantic similarity is analyzed to be not less than a preset value, the response voice is maintained unchanged.
4. The method according to claim 1, characterized in that: Converting the received part of the speech sentence into the first speech text specifically: sending the received part of the speech sentence to the speech recognition engine, and converting the received part of the speech sentence into the first speech text by using the speech recognition engine.
5. The method according to claim 1, characterized in that: Generate response speech according to the first speech text specifically: send the first speech text to the large language model engine, use the large language model engine to generate response text according to the first speech text, then send the response text to the text-to-speech engine, and use the text-to-speech engine to convert the response text into response speech.
6. The method according to claim 1, characterized in that: Generate updated speech according to the second speech text specifically: send the second speech text to the large language model engine, use the large language model engine to generate updated text according to the second speech text, then send the updated text to the text-to-speech engine, and use the text-to-speech engine to convert the updated text into updated speech.
7. The method according to claim 1, characterized in that: Analyze the semantic similarity between the second voice text and the first voice text specifically: send the updated part of the voice sentence to the voice recognition engine, use the voice recognition engine to combine the previously received part and the updated part of the voice sentence into the second voice text, and then analyze the semantic similarity between the second voice text containing the updated part and the first voice text before the updated part.
8. The method according to claim 1, characterized in that: If no voice input is detected within a certain period of time, the end of voice sentence input is detected.
9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.
10. A voice interaction system, which is applied to a telephone voice assistant, characterized in that: The invention comprises a voice activity detector, a voice recognition engine, a large language model engine, a text-to-speech engine and a main control module, wherein the main control module is respectively connected to the voice activity detector, the voice recognition engine, the large language model engine and the text-to-speech engine, and the main control module comprises a memory and a processor connected to each other, wherein the memory stores the computer program product according to claim 9; the main control module uses the voice activity detector to detect whether the input of the voice sentence is finished, uses the voice recognition engine to convert the received part of the voice sentence into a first voice text, uses the large language model engine and the text-to-speech engine to generate a response voice according to the first voice text, and uses the large language model engine and the text-to-speech engine to generate an update voice according to the second voice text.