Voice interaction method and apparatus

Abstract

The embodiments of the present application relate to the technical field of voice interaction. Provided are a voice interaction method and an apparatus. The method comprises: during reception of a voice input from a user, detecting in real time whether a pause duration of the voice input is longer than or equal to a first duration; when the pause duration of the voice input is longer than or equal to the first duration, acquiring a first reply text corresponding to a first voice recognition result, and detecting whether the pause duration of the voice input is longer than or equal to a second duration, the first voice recognition result being a voice recognition result obtained by performing voice recognition on the voice input when the pause duration of the voice input is equal to the first duration, and the second duration being longer than the first duration; and if the pause duration of the voice input is longer than or equal to the second duration, generating, on the basis of the first reply text, a voice output corresponding to the voice input.

Description

A voice interaction method and device

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410502254.7, filed on April 24, 2024, entitled “A Voice Interaction Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of voice interaction technology, and in particular to a voice interaction method and apparatus. Background Technology

[0004] With the rapid development of science and technology, voice interaction systems based on large language models (LLMs) have become a hot research topic. Summary of the Invention

[0005] In view of this, embodiments of this application provide a voice interaction method and apparatus for reducing the response delay of a voice interaction system while avoiding interrupting the user's voice input.

[0006] To achieve the above objectives, the technical solutions provided in this application are as follows:

[0007] In a first aspect, embodiments of this application provide a voice interaction method, including:

[0008] During the process of receiving user voice input, the pause duration of the voice input is detected in real time to see if it is greater than or equal to the first duration.

[0009] When the pause duration of the voice input is greater than or equal to the first duration, the first response text corresponding to the first voice recognition result is obtained, and it is detected whether the pause duration of the voice input is greater than or equal to the second duration; the first voice recognition result is the voice recognition result obtained by performing voice recognition on the voice input when the pause duration of the voice input is equal to the first duration, and the second duration is greater than the first duration;

[0010] If the pause duration of the voice input is greater than or equal to the second duration, then the voice output corresponding to the voice input is generated based on the first reply text.

[0011] As an optional implementation of this application, the method further includes:

[0012] If the pause duration of the voice input is less than the second duration, the first reply text is discarded, and the pause duration of the voice input is re-checked to see if it is greater than or equal to the first duration.

[0013] As an optional implementation of this application, before obtaining the first response text corresponding to the first speech recognition result, the method further includes: determining whether the semantics of the first speech recognition result are complete;

[0014] The step of obtaining the first response text corresponding to the first speech recognition result includes: obtaining the first response text in response to determining the semantic integrity of the first speech recognition result.

[0015] As an optional implementation of this application, if the semantics of the first speech recognition result is incomplete, then when the pause duration of the speech input is equal to the second duration, the second response text corresponding to the second speech recognition result is obtained, and the speech output corresponding to the speech input is generated based on the second response text;

[0016] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0017] As an optional implementation of this application, determining whether the semantics of the first speech recognition result are complete includes:

[0018] The probability of semantic completeness of the first speech recognition result is obtained by using a pre-trained completeness prediction model.

[0019] Determine whether the probability of semantic integrity of the first speech recognition result is greater than a probability threshold;

[0020] If so, then the semantic integrity of the first speech recognition result is determined;

[0021] If not, then the semantics of the first speech recognition result are determined to be incomplete.

[0022] As an optional implementation of this application, the step of generating the voice output corresponding to the voice input based on the first reply text includes:

[0023] Determine whether the second speech recognition result is the same as the first speech recognition result. The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0024] If the second speech recognition result is the same as the first speech recognition result, then speech synthesis is performed on the first reply text to generate the speech output corresponding to the speech input.

[0025] As an optional implementation of this application, the method further includes:

[0026] If the second speech recognition result is different from the first speech recognition result, then the second response text corresponding to the second speech recognition result is obtained, and the second response text is processed by speech synthesis to generate the speech output corresponding to the speech input.

[0027] Secondly, embodiments of this application provide a voice interaction device, including:

[0028] The detection unit is used to detect in real time whether the pause duration of the voice input is greater than or equal to a first duration during the process of receiving the user's voice input;

[0029] The acquisition unit is configured to acquire a first response text corresponding to a first speech recognition result when the pause duration of the speech input is greater than or equal to the first duration, wherein the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration;

[0030] The detection unit is further configured to detect whether the pause duration of the voice input is greater than or equal to a second duration when the pause duration of the voice input is greater than or equal to the first duration, wherein the second duration is greater than the first duration;

[0031] The output unit is configured to generate a voice output corresponding to the voice input based on the first reply text when the pause duration of the voice input is greater than or equal to the second duration.

[0032] As an optional implementation method of this application,

[0033] The output unit is further configured to discard the first reply text if the pause duration of the voice input is less than the second duration;

[0034] The detection unit is also used to re-detect whether the pause duration of the voice input is greater than or equal to the first duration.

[0035] As an optional implementation of this application, the acquisition unit is further configured to determine whether the semantics of the first speech recognition result are complete before acquiring the first reply text corresponding to the first speech recognition result;

[0036] The acquisition unit is specifically used to acquire the first response text in response to determining the semantic integrity of the first speech recognition result.

[0037] As an optional implementation of this application, the acquisition unit is further configured to acquire the second response text corresponding to the second speech recognition result when the pause duration of the speech input is equal to the second duration, in the case where the semantics of the first speech recognition result is incomplete;

[0038] The output unit is further configured to generate a voice output corresponding to the voice input based on the second reply text;

[0039] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0040] As an optional implementation of this application, the acquisition unit is specifically used to acquire the probability of semantic completeness of the first speech recognition result through a pre-trained completeness prediction model; determine whether the probability of semantic completeness of the first speech recognition result is greater than a probability threshold; if yes, determine that the first speech recognition result is semantically complete; if no, determine that the first speech recognition result is semantically incomplete.

[0041] As an optional implementation of this application, the output unit is specifically used to determine whether the second speech recognition result is the same as the first speech recognition result. The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration. If the second speech recognition result is the same as the first speech recognition result, speech synthesis is performed on the first reply text to generate the speech output corresponding to the speech input.

[0042] As an optional implementation of this application, the output unit is further configured to obtain a second response text corresponding to the second speech recognition result when the second speech recognition result is different from the first speech recognition result, and to perform speech synthesis on the second response text to generate a speech output corresponding to the speech input.

[0043] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to cause the electronic device to implement the voice interaction method described in any of the above embodiments when executing the computer program.

[0044] Fourthly, embodiments of this application provide a computer-readable storage medium that, when executed by a computing device, enables the computing device to implement any of the aforementioned voice interaction methods.

[0045] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, enables the computer to implement any of the aforementioned voice interaction methods.

[0046] The voice interaction method provided in this application embodiment detects in real time whether the pause duration of the voice input is greater than or equal to a first duration during the process of receiving user voice input. When the pause duration is greater than or equal to the first duration, it obtains a first response text corresponding to the first speech recognition result obtained by performing speech recognition on the voice input when the pause duration is equal to the first duration. It also detects whether the pause duration is greater than or equal to a second duration, and if the pause duration is greater than or equal to the second duration, it generates a voice output corresponding to the voice input based on the first response text. Since the second duration is greater than the first duration, and this application embodiment obtains the first response text corresponding to the first speech recognition result when the pause duration is greater than or equal to the first duration, and then directly outputs based on the first response text when the pause duration is greater than or equal to the second duration, this application embodiment can reduce the response delay of the voice interaction system. Furthermore, in this embodiment of the application, when the pause duration of the voice input is greater than or equal to the first duration, the user's voice input will not be directly interrupted. Instead, the pause duration of the voice input will continue to be detected as greater than or equal to the second duration. Only when the pause duration of the voice input is greater than or equal to the second duration will the voice output corresponding to the voice input be generated based on the first reply text. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings that need to be called in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 is a schematic diagram of the structure of the voice interaction system provided in an embodiment of this application;

[0050] Figure 2 is a flowchart of one of the voice interaction methods provided in the embodiments of this application;

[0051] Figure 3 is a second flowchart of the voice interaction method provided in the embodiment of this application;

[0052] Figure 4 is a flowchart of the third embodiment of the voice interaction method provided in this application.

[0053] Figure 5 is a structural schematic diagram of the voice interaction device provided in an embodiment of this application;

[0054] Figure 6 is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application. Detailed Implementation

[0055] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0056] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0057] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Furthermore, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0058] Voice interaction systems typically wait for a pause in the user's voice input to exceed a certain duration before ending the reception of the input and initiating downstream tasks. Therefore, after a user stops speaking, they must wait for the required pause duration and the processing time of the downstream task before hearing a response from the voice interaction system. In traditional tasks like using speakers and mobile assistants, user voice input is usually short commands, and downstream system processing is quick, so there are no significant delays or user interruptions. However, for current voice interaction systems based on large language models, user voice input is often lengthy and completely open-ended, leading to frequent hesitations and pauses. Therefore, the waiting time must be set much longer than in traditional tasks to prevent premature interruption. The large language model also requires a longer time to generate a response after receiving the speech recognition result. The combined effect of these two delays significantly increases the response latency of voice interaction systems based on large language models, severely impacting the user experience.

[0059] To reduce the response latency of a voice interaction system based on a large language model while avoiding interruption of the user's voice input, this application provides a voice interaction method and apparatus. This method obtains a first response text corresponding to a first speech recognition result through a pre-trained large language model when the pause duration of the voice input is greater than or equal to a first duration. Then, when the pause duration of the voice input is greater than or equal to a second duration, it directly outputs based on the first response text, thereby reducing the response latency of the voice interaction system based on the large language model. Furthermore, since the user's voice input is not directly interrupted when the pause duration of the voice input is greater than or equal to the first duration, but rather the pause duration is continuously checked to see if it is greater than or equal to the second duration, and the voice output corresponding to the voice input is generated based on the first response text only when the pause duration is greater than or equal to the second duration, this application also avoids interrupting the user's voice input. Therefore, this application can reduce the response latency of the voice interaction system while avoiding interruption of the user's voice input.

[0060] Referring to FIG1, the voice interaction system based on a large language model in this embodiment includes: a voice input module 11, a voice recognition module 12, a completeness prediction module 13, a large language model 14, and a voice output module 15.

[0061] The voice input module 11 is used to receive the user's voice input and can be an audio receiving device such as a microphone.

[0062] The speech recognition module 12 is used to convert the speech input received by the speech input module 11 into text content. In some embodiments, the speech recognition module 12 can be a streaming speech recognition module that recognizes speech input in real time. Compared to recognizing speech input after it ends, streaming speech recognition can reduce the time consumption of speech recognition because the streaming speech recognition module can recognize speech input in real time.

[0063] The completeness prediction module 13 is used to predict the semantic completeness of the speech recognition result of the speech recognition module 12. In some embodiments, the completeness prediction module 13 can be a completeness prediction module based on a binary classification model, and the output of the completeness prediction module 13 is the semantic completeness or semantic incompleteness of the speech recognition result. In other embodiments, the output of the completeness prediction module 13 can also be the probability of the semantic completeness of the speech recognition result.

[0064] The large language model 14 is used to generate the response text corresponding to the speech recognition results. The large language model 14 can be a model obtained by training machine learning models such as deep learning neural networks and convolutional neural networks on a sample dataset. The sample dataset can include multiple text inputs and the corresponding text outputs for each text input.

[0065] The voice output module 15 may include a voice synthesis unit 151 and a voice playback module 152. The voice synthesis module is used to convert the output of the large language model 14 into voice, and the voice playback module is used to play the voice synthesized by the voice synthesis unit 151 through a speaker or other sound playback device.

[0066] This application provides a voice interaction method. Referring to FIG2, the voice interaction method includes the following steps S21 to S24:

[0067] S21. During the process of receiving the user's voice input, the pause duration of the voice input is detected in real time to see if it is greater than or equal to the first duration.

[0068] In some embodiments, the implementation of real-time detection of whether the pause duration of the voice input is greater than or equal to a first duration includes: detecting pauses in the voice input using Voice Activity Detection (VAD) technology; when a pause is detected, starting a timer with a timeout duration of the first duration; if the timer expires, determining that the pause duration of the voice input is greater than or equal to the first duration; if voice input reception resumes before the timer expires, then re-detecting pauses in the voice input using the Voice Activity Detection (VAD) technology.

[0069] In some embodiments, pauses in speech input can be detected using speech endpoint detection techniques based on thresholds, statistical models, or deep learning.

[0070] In step S21 above, if the pause duration of the detected voice input is greater than or equal to the first duration, then steps S22 and S23 are executed as follows:

[0071] S22. Obtain the first response text corresponding to the first speech recognition result.

[0072] Wherein, the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration.

[0073] In some embodiments, obtaining the first response text corresponding to the first speech recognition result includes:

[0074] The first response text corresponding to the first speech recognition result is obtained by using a pre-trained large language model.

[0075] The output of the large language model when the first speech recognition result is used as the input of the first speech recognition result is used as the first response text corresponding to the first speech recognition result.

[0076] In some embodiments, the large language model can be a model obtained by training a machine learning model based on a sample dataset. The sample dataset may include multiple sample question contents and the corresponding question-and-answer results for each sample question content. Training the machine learning model based on the sample dataset includes: inputting the sample question contents into the machine learning model, obtaining the question-and-answer results predicted by the machine learning model, calculating the loss value between the question-and-answer results predicted by the machine learning model and the question-and-answer results corresponding to the sample question contents based on a preset loss function, and adjusting the model parameters of the machine learning model according to the loss value.

[0077] S23. Detect whether the pause duration of the voice input is greater than or equal to the second duration.

[0078] Wherein, the second duration is longer than the first duration.

[0079] In step S23 above, if the pause duration of the voice input is greater than or equal to the second duration, then step S24 is executed as follows:

[0080] S24. Generate the voice output corresponding to the voice input based on the first reply text.

[0081] In some embodiments, generating a voice output corresponding to the voice input based on the first reply text includes:

[0082] The first reply text is processed into speech, and the speech synthesis result of the first reply text is played back.

[0083] The voice interaction method provided in this application embodiment detects in real time whether the pause duration of the voice input is greater than or equal to a first duration during the process of receiving user voice input. When the pause duration is greater than or equal to the first duration, it obtains a first response text corresponding to the first speech recognition result obtained by performing speech recognition on the voice input when the pause duration is equal to the first duration. It also detects whether the pause duration is greater than or equal to a second duration, and if the pause duration is greater than or equal to the second duration, it generates a voice output corresponding to the voice input based on the first response text. Since the second duration is greater than the first duration, and this application embodiment obtains the first response text corresponding to the first speech recognition result when the pause duration is greater than or equal to the first duration, and then directly outputs based on the first response text when the pause duration is greater than or equal to the second duration, this application embodiment can reduce the response delay of the voice interaction system. Furthermore, because this embodiment does not directly interrupt the user's voice input when the pause duration of the voice input is greater than or equal to the first duration, but instead continues to detect whether the pause duration of the voice input is greater than or equal to the second duration, and only generates the voice output corresponding to the voice input based on the first reply text when the pause duration of the voice input is greater than or equal to the second duration, this embodiment can also avoid interrupting the user's voice input. In summary, this embodiment can reduce the response delay of the voice interaction system while avoiding interruption of the user's voice input.

[0084] As an extension and refinement of the above embodiments, this application provides another voice interaction method. Referring to FIG3, the voice interaction method includes the following steps:

[0085] S301. During the process of receiving user voice input, the pause duration of the voice input is detected in real time to see if it is greater than or equal to the first duration.

[0086] In step S301 above, if the pause duration of the voice input is greater than or equal to the first duration, then steps S302 and S303 are executed as follows:

[0087] S302. Obtain the first response text corresponding to the first speech recognition result.

[0088] Wherein, the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration.

[0089] S303. Detect whether the pause duration of the voice input is greater than or equal to the second duration.

[0090] Wherein, the second duration is longer than the first duration.

[0091] In step S303 above, if the pause duration of the voice input is less than the second duration, then step S304 is executed as follows:

[0092] S304. Discard the first reply text and re-detect whether the pause duration of the voice input is greater than or equal to the first duration.

[0093] That is, if the pause duration of the voice input is greater than or equal to the first duration and less than or equal to the second duration, the first reply text is discarded, and the process returns to step S301 to re-detect whether the pause duration of the voice input is greater than or equal to the first duration.

[0094] In step S303 above, if the pause duration of the voice input is greater than or equal to the second duration, then steps S305 to S308 are executed:

[0095] S305. Determine whether the second speech recognition result is the same as the first speech recognition result.

[0096] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0097] Since noise, ambient sound, and other factors can affect speech recognition results, the first speech recognition result obtained when the pause duration of the speech input is equal to the first duration may differ from the second speech recognition result obtained when the pause duration of the speech input is equal to the second duration.

[0098] In step S305 above, if the second speech recognition result is the same as the first speech recognition result, then step S306 is executed as follows:

[0099] S306. Perform speech synthesis on the first reply text to generate the speech output corresponding to the speech input.

[0100] In step S305 above, if the second speech recognition result is the same as the first speech recognition result, then the following steps S307 and S308 are executed:

[0101] S307. Obtain the second response text corresponding to the second speech recognition result.

[0102] S308. Perform speech synthesis on the second reply text to generate the speech output corresponding to the speech input.

[0103] As an extension and refinement of the above embodiments, this application provides another voice interaction method. Referring to FIG4, the voice interaction method includes the following steps:

[0104] S401. During the process of receiving the user's voice input, the pause duration of the voice input is detected in real time to see if it is greater than or equal to the first duration.

[0105] In step S401 above, if the pause duration of the voice input is greater than or equal to the first duration, then steps S402 and S403 are executed as follows:

[0106] S402. Determine whether the semantics of the first speech recognition result are complete.

[0107] Wherein, the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration.

[0108] In some embodiments, step S402 (determining whether the semantics of the first speech recognition result is complete) includes the following steps a to d:

[0109] Step a: Obtain the probability of semantic completeness of the first speech recognition result through a pre-trained completeness prediction model.

[0110] In some embodiments, the integrity prediction model can be a model obtained by training a machine learning model based on sample data. The sample data may include sample text and label data representing the semantic completeness of the sample text. The process of training the machine learning model based on sample data may include: inputting the sample text into the machine learning model, obtaining the label data predicted by the machine learning model to represent the semantic completeness of the sample text, calculating the loss value between the predicted label data and the actual label data of the sample text according to a preset loss function, and adjusting the model parameters of the machine learning model based on the loss value.

[0111] In some embodiments, the output of the integrity prediction model includes label data and the confidence level of the label data. When the label data represents the semantic integrity of the first speech recognition result, the confidence level is determined as the probability that the first speech recognition result is semantically complete. When the label data represents the semantic incompleteness of the first speech recognition result, the difference between 1 and the confidence level is determined as the probability that the first speech recognition result is semantically complete.

[0112] Step b: Determine whether the probability of semantic integrity of the first speech recognition result is greater than the probability threshold.

[0113] In step b above, if the probability of semantic integrity of the first speech recognition result is greater than the probability threshold, then step c is executed as follows:

[0114] Step c: Determine the semantic integrity of the first speech recognition result.

[0115] In step b above, if the probability of semantic integrity of the first speech recognition result is greater than the probability threshold, then step d is executed as follows:

[0116] Step d: Determine that the semantics of the first speech recognition result are incomplete.

[0117] S403. Detect whether the pause duration of the voice input is greater than or equal to the second duration.

[0118] Wherein, the second duration is longer than the first duration.

[0119] It should be noted that the judgment result of step S402 does not affect the execution of step S403. That is, step S403 will be executed as long as the pause duration of the voice input is detected to be greater than or equal to the first duration.

[0120] If it is determined in step S402 that the semantics of the first speech recognition result are incomplete, then, based on the judgment result of step S403, the subsequent process of the voice interaction method provided in this application embodiment includes:

[0121] If the pause duration of the voice input is less than the second duration, return to step S401 to detect in real time whether the pause duration of the voice input is greater than or equal to the first duration.

[0122] If the pause duration of the voice input is greater than or equal to the second duration, the following steps S404 and S405 are performed:

[0123] S404. Obtain the second response text corresponding to the second speech recognition result.

[0124] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0125] S405. Generate the voice output corresponding to the voice input based on the second reply text.

[0126] If it is determined in step S402 that the semantics of the first speech recognition result are incomplete, then step S406 is executed as follows:

[0127] S406. Obtain the first response text corresponding to the first speech recognition result.

[0128] Wherein, the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration.

[0129] After performing step S406 above, and based on the judgment result of step S403, the subsequent process of the voice interaction method provided in this application embodiment includes:

[0130] If the pause duration of the voice input is less than the second duration, perform the following step S407:

[0131] S407. Discard the first reply text and re-detect whether the pause duration of the voice input is greater than or equal to the first duration.

[0132] If the pause duration of the voice input is greater than or equal to the second duration, perform the following steps S408 to S410:

[0133] S408. Determine whether the second speech recognition result is the same as the first speech recognition result.

[0134] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0135] In step S408 above, if the second speech recognition result is the same as the first speech recognition result, then step S409 is executed as follows:

[0136] S409. Perform speech synthesis on the first reply text to generate the speech output corresponding to the speech input.

[0137] In step S408 above, if the second speech recognition result is the same as the first speech recognition result, then the following steps S410 and S411 are executed:

[0138] S410. Obtain the second response text corresponding to the second speech recognition result.

[0139] S411. Perform speech synthesis on the second reply text to generate the speech output corresponding to the speech input.

[0140] In the above embodiment, when the pause duration of the input is greater than or equal to the first duration, it first determines whether the semantics of the first speech recognition result obtained by performing speech recognition on the speech input is complete when the pause duration of the speech input is equal to the first duration. Only when the semantics of the first speech recognition result is complete will the first response text corresponding to the first speech recognition result be obtained. Therefore, when obtaining the response text corresponding to the speech recognition result through the large language model, the above implementation can avoid frequent calls to the large language model, thereby reducing the workload of the large language model.

[0141] Based on the same inventive concept, as an implementation of the above method, this application embodiment also provides a voice interaction device. This embodiment corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the voice interaction device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0142] This application provides a voice interaction device. Figure 5 is a schematic diagram of the structure of the voice interaction device. As shown in Figure 5, the voice interaction device 500 includes:

[0143] The detection unit 51 is used to detect in real time whether the pause duration of the voice input is greater than or equal to the first duration during the process of receiving the user's voice input.

[0144] The acquisition unit 52 is used to acquire a first response text corresponding to a first speech recognition result when the pause duration of the speech input is greater than or equal to the first duration. The first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration.

[0145] The detection unit 51 is further configured to detect whether the pause duration of the voice input is greater than or equal to a second duration when the pause duration of the voice input is greater than or equal to the first duration, wherein the second duration is greater than the first duration.

[0146] The output unit 53 is configured to generate a voice output corresponding to the voice input based on the first reply text when the pause duration of the voice input is greater than or equal to the second duration.

[0147] As an optional implementation method of this application,

[0148] The output unit 53 is also used to discard the first reply text if the pause duration of the voice input is less than the second duration;

[0149] The detection unit 51 is also used to re-detect whether the pause duration of the voice input is greater than or equal to the first duration.

[0150] As an optional implementation of this application, the acquisition unit 52 is further configured to determine whether the semantics of the first speech recognition result is complete before acquiring the first reply text corresponding to the first speech recognition result;

[0151] The acquisition unit 52 is specifically used to acquire the first reply text in response to determining the semantic integrity of the first speech recognition result.

[0152] As an optional implementation of this application, the acquisition unit 52 is further configured to acquire the second response text corresponding to the second speech recognition result when the semantics of the first speech recognition result are incomplete and the pause duration of the speech input is equal to the second duration.

[0153] The output unit 53 is further configured to generate a voice output corresponding to the voice input based on the second reply text;

[0154] The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

[0155] As an optional implementation of this application, the acquisition unit 52 is specifically used to acquire the probability of semantic completeness of the first speech recognition result through a pre-trained completeness prediction model; determine whether the probability of semantic completeness of the first speech recognition result is greater than a probability threshold; if yes, determine that the first speech recognition result is semantically complete; if no, determine that the first speech recognition result is semantically incomplete.

[0156] As an optional implementation of this application, the output unit 53 is specifically used to determine whether the second speech recognition result is the same as the first speech recognition result. The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration. If the second speech recognition result is the same as the first speech recognition result, speech synthesis is performed on the first reply text to generate the speech output corresponding to the speech input.

[0157] As an optional implementation of this application, the output unit 53 is further configured to obtain a second response text corresponding to the second speech recognition result when the second speech recognition result is different from the first speech recognition result, and to perform speech synthesis on the second response text to generate a speech output corresponding to the speech input.

[0158] The voice interaction device provided in this application embodiment can execute the voice interaction method provided in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0159] Based on the same inventive concept, this application also provides an electronic device. Figure 6 is a schematic diagram of the structure of the electronic device provided in this application embodiment. As shown in Figure 6, the electronic device provided in this embodiment includes: a memory 601 and a processor 602. The memory 601 is used to store a computer program, and the processor 602 is used to execute the voice interaction method provided in the above embodiment when executing the computer program.

[0160] Based on the same inventive concept, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the computing device to implement the voice interaction method provided in the above embodiments.

[0161] Based on the same inventive concept, this application also provides a computer program product that, when run on a computer, enables the computing device to implement the voice interaction method provided in the above embodiments.

[0162] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0163] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0164] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0165] Computer-readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media can store information using any method or technology; the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A voice interaction method, comprising: During the process of receiving user voice input, the pause duration of the voice input is detected in real time to see if it is greater than or equal to the first duration. When the pause duration of the voice input is greater than or equal to the first duration, the first response text corresponding to the first voice recognition result is obtained, and it is detected whether the pause duration of the voice input is greater than or equal to the second duration; the first voice recognition result is the voice recognition result obtained by performing voice recognition on the voice input when the pause duration of the voice input is equal to the first duration, and the second duration is greater than the first duration; If the pause duration of the voice input is greater than or equal to the second duration, then the voice output corresponding to the voice input is generated based on the first reply text.

2. The method according to claim 1, further comprising: If the pause duration of the voice input is less than the second duration, the first reply text is discarded, and the pause duration of the voice input is re-checked to see if it is greater than or equal to the first duration.

3. The method according to claim 1, wherein before obtaining the first response text corresponding to the first speech recognition result, it further includes: Determine whether the semantics of the first speech recognition result are complete; The step of obtaining the first response text corresponding to the first speech recognition result includes: obtaining the first response text in response to determining the semantic integrity of the first speech recognition result.

4. The method according to claim 3, wherein If the semantics of the first speech recognition result are incomplete, then when the pause duration of the speech input is equal to the second duration, the second response text corresponding to the second speech recognition result is obtained, and the speech output corresponding to the speech input is generated based on the second response text; in, The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration.

5. The method according to claim 3, wherein determining whether the semantics of the first speech recognition result are complete includes: The probability of semantic completeness of the first speech recognition result is obtained by using a pre-trained completeness prediction model. Determine whether the probability of semantic integrity of the first speech recognition result is greater than a probability threshold; If so, then the semantic integrity of the first speech recognition result is determined; If not, then the semantics of the first speech recognition result are determined to be incomplete.

6. The method according to claim 1, wherein generating the voice output corresponding to the voice input based on the first reply text includes: Determine whether the second speech recognition result is the same as the first speech recognition result. The second speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the second duration. If the second speech recognition result is the same as the first speech recognition result, then speech synthesis is performed on the first reply text to generate the speech output corresponding to the speech input.

7. The method according to claim 6, further comprising: If the second speech recognition result is different from the first speech recognition result, then the second response text corresponding to the second speech recognition result is obtained, and speech synthesis is performed on the second response text to generate the speech output corresponding to the speech input.

8. A voice interaction device, comprising: The detection unit is used to detect in real time whether the pause duration of the voice input is greater than or equal to a first duration during the process of receiving the user's voice input; The acquisition unit is configured to acquire a first response text corresponding to a first speech recognition result when the pause duration of the speech input is greater than or equal to the first duration, wherein the first speech recognition result is the speech recognition result obtained by performing speech recognition on the speech input when the pause duration of the speech input is equal to the first duration; The detection unit is further configured to detect whether the pause duration of the voice input is greater than or equal to a second duration when the pause duration of the voice input is greater than or equal to the first duration, wherein the second duration is greater than the first duration; The output unit is configured to generate a voice output corresponding to the voice input based on the first reply text when the pause duration of the voice input is greater than or equal to the second duration.

9. An electronic device, comprising: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to cause the electronic device to implement the voice interaction method according to any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium storing a computer program that, when executed by a computing device, causes the computing device to implement the voice interaction method according to any one of claims 1-7.

Images