Voice interaction method, device, electronic device and storage medium

Abstract

The present disclosure provides a method, device, electronic device and readable storage medium for voice interaction, which relates to the field of computer technology, and more specifically to the field of artificial intelligence technology such as voice technology and natural language processing. A specific implementation scheme is as follows: performing voice recognition on a request statement, and obtaining at least one intermediate recognition result within a first preset time after receiving the request statement; wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement; in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, calling a dialogue service to obtain a first reply statement based on the first intermediate recognition result, and setting the first intermediate recognition result to a locked state, wherein the locked state is used to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the first intermediate recognition result.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to artificial intelligence technologies such as speech technology and natural language processing, and especially to methods, devices, electronic devices, and storage media for speech interaction. Background Art

[0002] With the advancement of artificial intelligence technology, human-computer voice interaction (abbreviated as: voice interaction) has also developed rapidly and been widely used. For example, it can be widely used in smart devices such as smart TVs, smart speakers, virtual reality (VR) glasses, and various voice assistant applications (APPs).

[0003] In traditional human-computer voice interaction, speech recognition and the subsequent call to the dialogue service for response are performed serially. That is, speech recognition is performed only after the end of Voice Activity Detection (VAD), and the dialogue service is called based on the speech recognition results. This results in long response times for voice interactions, impacting the user experience. To address this issue, existing technologies have proposed a solution for pre-fetching dialogue resources in a streaming manner. By performing streaming speech recognition in advance, the VAD process and the subsequent call to the dialogue service for response are parallelized, thereby reducing the response time of voice interactions. Summary of the Invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for voice interaction.

[0005] According to one aspect of the present disclosure, a method for voice interaction is provided, comprising:

[0006] Performing voice recognition on the request statement, and obtaining at least one intermediate recognition result within a first preset time after receiving the request statement; wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement;

[0007] In response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, based on the first intermediate recognition result, the dialogue service is called to obtain a first reply statement, and the first intermediate recognition result is set to a locked state, wherein the locked state is used to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the first intermediate recognition result, and the time when the other recognition results are obtained is later than the time when the first intermediate recognition result is obtained.

[0008] According to another aspect of the present disclosure, a voice interaction apparatus is provided, comprising:

[0009] a speech recognition unit configured to perform speech recognition on the request statement and obtain at least one intermediate recognition result within a first preset time after receiving the request statement; wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement;

[0010] An acquisition unit is used to, in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, call a dialogue service to obtain a first reply statement based on the first intermediate recognition result, and set the first intermediate recognition result to a locked state, wherein the locked state is used to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the first intermediate recognition result, and the time when the other recognition results are obtained is later than the time when the first intermediate recognition result is obtained.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0015] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0016] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method of the aspect and any possible implementation manner described above.

[0017] It can be seen from the above technical solution that the embodiment of the present disclosure obtains at least one intermediate recognition result within a first preset time after receiving the request statement by performing voice recognition on the request statement, wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement, and in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, that is, based on the first intermediate recognition result, calling the dialogue service to obtain the first reply statement, can not only reduce the response time of the voice interaction by pulling the dialogue resources in advance, but also avoid calling the dialogue service in large quantities to perform dialogue model calculation and resource pulling based on the semantically incomplete intermediate recognition results, which can greatly reduce invalid calls to the dialogue service and reduce the number of requests for the dialogue service, thereby saving the computing resources and storage resources of the dialogue service, as well as the charged resource services, thereby reducing costs.

[0018] In addition, based on the first intermediate recognition result, when the dialogue service is called to obtain the first reply statement, the first intermediate recognition result is set to a locked state, thereby preventing subsequent calls to the dialogue service based on other recognition results that are the same as the first intermediate recognition result. This can avoid the situation where the previous recognition result takes a long time to call the dialogue service, resulting in the subsequent recognition result continuing to call the dialogue service, and the resulting failure of the streaming early resource pulling solution. Instead, the dialogue result of the previous recognition result is waited for, thereby reducing the response time of the voice interaction and the repeated calls to the dialogue service, reducing the number of requests for the dialogue service, saving the computing resources and storage resources of the dialogue service, and charging resource services, thereby reducing costs.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of an application embodiment of the solution of pulling conversation resources in advance according to an embodiment of the present disclosure;

[0025] Figure 5yes Figure 4 A schematic diagram of time nodes of the illustrated embodiment;

[0026] Figure 6 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0028] Figure 8 The present invention is a block diagram of an electronic device used to implement the voice interaction method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0031] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers, vehicle-mounted computing devices and other smart devices; display devices may include but are not limited to personal computers, televisions, vehicle-coupled displays and other devices with display functions.

[0032] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0033] The existing streaming pre-pull dialogue resource solution can significantly reduce the response time of voice interaction. However, during the streaming voice recognition process, it is not known when the user will stop speaking. When the voice recognition result changes, the dialogue service must be continuously called to obtain the corresponding result and cache it. This leads to an increase in the number of dialogue service requests and a large waste of dialogue service computing resources. At the same time, for the same recognition result, if the previous recognition result takes a long time to call the dialogue service, when the next recognition result is obtained, the result returned for the previous recognition result is not obtained. At this time, the next recognition result will continue to call the dialogue service, causing the streaming pre-pull resource solution to fail and resulting in an increase in the number of dialogue service requests. This is especially true for some situations where the application programming interface (API) of a paid resource service needs to be called. For example, when the dialogue service needs to call the interface of a weather service provider or audio content provider to obtain response resources, it needs to pay on a per-time basis. The increase in the number of dialogue service requests will further increase economic costs.

[0034] For example, for the user's request sentence "I want to listen to Daoxiang", the conversation service will receive the following in sequence:

[0035] A. I

[0036] B. I think

[0037] C. I want to listen

[0038] D. I want to listen to the rice

[0039] E. I want to listen to the rice fragrance (the intermediate recognition result obtained before the end of VAD, corresponding to Figure 4 Moment 1.5)

[0040] F. I want to listen to the final recognition result after the end of VAD, corresponding to Figure 4 Moment 3)

[0041] Among them, since each intermediate recognition result A, B, C, D, and E is different from the previous one, it is necessary to call the dialogue service to obtain the corresponding result and cache it, which leads to an increase in the number of requests for the dialogue service. The number of requests for the dialogue service is about n-1 times that of the traditional solution, where n is the average length of all intermediate recognition results and the final recognition result.

[0042] Compared with traditional human-computer voice interaction solutions, this streaming solution of pre-pulling dialogue resources can directly return the dialogue result of the intermediate recognition result E when the final recognition result F is received to reduce the response time of voice interaction. However, the previous intermediate recognition results A, B, C, and D request results for the dialogue service will not be adopted, and the dialogue service still needs to calculate the dialogue model and obtain resources, which will cause a large amount of dialogue service computing resources to be wasted, especially for some situations where paid resource service APIs need to be called, such as calling the interfaces of weather service providers and audio content providers to obtain response resources, which are charged on a per-time basis, which will further increase the corresponding economic costs.

[0043] In addition, for the same recognition results E and F, if the intermediate recognition result E takes a long time to call the dialogue service, when the final recognition result F is obtained, the result returned for the intermediate recognition result E is not obtained. At this time, the dialogue service will continue to be called based on the final recognition result F, which will cause the streaming early resource pulling solution to fail, and will also cause an increase in the number of dialogue service requests, which will further increase the economic cost.

[0044] Therefore, there is an urgent need to provide a voice interaction processing method to reduce invalid calls to dialogue services while reducing the response time of voice interaction, thereby reducing costs.

[0045] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure, as shown in Figure 1 shown.

[0046] 101. Perform voice recognition on a request statement, and obtain at least one intermediate recognition result within a first preset time after receiving the request statement.

[0047] Among them, the request statement is a statement sent by the user that he wants to get a reply, for example, it can be words of different needs such as "play children's songs" and "I want to listen to rice fragrance", and the present embodiment does not limit this.

[0048] Among them, the end moment of the first preset time is earlier than the end point moment of the voice activity detection (VAD end) of the request sentence. The first preset time refers to a period of time that starts counting after the tail sound of the request sentence sent by the user falls, that is, the first preset time is the duration of the silent state detected by the voice interaction device. The length of the first preset time can be set as needed. In actual use, in order to reduce the user's waiting time, the length of the first preset time can be set to 5ms, 10ms, 20ms, etc. This is just an example and cannot be used as a limitation on the length of the first preset time in this disclosure.

[0049] The intermediate recognition result is the result obtained by performing voice recognition on the request sentence sent by the user within a first preset time after the last tone of the request sentence falls.

[0050] 102. In response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, calling a dialogue service to obtain a first reply statement based on the first intermediate recognition result, and setting the first intermediate recognition result to a locked state.

[0051] The locked state is used to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the first intermediate recognition result, and the time when the other recognition results are obtained is later than the time when the first intermediate recognition result is obtained.

[0052] The semantic integrity of the intermediate recognition result means that the intermediate recognition result has complete semantics and expresses a complete meaning.

[0053] Among them, the first reply statement is to call the dialogue service based on the semantically complete first intermediate recognition result. The dialogue service determines the user's intention based on the first intermediate recognition result and the dialogue state, calculates the dialogue model and obtains resources, and generates a reply statement to be broadcast to the user.

[0054] It should be noted that part or all of the execution entities of 101 to 102 may be applications located in the local terminal, i.e., the terminal device of the service provider, or may also be functional units such as plug-ins or software development kits (SDKs) set in the applications located in the local terminal, or may also be processing engines located in the network-side server, or may also be distributed systems located on the network side. This embodiment does not specifically limit this.

[0055] It is understandable that the application may be a native program (nativeApp) installed on the terminal, or may be a webpage program (webApp) of a browser on the terminal, and this embodiment does not specifically limit this.

[0056] In this way, in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, i.e., calling the dialogue service to obtain the first reply statement based on the first intermediate recognition result, the response time of the voice interaction can be reduced by pre-pulling dialogue resources, while also avoiding the need to call the dialogue service extensively to perform dialogue model calculations and resource retrieval based on semantically incomplete intermediate recognition results. This can significantly reduce invalid calls to the dialogue service and the number of requests to the dialogue service, thereby conserving computing and storage resources for the dialogue service, as well as paid resource services, and thus reducing costs. Furthermore, when calling the dialogue service to obtain the first reply statement based on the first intermediate recognition result, the first intermediate recognition result is set to a locked state, thereby preventing subsequent calls to the dialogue service based on other recognition results identical to the first intermediate recognition result. This can avoid the situation where the dialogue service is called for the same recognition result due to the previous recognition result taking a long time to call the dialogue service, resulting in the subsequent recognition result being ineffective, and the resulting streaming pre-pulling of resources being ineffective. Instead, the dialogue service waits for the dialogue result of the previous recognition result, thereby reducing the response time of the voice interaction and repeated calls to the dialogue service, reducing the number of requests to the dialogue service, conserving computing and storage resources for the dialogue service, as well as paid resource services, and thus reducing costs.

[0057] For example, in conjunction with the above-mentioned user's request sentence "I want to listen to the rice fragrance", the semantics of the intermediate recognition results A, B, C, and D are all incomplete, that is, they are all incomplete requests. Based on this embodiment, the dialogue service is no longer called, thereby reducing the number of requests for the dialogue service from the intermediate recognition results. Intermediate recognition results E and F are both semantically complete, that is, they are all complete requests, and the recognition results of the two are the same. Intermediate recognition result E is the first intermediate recognition result with complete semantics. According to the intermediate recognition result E, the dialogue service is called to obtain the first reply sentence, and the intermediate recognition result E is set to a locked state, thereby preventing the final recognition result F from calling the dialogue service. This can avoid the final recognition result F from continuing to call the dialogue service due to the long time it takes for the intermediate recognition result E to call the dialogue service, and the resulting failure of the streaming early resource pulling solution. Instead, the dialogue result of the intermediate recognition result E is waited for, thereby reducing the response time of the voice interaction and the repeated calls to the dialogue service, reducing the number of requests for the dialogue service, saving the computing resources and storage resources of the dialogue service, and charging resource services, thereby reducing costs.

[0058] Optionally, in a possible implementation of this embodiment, the other recognition results in 102 may include: the intermediate recognition results other than the first intermediate recognition result in the at least one intermediate recognition result, and the final recognition result corresponding to the request statement obtained within the second preset time after the request statement is obtained. The end time of the second preset time is later than the end time of the voice activity detection of the request statement. The second preset time is a period of time that starts counting after the tail sound of the request statement sent by the user falls, and the length of this timing time is greater than the length of the first preset time. It can be understood that the second preset time is also the duration of the silent state detected by the voice interaction device.

[0059] Based on this embodiment, the subsequent calling of the dialogue service based on other intermediate recognition results and final recognition results that are the same as the first intermediate recognition result can be prevented, and the situation in which the subsequent recognition result continues to call the dialogue service due to the long time taken to call the dialogue service for the previous recognition result for the same recognition result, as well as the resulting failure of the streaming advance resource pulling solution, can be avoided. Instead, the dialogue result of the previous recognition result is waited for, thereby reducing the response time of voice interaction and repeated calls to the dialogue service, reducing the number of requests for the dialogue service, saving the computing resources and storage resources of the dialogue service, and charging resource services, thereby reducing costs.

[0060] Optionally, in a possible implementation of this embodiment, in 101, it may also include: in response to obtaining the first reply statement, caching the first reply statement, and restoring the first intermediate recognition result from a locked state to an unlocked state, that is, unlocking the first intermediate recognition result.

[0061] Based on this embodiment, after obtaining the first reply statement, the first reply statement can be cached. Since the first reply statement has been obtained, it is possible to avoid the situation where the previous recognition result takes a long time to call the dialogue service, resulting in the subsequent recognition result continuing to call the dialogue service, and the resulting failure of the streaming early resource pulling solution. The first intermediate recognition result can be restored from a locked state to an unlocked state, and the normal voice interaction process can be restored to ensure the standardization of the voice interaction business process; in addition, the dialogue service can be called normally to obtain the reply statement for the same request statement sent by the user subsequently, avoiding the situation where the same recognition result is blocked from calling the dialogue service because the intermediate recognition result is in a locked state.

[0062] Optionally, in a possible implementation of this embodiment, after 101, it may also include: in response to not obtaining the first reply statement within a third preset time after calling the dialogue service to obtain the first reply statement, restoring the first intermediate recognition result from a locked state to an unlocked state, so as to call the dialogue service based on other recognition results that are the same as the first intermediate recognition result.

[0063] Based on this embodiment, if the first reply statement is not obtained within the third preset time after calling the dialogue service to obtain the first reply statement, the first intermediate recognition result will be restored from the locked state to the unlocked state. In this way, the dialogue service can be called based on other recognition results that are the same as the first intermediate recognition result to obtain the reply result, thereby improving the reliability of voice interaction and avoiding the inability to obtain the dialogue result due to transmission link failure, which affects the user experience; in addition, the dialogue service can be called normally to obtain the reply statement for the same request statement sent by the user subsequently, thereby avoiding preventing the same recognition result from calling the dialogue service because the intermediate recognition result is in a locked state.

[0064] Figure 2 Schematic diagram of the second embodiment of the present disclosure. Figure 2 As shown, in Figure 1 Based on the embodiment shown, after step 102, the following steps may also be included:

[0065] 201. In response to identifying a semantically complete second intermediate recognition result from the at least one intermediate recognition result, determine whether the second intermediate recognition result is the same as the first intermediate recognition result.

[0066] The second intermediate recognition result is obtained later than the first intermediate recognition result.

[0067] In response to the second intermediate recognition result being the same as the first intermediate recognition result, step 202 is executed. Otherwise, in response to the second intermediate recognition result being different from the first intermediate recognition result, that is, the second intermediate recognition result has changed relative to the first intermediate recognition result, the subsequent operations of this embodiment are not executed. Optionally, the dialogue service may be called for the second intermediate recognition result to obtain a second reply statement, such as Figure 3 shown.

[0068] 202. Identify whether the first intermediate recognition result is in a locked state.

[0069] In response to the first intermediate recognition result not being in the locked state, execute 203. Otherwise, if the first intermediate recognition result is in the locked state, wait for the first reply result to be returned without performing other operations, that is, do not call the dialogue service to obtain the second reply statement according to the second intermediate recognition result.

[0070] 203. Obtain the cached first reply statement and play it.

[0071] Optionally, in one possible implementation of this embodiment, a synchronization mechanism, such as a semaphore, mutex, or conditional variable in multi-threaded or multi-process programming, can be used to set an intermediate recognition result to a locked state to wait for the return of a corresponding response result. The disclosed embodiment does not limit the setting of the locked state and the corresponding unlocking method.

[0072] Based on this embodiment, when a semantically complete second intermediate recognition result is identified from the at least one intermediate recognition result, it is first determined whether the second intermediate recognition result is the same as the first intermediate recognition result. If the second intermediate recognition result is the same as the first intermediate recognition result, it is determined whether the first intermediate recognition result is in a locked state. If the first intermediate recognition result is not in a locked state, it means that the first reply statement has been obtained. At this time, the cached first reply statement can be directly obtained and played, which can reduce the response time of voice interaction and repeated calls to the dialogue service, reduce repeated calls to the dialogue service for the same recognition result, save computing resources and storage resources of the dialogue service, and charged resource services, thereby reducing costs.

[0073] Figure 3 Schematic diagram of the third embodiment of the present disclosure. Figure 3 As shown, in Figure 2 Based on the embodiment shown, after step 201, in response to the second intermediate recognition result being different from the first intermediate recognition result, the following steps may also be included:

[0074] 301. Based on the second intermediate recognition result, call the dialogue service to obtain a second reply statement, and set the second intermediate recognition result to a locked state.

[0075] The locked state is used to indicate that calling of the dialogue service based on other recognition results that are the same as the second intermediate recognition result is prohibited, and the time when the other recognition results are obtained is later than the time when the second intermediate recognition result is obtained.

[0076] 302. In response to obtaining the second reply statement, cache the second reply statement, and restore the second intermediate recognition result from a locked state to an unlocked state.

[0077] 303. In response to identifying a semantically complete third intermediate recognition result from the at least one intermediate recognition result, determine whether the third intermediate recognition result is the same as the second intermediate recognition result.

[0078] The third intermediate recognition result is obtained later than the second intermediate recognition result.

[0079] In response to the third intermediate recognition result being the same as the second intermediate recognition result, execution proceeds to 304. Otherwise, in response to the third intermediate recognition result being different from the second intermediate recognition result, i.e., the third intermediate recognition result has changed relative to the second intermediate recognition result, subsequent operations of this embodiment are not performed. Optionally, a dialogue service may be invoked for the third intermediate recognition result to obtain a third reply statement. The specific implementation process for invoking the dialogue service for the third intermediate recognition result to obtain the third reply statement can be referenced to the specific implementation process for invoking the dialogue service for the second intermediate recognition result to obtain the second reply statement of this embodiment, which will not be further described here.

[0080] 304. Identify whether the second intermediate identification result is in a locked state.

[0081] In response to the second intermediate recognition result not being in the locked state, execute 305. Otherwise, if the second intermediate recognition result is in the locked state, continue to wait for the second reply result to be returned without performing other operations, that is, do not call the dialogue service to obtain the third reply statement according to the third intermediate recognition result.

[0082] 305. Obtain the cached second reply statement and play it.

[0083] Based on this embodiment, if the second intermediate recognition result is different from the first intermediate recognition result, it means that the content of the user's request statement has not been fully expressed, and the corresponding reply statements of the two are different. At this time, the first reply statement will no longer be used as the dialogue result for replying to the user. Instead, the dialogue service is called to obtain the second reply statement, and then it is determined whether the third intermediate recognition result is the same as the second intermediate recognition result to determine whether to use the second reply statement as the dialogue result for replying to the user. If the third intermediate recognition result is the same as the second intermediate recognition result, it means that the content of the user's request statement has been fully expressed, and the corresponding reply statements of the two are the same. Wait for the second reply statement to be played as the dialogue result for replying to the user. There is no need to call the dialogue service for subsequent recognition results (including intermediate recognition results and final recognition results). The response time of voice interaction can be reduced by pulling dialogue resources in advance, and repeated calls to the dialogue service can be reduced, thereby reducing the number of requests for the dialogue service, saving computing resources and storage resources of the dialogue service, as well as charged resource services, thereby reducing costs. If it is determined that the third intermediate recognition result is different from the second intermediate recognition result, it means that the content of the user's request statement has not been fully expressed, and the corresponding reply statements are different. Continue to call the dialogue service for the third intermediate recognition result to obtain the third reply statement, and so on, to determine the intermediate recognition result with complete semantics and accurate and complete user request information, so as to accurately determine the dialogue result for the user and improve the accuracy of the dialogue result.

[0084] Optionally, in a possible implementation of this embodiment, in 201, whether the second intermediate recognition result is the same as the first intermediate recognition result may be determined in the following manner:

[0085] Perform semantic analysis on the first intermediate recognition result to obtain a first semantic analysis result, which includes: domain, intent and slot information. The domain is the domain to which the request statement belongs, such as alarm clock, weather, music, etc. The intent is the specific intent of the request statement in the current domain. For example, in the alarm clock domain, there are intents such as setting the alarm clock and deleting the alarm clock. The slot is the specific slot information of the request statement in the current domain and intent. For example, the request statement "How is the weather in Beijing?" has the domain of weather and the intent of querying the weather. The slot information includes: slot "city" and the slot value is "Beijing"; the request statement "I want to listen to Jay Chou's songs" has the domain of music and the intent of music search. The slot information is: slot "singer" and the slot value is "Jay Chou";

[0086] Performing semantic parsing on the second intermediate recognition result to obtain a second semantic parsing result, the second semantic parsing result including: domain, intent, and slot information;

[0087] Whether the second intermediate recognition result is the same as the first intermediate recognition result is determined based on whether the second semantic parsing result is the same as the first semantic parsing result.

[0088] Specifically, when the second semantic parsing result is the same as the first semantic parsing result, it can be determined that the second intermediate recognition result is the same as the first intermediate recognition result; otherwise, when the second semantic parsing result is different from the first semantic parsing result, it can be determined that the second intermediate recognition result is different from the first intermediate recognition result.

[0089] Based on this embodiment, the intermediate recognition results can be semantically parsed to obtain the semantic parsing results of the intermediate recognition results. By comparing whether the two semantic parsing results are the same, it is determined whether the two intermediate recognition results are the same, thereby improving the objectivity and accuracy of the determination result of whether the two intermediate recognition results are the same.

[0090] Through research, the inventors of this disclosure discovered that the execution results of task-based dialogue processing (such as music on demand and setting alarm reminders) are strongly dependent on the semantic parsing results: that is, if the semantic parsing results of two request statements are the same, that is, the corresponding information in the semantic parsing results is the same, then the dialogue responses are also the same. Based on this embodiment, the semantic parsing results of the intermediate recognition result E and the final recognition result F are the same. The first reply statement corresponding to the intermediate recognition result E is used as the dialogue result, and the dialogue service is no longer called for the final recognition result F. This can not only reduce the response time of voice interaction by pulling dialogue resources in advance, but also further reduce repeated calls to the dialogue service, thereby further reducing the number of dialogue service requests, further saving the computing resources and storage resources of the dialogue service, as well as the paid resource services, thereby reducing costs.

[0091] Optionally, in a possible implementation of this embodiment, a pre-trained semantic parsing model can be used to perform semantic parsing on the intermediate recognition result to obtain a semantic parsing result of the intermediate recognition result. The semantic parsing model in the disclosed embodiment can be implemented based on a neural network model using a deep learning method, such as a deep neural network (DNN), a long short-term memory (LSTM), an LSTM+CRF model composed of an LSTM and conditional random fields (CRF), a model based on a multi-head attention mechanism (transformer), etc., and the disclosed embodiment does not impose any restrictions on this.

[0092] Based on this embodiment, the semantic parsing model pre-trained based on deep learning has a certain degree of generalization, and can accurately and comprehensively perform semantic parsing on various intermediate recognition results, and quickly and accurately obtain the semantic parsing results of each input information.

[0093] Optionally, in a possible implementation of this embodiment, the second semantic parsing result may be considered identical to the first semantic parsing result if the domain in the second semantic parsing result is the same as the domain in the first semantic parsing result, the intent in the second semantic parsing result is the same as the intent in the first semantic parsing result, and the slot information in the second semantic parsing result is the same as the slot information in the first semantic parsing result. Otherwise, if the second semantic parsing result differs from the first semantic parsing result in any one or more of the domain, intent, and slot information, the second semantic parsing result is considered different from the first semantic parsing result.

[0094] For example, combining the above-mentioned user's request statement "I want to listen to the song Daoxiang", the intermediate recognition results E and F are both semantically complete. Among them, the intermediate recognition result E is the first intermediate recognition result with complete semantics, and its semantic parsing result (i.e., the first semantic parsing result) includes: domain:music, intent:search_music, slots:{song:Daoxiang}; the intermediate recognition result F is the second intermediate recognition result with complete semantics, and its semantic parsing result (i.e., the second semantic parsing result) includes: domain:music, intent:search_music, slots:{song:Daoxiang}. After comparison, the domain, intent, and slot information of the intermediate recognition results E and F are respectively the same, and it is considered that the first semantic parsing result is the same as the second semantic parsing result. At this time, it can be determined that the second intermediate recognition result is the same as the first intermediate recognition result. If the first intermediate recognition result is in a locked state, the first reply result is waited for to be returned, without calling the dialogue service to obtain the second reply statement based on the second intermediate recognition result.

[0095] Similarly, the above method can be used to determine whether any two recognition results (including intermediate recognition results and intermediate recognition results, and intermediate recognition results and final recognition results) are the same, for example, to determine whether the third intermediate recognition result is the same as the second intermediate recognition result, whether the third intermediate recognition result is the same as the final recognition result, etc., which will not be repeated in the embodiments of the present disclosure.

[0096] Based on this embodiment, the intermediate recognition results can be semantically parsed to obtain the semantic parsing results of the intermediate recognition results. By comparing whether the domain, intention and slot information in the two semantic parsing results are the same, it is determined whether the two intermediate recognition results are the same, thereby improving the objectivity and accuracy of the comparison results on whether the two semantic parsing results are consistent, thereby improving the objectivity and accuracy of the determination results on whether the two intermediate recognition results are the same.

[0097] Optionally, in a possible implementation of this embodiment, in 101, it may also include: in response to obtaining the first intermediate recognition result of the at least one intermediate recognition result, sequentially identifying whether the semantics of the at least one intermediate recognition result are complete according to the time sequence of obtaining the intermediate recognition results.

[0098] In the embodiments of the present disclosure, multiple methods may be used to identify whether the semantics of each intermediate recognition result is complete.

[0099] For example, in a possible implementation of this embodiment, whether the semantics of each intermediate recognition result is complete may be identified in the following manner:

[0100] For each intermediate recognition result in turn, the semantic integrity model is used to obtain a first probability that each intermediate recognition result is a prefix in the historical final recognition result, and a second probability that each intermediate recognition result is a historical final recognition result. Then, based on the first probability and the second probability, a third probability that each intermediate recognition result is semantically complete is determined; and then, based on whether the third probability is greater than a preset threshold, whether the semantics of each intermediate recognition result is determined.

[0101] The semantic integrity model is calculated based on the final recognition results in user logs. For example, the total number of occurrences of the final recognition result "I want to listen to the rice fragrance" and the final recognition result "I want to set an alarm for 6:50 tomorrow morning" in large-scale online user logs are counted. The total number of occurrences of each final recognition result in large-scale online user logs is statistically calculated to obtain the semantic integrity model. The semantic integrity model can be updated based on updates to online user logs at a certain interval, such as one week. The present embodiment does not restrict whether the semantic integrity model is updated or the update interval.

[0102] In the embodiment of the present disclosure, the historical final recognition result is the final recognition result in the online large-scale user log. Each time each online user performs voice interaction and issues a request statement, at least one intermediate recognition result and one final recognition result will be obtained. The device for implementing the voice interaction of the embodiment of the present disclosure will generate a user log for each user, recording at least one intermediate recognition result and one final recognition result in each voice interaction of the user.

[0103] In the disclosed embodiment, the first probability of an intermediate recognition result being a prefix in a historical final recognition result is the probability that the intermediate recognition result appears only as the front part of the historical final recognition result, rather than as the complete historical final recognition result. The second probability of an intermediate recognition result being a historical final recognition result is the probability that the intermediate recognition result appears alone as the historical final recognition result (i.e., as a complete historical final recognition result).

[0104] In the embodiment of the present disclosure, the higher the first probability and the lower the second probability, the lower the third probability that the intermediate recognition result is semantically complete. For example, for the intermediate result "I want to listen", in the user's expression, it rarely appears as a request sentence alone, and almost always appears as a prefix of a request sentence, so the third probability of the intermediate result "I want to listen" is relatively low, so the intermediate result "I want to listen" is judged to be semantically incomplete. In the embodiment of the present disclosure, when the third probability is greater than a preset threshold, it can be determined that the semantics of the intermediate recognition result are complete, the meaning expressed is relatively complete, and it is a complete sentence, so that the intermediate recognition result can be used as a complete sentence, and then the first reply sentence is determined based on the intermediate recognition result; otherwise, when the third probability is less than or equal to the preset threshold, it is determined that the semantics of the intermediate recognition result are incomplete, the meaning expressed is not complete, and it may not be a complete sentence, and the intermediate recognition result is unreliable. The preset threshold can be set according to actual needs, for example, it can be set to 0.5, and can be adjusted according to actual needs.

[0105] Alternatively, in another possible implementation of this embodiment, the following method may be used to identify whether the semantics of each intermediate recognition result is complete:

[0106] For each intermediate recognition result, obtain the word vector of each intermediate recognition result. For example, the intermediate recognition result can be converted from text to word vector by using the word to vector method;

[0107] Obtaining a first probability that each intermediate recognition result serves as a prefix in a historical final recognition result, and a second probability that each intermediate recognition result serves as a historical final recognition result. For example, the semantic integrity model described above can be used to obtain a first probability that each intermediate recognition result serves as a prefix in a historical final recognition result, and a second probability that each intermediate recognition result serves as a historical final recognition result.

[0108] Obtain the popularity of each intermediate recognition result. The popularity of an intermediate recognition result refers to the usage of a request statement containing the intermediate recognition result sent by a user through the device for implementing voice interaction in the embodiment of the present disclosure within a certain time period or all past historical time. This can be obtained by statistically calculating the intermediate recognition results and final recognition results in large-scale online user logs.

[0109] Inputting the word vector, the first probability, the second probability, and the heat into a pre-trained neural network model, and outputting a fourth probability that each intermediate recognition result is semantically complete through the neural network model;

[0110] Whether the semantics of each intermediate recognition result is complete is determined based on whether the fourth probability is greater than a preset threshold.

[0111] In the embodiment of the present disclosure, when the fourth probability is greater than a preset threshold, the intermediate recognition result can be determined to be semantically complete, the meaning expressed is relatively complete, and it is a complete sentence, so that the intermediate recognition result can be used as a complete sentence, and the first reply sentence can be determined based on the intermediate recognition result; otherwise, when the fourth probability is less than or equal to the preset threshold, the intermediate recognition result is determined to be semantically incomplete, the meaning expressed is not complete, and it may not be a complete sentence, and the intermediate recognition result is unreliable. The preset threshold can be set according to actual needs, for example, it can be set to 0.5, and can be adjusted according to actual needs.

[0112] Based on this embodiment, starting from obtaining the first intermediate recognition result among the at least one intermediate recognition result, that is, in the chronological order of obtaining the intermediate recognition results, it is sequentially identified whether the semantics of the at least one intermediate recognition result are complete, so that the dialogue service is called to obtain the reply statement only based on the semantically complete intermediate recognition result. This can avoid a large number of calls to the dialogue service for dialogue model calculation and resource pulling based on semantically incomplete intermediate recognition results, can greatly reduce invalid calls to the dialogue service, reduce the number of requests for the dialogue service, thereby saving the computing resources and storage resources of the dialogue service, as well as charged resource services, thereby reducing costs.

[0113] Furthermore, in a possible implementation of this embodiment, after step 101, the following steps may be included:

[0114] Within a second preset time after receiving the request statement, a final recognition result corresponding to the request statement is obtained, wherein the end time of the second preset time is later than the end time of the voice activity detection of the request statement. The second preset time is a period of time starting after the end of the request statement sent by the user falls, and the length of this time period is longer than the length of the first preset time. It is understandable that the second preset time is also the duration of the silent state detected by the voice interaction device;

[0115] In response to the absence of a semantically complete intermediate recognition result in the at least one intermediate recognition result, calling a dialogue service to obtain a final reply sentence for playback according to the final recognition result.

[0116] Based on this embodiment, the final recognition result of the user's request statement can be obtained, so that when a semantically complete intermediate recognition result is not identified from the at least one intermediate recognition result, the final reply statement can be determined and played according to the final recognition result of the request statement to ensure the accuracy of the voice interaction result and avoid erroneous voice interaction results affecting the user experience.

[0117] Optionally, in the above embodiment of the present disclosure, if there are no two identical recognition results, the dialogue service can be called based on the final recognition result to obtain the final reply statement for playback to ensure the accuracy of the voice interaction results and avoid erroneous voice interaction results affecting the user experience.

[0118] Figure 4 This is a schematic diagram of an application embodiment of the solution of pulling conversation resources in advance according to an embodiment of the present disclosure. Figure 5 yes Figure 4 The time node diagram of the embodiment shown is as follows: Figure 4 and Figure 5 The embodiment of the present disclosure is further described by taking the specific voice interaction process of the user issuing the request sentence "I want to listen to the rice fragrance" as an example. Figure 5 In the figure, A, B, C, D, E, and F are the time nodes for obtaining the recognition results A, B, C, D, E, and F respectively. e and f are the time nodes for obtaining the dialogue results (i.e., reply sentences) after calling the dialogue service for the intermediate recognition result E and the final recognition result F respectively. h1 and h2 are the time consumptions for obtaining the dialogue results after calling the dialogue service for the intermediate recognition result E and the final recognition result F respectively.

[0119] During a complete voice interaction process, from the moment the user sends a request to the moment the voice interaction device returns a reply, the following stages are experienced:

[0120] The user sends a request sentence "I want to listen to Daoxiang", and the ending tone of the voice signal of the request sentence (that is, the moment when the user's voice falls) is 1;

[0121] After time 1, the device enters VAD detection, that is, it continuously detects silence for the second preset time t1 until it reaches time 2, the VAD end point, at which point the voice interaction device will consider that the user has finished speaking.

[0122] After the first preset time has passed from time 1 to time 1.5 (corresponding to time E), at least one intermediate recognition result is obtained, including: A. I; B. I want; C. I want to listen; D. I want to listen to rice; E. I want to listen to the smell of rice;

[0123] Identify in sequence whether the semantics of the intermediate recognition results A, B, C, D, and E are complete, and identify that the semantics of the intermediate recognition results A, B, C, and D are incomplete, while the semantics of the intermediate recognition result E is complete;

[0124] At time E, the dialog service is called based on the intermediate recognition result E to calculate the dialog model and obtain resources. The intermediate recognition result E is also locked. After time t3 from time 1.5, the first reply text, i.e., the text-to-speech (tts) text, is obtained at time 4.

[0125] The first reply text is speech synthesized. After time t4 (h1 = t3 + t4) from time 4, the first reply sentence "OK" is obtained at time 5 (corresponding to time e). The corresponding system interface is called to play the first reply sentence. After time t5 reaches time 6, the user hears the playback audio of the first reply sentence.

[0126] At the same time, from moment 1 to moment 2 (VAD end point) after the second preset time, the voice interaction device continues to detect silence, thinking that the user has finished speaking, and continues voice recognition, reaching moment 3 (corresponding to moment F), and obtains the final recognition result: F. I want to listen to the fragrance of rice.

[0127] In specific implementation, moment 6 may be earlier than moment 3, or it may be later than moment 3 due to the relatively short second preset time. However, since the intermediate recognition result E is obtained earlier than the final recognition result F, the dialogue resources will be pulled in advance, thereby reducing the response time of the voice interaction, so that the user can hear the audio of the reply sentence earlier, thereby improving the user experience; at the same time, calling the dialogue service only for the semantically complete intermediate recognition results can avoid a large number of calls to the dialogue service based on the semantically incomplete intermediate recognition results, and repeated calls to the dialogue service for multiple semantically complete intermediate recognition results to perform dialogue model calculations and resource pulling, which can greatly reduce invalid calls to the dialogue service and reduce the number of requests to the dialogue service, thereby saving calls to the dialogue service. In addition, the conversation service is called according to the intermediate recognition result E to obtain the first reply statement, and the intermediate recognition result E is set to a locked state, so as to prevent the final recognition result F from calling the conversation service, and avoid the final recognition result F continuing to call the conversation service due to the long time taken by the intermediate recognition result E to call the conversation service, and the resulting failure of the streaming advance resource pulling solution. Instead, the conversation result of the intermediate recognition result E is waited for, thereby reducing the response time of the voice interaction and the repeated calls to the conversation service, reducing the request amount of the conversation service, saving the computing resources and storage resources of the conversation service, and charging resource services, thereby reducing costs.

[0128] In this embodiment, in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, that is, based on the first intermediate recognition result, calling the dialogue service to obtain the first reply statement, can not only reduce the response time of the voice interaction by pulling the dialogue resources in advance, but also avoid calling the dialogue service in large quantities based on the semantically incomplete intermediate recognition results to perform dialogue model calculation and resource pulling, which can greatly reduce invalid calls to the dialogue service and reduce the number of requests for the dialogue service, thereby saving the computing resources and storage resources of the dialogue service, as well as charged resource services, thereby reducing costs.

[0129] In addition, based on the first intermediate recognition result, when the dialogue service is called to obtain the first reply statement, the first intermediate recognition result is set to a locked state, thereby preventing subsequent calls to the dialogue service based on other recognition results that are the same as the first intermediate recognition result. This can avoid the situation where the previous recognition result takes a long time to call the dialogue service, resulting in the subsequent recognition result continuing to call the dialogue service, and the resulting failure of the streaming early resource pulling solution. Instead, the dialogue result of the previous recognition result is waited for, thereby reducing the response time of the voice interaction and the repeated calls to the dialogue service, reducing the number of requests for the dialogue service, saving the computing resources and storage resources of the dialogue service, and charging resource services, thereby reducing costs.

[0130] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0131] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Figure 6 is a schematic diagram according to a fourth embodiment of the present disclosure, as shown in Figure 6As shown. The voice interaction device 600 of this embodiment may include a voice recognition unit 601 and an acquisition unit 602. The voice recognition unit 601 is used to perform voice recognition on a request statement, and obtain at least one intermediate recognition result within a first preset time after receiving the request statement; wherein the end moment of the first preset time is earlier than the end moment of the voice activity detection of the request statement; the acquisition unit 602 is used to respond to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, call a dialogue service to obtain a first reply statement according to the first intermediate recognition result, and set the first intermediate recognition result to a locked state, wherein the locked state is used to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the first intermediate recognition result, and the time when the other recognition results are obtained is later than the time when the first intermediate recognition result is obtained.

[0133] It should be noted that part or all of the voice interaction device 600 of this embodiment can be an application located in the local terminal, i.e., the terminal equipment of the service provider, or it can also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or it can also be a processing engine located in the network side server, or it can also be a distributed system located on the network side. This embodiment does not specifically limit this.

[0134] It is understandable that the application may be a native program (nativeApp) installed on the terminal, or may be a webpage program (webApp) of a browser on the terminal, and this embodiment does not specifically limit this.

[0135] Optionally, in a possible implementation of this embodiment, the other recognition results may include: an intermediate recognition result among the at least one intermediate recognition result, and a final recognition result corresponding to the request statement obtained within a second preset time after obtaining the request statement.

[0136] Figure 7 is a schematic diagram according to a fifth embodiment of the present disclosure, as shown in Figure 7 As shown. Figure 6 Based on the shown embodiment, the voice interaction device 700 of this embodiment may also include: a cache unit 701, which is used to cache the first reply statement in response to obtaining the first reply statement, and restore the first intermediate recognition result from a locked state to an unlocked state.

[0137] Also, see Figure 7The voice interaction device 700 of this embodiment may also include: a setting unit 702, which is used to restore the first intermediate recognition result from a locked state to an unlocked state in response to not obtaining the first reply statement within a third preset time after the dialogue service is called to obtain the first reply statement, so as to call the dialogue service based on other recognition results that are the same as the first intermediate recognition result.

[0138] Also, see Figure 7 The voice interaction device 700 of this embodiment may further include: a determination unit 703, an identification unit 704, and a playback unit 705. The determination unit 703 is configured to, in response to identifying a semantically complete second intermediate recognition result from the at least one intermediate recognition result, determine whether the second intermediate recognition result is identical to the first intermediate recognition result; wherein the second intermediate recognition result is obtained later than the first intermediate recognition result; the identification unit 704 is configured to, in response to the second intermediate recognition result being identical to the first intermediate recognition result, identify whether the first intermediate recognition result is in a locked state; and the playback unit 705 is configured to, in response to the first intermediate recognition result not being in a locked state, obtain and play the cached first reply statement.

[0139] Optionally, in a possible implementation of this embodiment, the acquisition unit 602 is further configured to, in response to the second intermediate recognition result being different from the first intermediate recognition result, call the dialogue service to obtain a second reply statement based on the second intermediate recognition result, and set the second intermediate recognition result to a locked state, where the locked state is used to indicate that the dialogue service is blocked from being called based on other recognition results that are the same as the second intermediate recognition result, where the time when the other recognition results are obtained is later than the time when the second intermediate recognition result is obtained; the caching unit 701 is further configured to, in response to obtaining the second reply statement, cache the second reply statement, and restore the second intermediate recognition result from a locked state to an unlocked state; the determination unit 703 is further configured to, in response to identifying a semantically complete third intermediate recognition result from the at least one intermediate recognition result, determine whether the third intermediate recognition result is the same as the second intermediate recognition result, wherein the time when the third intermediate recognition result is obtained is later than the time when the second intermediate recognition result is obtained; the identification unit 704 is further configured to, in response to the third intermediate recognition result being the same as the second intermediate recognition result, identify whether the second intermediate recognition result is in a locked state; and the playback unit 705 is further configured to, in response to the second intermediate recognition result not being in a locked state, obtain and play the cached second reply statement.

[0140] Optionally, in a possible implementation of this embodiment, the determination unit 703 is specifically used to: perform semantic parsing on the first intermediate recognition result to obtain a first semantic parsing result, the first semantic parsing result including: domain, intent and slot information; perform semantic parsing on the second intermediate recognition result to obtain a second semantic parsing result, the second semantic parsing result including: domain, intent and slot information; and determine whether the second intermediate recognition result is the same as the first intermediate recognition result based on whether the second semantic parsing result is the same as the first semantic parsing result.

[0141] Optionally, in a possible implementation of this embodiment, the second semantic parsing result is the same as the first semantic parsing result, which may include: the domain in the second semantic parsing result is the same as the domain in the first semantic parsing result, the intention in the second semantic parsing result is the same as the intention in the first semantic parsing result, and the slot information in the second semantic parsing result is the same as the slot information in the first semantic parsing result.

[0142] Also, see Figure 7 The voice interaction device 700 of this embodiment may also include: a semantic complete recognition unit 706, which is used to respond to obtaining the first intermediate recognition result of the at least one intermediate recognition result and identify whether the semantics of the at least one intermediate recognition result are complete in sequence according to the time sequence of obtaining the intermediate recognition results.

[0143] Optionally, in one possible implementation of this embodiment, the speech recognition unit 601 is further configured to obtain a final recognition result corresponding to the request statement within a second preset time after receiving the request statement, wherein the end time of the second preset time is later than the end time of the voice activity detection of the request statement. Accordingly, in this embodiment, the acquisition unit 602 is further configured to, in response to the absence of a semantically complete intermediate recognition result in the at least one intermediate recognition result, call the dialogue service to obtain a second reply statement for playback based on the final recognition result.

[0144] In this embodiment, in response to identifying a semantically complete first intermediate recognition result from the at least one intermediate recognition result, that is, based on the first intermediate recognition result, calling the dialogue service to obtain the first reply statement, can not only reduce the response time of the voice interaction by pulling the dialogue resources in advance, but also avoid calling the dialogue service in large quantities based on the semantically incomplete intermediate recognition results to perform dialogue model calculation and resource pulling, which can greatly reduce invalid calls to the dialogue service and reduce the number of requests for the dialogue service, thereby saving the computing resources and storage resources of the dialogue service, as well as charged resource services, thereby reducing costs.

[0145] In addition, based on the first intermediate recognition result, when the dialogue service is called to obtain the first reply statement, the first intermediate recognition result is set to a locked state, thereby preventing subsequent calls to the dialogue service based on other recognition results that are the same as the first intermediate recognition result. This can avoid the situation where the previous recognition result takes a long time to call the dialogue service, resulting in the subsequent recognition result continuing to call the dialogue service, and the resulting failure of the streaming early resource pulling solution. Instead, the dialogue result of the previous recognition result is waited for, thereby reducing the response time of the voice interaction and the repeated calls to the dialogue service, reducing the number of requests for the dialogue service, saving the computing resources and storage resources of the dialogue service, and charging resource services, thereby reducing costs.

[0146] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product. Furthermore, an artificial intelligence device including the provided electronic device is also provided.

[0147] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0148] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0149] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the method of voice interaction. For example, in some embodiments, the method of voice interaction can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method of voice interaction described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the method of voice interaction by any other appropriate means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0157] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0158] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A voice interaction method, comprising: Performing voice recognition on the request statement, and obtaining a plurality of intermediate recognition results within a first preset time after receiving the request statement; wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement; In response to obtaining a first intermediate recognition result among the plurality of intermediate recognition results, sequentially determining whether the plurality of intermediate recognition results are semantically complete based on a first probability of each intermediate recognition result being a prefix of a historical final recognition result and a second probability of each intermediate recognition result being a historical final recognition result in a chronological order of obtaining the intermediate recognition results; In response to identifying a semantically complete first intermediate recognition result from the multiple intermediate recognition results, calling a dialogue service to obtain a first reply statement based on the first intermediate recognition result, and setting the first intermediate recognition result to a locked state, wherein the locked state is used to indicate that calling the dialogue service based on other recognition results that are the same as the first intermediate recognition result is prohibited, where the other recognition results are obtained later than the first intermediate recognition result; After setting the first intermediate recognition result to a locked state, the method further includes: In response to obtaining the first reply statement, the first reply statement is cached, and the first intermediate recognition result is restored from a locked state to an unlocked state.

2. The method according to claim 1, wherein The other recognition results include: an intermediate recognition result among the multiple intermediate recognition results, and a final recognition result corresponding to the request statement obtained within a second preset time after the request statement is obtained.

3. The method according to claim 1, wherein After setting the first intermediate recognition result to a locked state, the method further includes: In response to not obtaining the first reply statement within a third preset time after the dialogue service is called to obtain the first reply statement, the first intermediate recognition result is restored from a locked state to an unlocked state so as to call the dialogue service based on other recognition results that are the same as the first intermediate recognition result.

4. The method according to any one of claims 1 to 3, wherein After setting the first intermediate recognition result to a locked state, the method further includes: In response to identifying a semantically complete second intermediate recognition result from the plurality of intermediate recognition results, determining whether the second intermediate recognition result is identical to the first intermediate recognition result; wherein the second intermediate recognition result is obtained later than the first intermediate recognition result; In response to the second intermediate identification result being the same as the first intermediate identification result, identifying whether the first intermediate identification result is in a locked state; In response to the first intermediate recognition result not being in a locked state, the cached first reply statement is obtained and played.

5. The method according to claim 4, further comprising: In response to the second intermediate recognition result being different from the first intermediate recognition result, invoking a dialogue service to obtain a second reply statement based on the second intermediate recognition result, and setting the second intermediate recognition result to a locked state, the locked state being used to indicate that invoking the dialogue service based on remaining recognition results that are the same as the second intermediate recognition result is prohibited, the remaining recognition results being obtained later than the second intermediate recognition result; In response to obtaining the second reply statement, caching the second reply statement and restoring the second intermediate recognition result from a locked state to an unlocked state; In response to identifying a semantically complete third intermediate recognition result from the plurality of intermediate recognition results, determining whether the third intermediate recognition result is identical to the second intermediate recognition result; wherein the third intermediate recognition result is obtained later than the second intermediate recognition result; In response to the third intermediate identification result being the same as the second intermediate identification result, identifying whether the second intermediate identification result is in a locked state; In response to the second intermediate recognition result not being in a locked state, the cached second reply statement is obtained and played.

6. The method according to claim 4, wherein: The determining whether the second intermediate recognition result is the same as the first intermediate recognition result includes: Performing semantic parsing on the first intermediate recognition result to obtain a first semantic parsing result, where the first semantic parsing result includes: domain, intent, and slot information; Performing semantic parsing on the second intermediate recognition result to obtain a second semantic parsing result, where the second semantic parsing result includes: domain, intent, and slot information; Whether the second intermediate recognition result is the same as the first intermediate recognition result is determined according to whether the second semantic parsing result is the same as the first semantic parsing result.

7. The method according to claim 6, wherein: The second semantic parsing result is the same as the first semantic parsing result, including: The domain in the second semantic parsing result is the same as the domain in the first semantic parsing result, the intention in the second semantic parsing result is the same as the intention in the first semantic parsing result, and the slot information in the second semantic parsing result is the same as the slot information in the first semantic parsing result.

8. The method according to claim 1, further comprising: Obtaining a final recognition result corresponding to the request sentence within a second preset time after receiving the request sentence, wherein an end time of the second preset time is later than an end time of voice activity detection of the request sentence; In response to the absence of a semantically complete intermediate recognition result among the plurality of intermediate recognition results, calling a dialogue service to obtain a final reply sentence for playback according to the final recognition result.

9. A voice interaction device, comprising: a speech recognition unit configured to perform speech recognition on the request statement and obtain a plurality of intermediate recognition results within a first preset time after receiving the request statement; wherein the end time of the first preset time is earlier than the end time of the voice activity detection of the request statement; a semantics completeness identification unit, configured to, in response to obtaining a first intermediate recognition result among the plurality of intermediate recognition results, sequentially identify whether the plurality of intermediate recognition results are semantically complete based on a first probability that each intermediate recognition result serves as a prefix of a historical final recognition result and a second probability that each intermediate recognition result serves as a historical final recognition result in a chronological order in which the intermediate recognition results are obtained; an acquiring unit configured to, in response to identifying a semantically complete first intermediate recognition result from the plurality of intermediate recognition results, invoke a dialogue service to acquire a first reply statement based on the first intermediate recognition result, and set the first intermediate recognition result to a locked state, the locked state being configured to indicate that invocation of the dialogue service based on other recognition results that are the same as the first intermediate recognition result is prohibited, the other recognition results being acquired later than the first intermediate recognition result; A cache unit is configured to cache the first reply statement in response to obtaining the first reply statement, and restore the first intermediate recognition result from a locked state to an unlocked state.

10. The device according to claim 9, wherein The other recognition results include: an intermediate recognition result among the multiple intermediate recognition results, and a final recognition result corresponding to the request statement obtained within a second preset time after the request statement is obtained.

11. The apparatus according to claim 9, further comprising: A setting unit is configured to restore the first intermediate recognition result from a locked state to an unlocked state in response to not obtaining the first reply statement within a third preset time after the dialogue service is called to obtain the first reply statement, so as to call the dialogue service based on other recognition results that are the same as the first intermediate recognition result.

12. The apparatus according to claim 10, further comprising: a determining unit configured to, in response to identifying a semantically complete second intermediate recognition result from the plurality of intermediate recognition results, determine whether the second intermediate recognition result is identical to the first intermediate recognition result; wherein the second intermediate recognition result is obtained later than the first intermediate recognition result; an identification unit, configured to, in response to the second intermediate identification result being the same as the first intermediate identification result, identify whether the first intermediate identification result is in a locked state; A playing unit is configured to obtain and play the cached first reply statement in response to the first intermediate recognition result not being in a locked state.

13. The device according to claim 12, wherein The acquiring unit is further configured to, in response to the second intermediate recognition result being different from the first intermediate recognition result, call the dialogue service to obtain a second reply statement based on the second intermediate recognition result, and set the second intermediate recognition result to a locked state, the locked state being configured to indicate that the dialogue service is prevented from being called based on other recognition results that are the same as the second intermediate recognition result, the other recognition results being obtained later than the second intermediate recognition result; The cache unit is further configured to cache the second reply statement in response to obtaining the second reply statement, and restore the second intermediate recognition result from a locked state to an unlocked state; The determining unit is further configured to, in response to identifying a semantically complete third intermediate recognition result from the plurality of intermediate recognition results, determine whether the third intermediate recognition result is identical to the second intermediate recognition result; wherein the third intermediate recognition result is obtained later than the second intermediate recognition result; The identification unit is further configured to, in response to the third intermediate identification result being the same as the second intermediate identification result, identify whether the second intermediate identification result is in a locked state; The playback unit is further configured to, in response to the second intermediate recognition result being not in a locked state, obtain and play the cached second reply statement.

14. The device according to claim 12, wherein The determining unit is specifically configured to: Performing semantic parsing on the first intermediate recognition result to obtain a first semantic parsing result, where the first semantic parsing result includes: domain, intent, and slot information; Performing semantic parsing on the second intermediate recognition result to obtain a second semantic parsing result, where the second semantic parsing result includes: domain, intent, and slot information; Whether the second intermediate recognition result is the same as the first intermediate recognition result is determined according to whether the second semantic parsing result is the same as the first semantic parsing result.

15. The device according to claim 14, wherein The second semantic parsing result is the same as the first semantic parsing result, including: The domain in the second semantic parsing result is the same as the domain in the first semantic parsing result, the intention in the second semantic parsing result is the same as the intention in the first semantic parsing result, and the slot information in the second semantic parsing result is the same as the slot information in the first semantic parsing result.

16. The device according to claim 9, wherein The speech recognition unit is further configured to obtain a final recognition result corresponding to the request sentence within a second preset time after receiving the request sentence, wherein an end time of the second preset time is later than an end time of the voice activity detection of the request sentence; The acquisition unit is further configured to, in response to the absence of a semantically complete intermediate recognition result among the multiple intermediate recognition results, call a dialogue service to obtain a final reply statement for playback according to the final recognition result.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images