Voice dialogue interaction method, system, electronic device and storage medium

Abstract

Embodiments of the present application provide a voice dialogue interaction method, system, electronic device and storage medium. The method comprises: receiving user voice input by a client, generating a general reply text for feeding back the user voice, and a final reply text supplemented with an information keyword of the general reply text; sending at least the final reply text to a voice synthesis service, the voice synthesis service generating a synthesized sound playback address to feed back to the client, and asynchronously splitting the final reply text based on a dialogue label to obtain a first text of the final reply text that needs to call a pre-synthesized voice and remaining text that needs to be synthesized except the first text; the voice synthesis service receiving an audio stream acquisition request initiated by the client based on the synthesized sound playback address, and streaming the pre-synthesized voice and synthesized voice of the remaining text to the client. Embodiments of the present application ensure the accuracy of the dialogue system, reduce the overall time consumption of the dialogue system, and improve the real-time performance of the dialogue system.

Description

Technical Field

[0001] This invention relates to the field of intelligent voice, and more particularly to a voice dialogue interaction method, system electronic device, and storage medium. Background Technology

[0002] In the communication process between smart voice devices and users, the smart voice device client collects the user's voice, sends the voice to the cloud for recognition, and the cloud-based dialogue system synthesizes the corresponding voice dialogue and then feeds it back to the client for playback. The synthesis of the voice dialogue typically takes hundreds of milliseconds to several seconds, depending on the length of the synthesized content. To avoid the synthesized voice blocking the voice interaction during the synthesis process, the common practice is to start streaming playback when the first frame of the synthesized voice is generated. If the audio is sent after the synthesis result is received, the user will have to wait a long time to receive the response. Even if streaming playback can be started when the first frame of the synthesized voice is generated, there will still be a latency of hundreds of milliseconds from the generation of the first frame in the cloud to the client receiving it. Therefore, reducing the time spent on the first frame can further reduce the latency of the synthesized voice playback. To reduce the time spent on the first frame, the following steps are usually taken: 1. Segmentation based on sentence punctuation; 2. Segmentation based on the word segmentation results of the sentence; 3. Pre-synthesis of the dialogue script.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0004] 1. Segmenting based on sentence punctuation marks results in a low probability that the generated clauses will be identical, which means that the already synthesized cache cannot be used as a pre-synthesized resource, resulting in a relatively poor time-saving effect.

[0005] 2. Segmentation is based on the word segmentation results of the sentence. Synthesis based solely on word level is not effective because the same word may have different pronunciations and tones in different sentences (e.g., polyphonic words). Simply smoothing the sound features of the previously processed words cannot solve the pronunciation and tone problems. This can easily lead to incorrect pronunciation of the synthesized sentences and heavy mechanical sounds, which affects the user's interactive experience.

[0006] 3. Pre-synthesis of the script: If pre-synthesis is only done from the synthesis module, it can usually only achieve sentence-level pre-synthesis, resulting in a relatively low probability of hitting the pre-synthesis and a relatively poor effect in reducing time consumption. Summary of the Invention

[0007] To at least address the problems of relatively poor first-frame reduction efficiency and inaccurate synthesized speech pronunciation in existing technologies, in a first aspect, embodiments of the present invention provide a voice dialogue interaction method, comprising:

[0008] The system receives user voice input from a client, generates a general response text for responding to the user voice, and a final response text that supplements the general response text with keywords. The final response text is labeled with dialogue tags, which are used to mark the general response text in the final response text.

[0009] The final response text is sent to the speech synthesis service at least once. The speech synthesis service generates a synthesized audio playback address and sends it back to the client. It also asynchronously segments the final response text based on the dialogue tags to obtain the first sentence text of the sentence that needs to call the pre-synthesized audio and the remaining text that needs to be synthesized after removing the first sentence text.

[0010] The speech synthesis service receives an audio stream acquisition request initiated by the client based on the synthesized speech playback address, and sends the pre-synthesized speech and the synthesized speech of the remaining text to the client in a streaming manner for voice dialogue interaction feedback.

[0011] Secondly, embodiments of the present invention provide a voice dialogue interaction system, including:

[0012] The reply text program module is used to receive user voice input from the client, generate a general reply text for responding to the user voice, and a final reply text that supplements the general reply text with information keywords. The final reply text has dialogue tags, which are used to mark the general reply text in the final reply text.

[0013] The segmentation module is used to send the final response text to the speech synthesis service, the speech synthesis service generates a synthesized audio playback address and feeds it back to the client, and asynchronously segments the final response text based on the dialogue tags to obtain the sentence beginning text of the final response text that needs to call the pre-synthesized audio and the remaining text that needs to be synthesized after removing the sentence beginning text;

[0014] An interactive program module is used by the speech synthesis service to receive an audio stream acquisition request initiated by the client based on the synthesized sound playback address, and to send the pre-synthesized speech and the synthesized speech of the remaining text to the client in a streaming manner for voice dialogue interaction feedback.

[0015] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the voice dialogue interaction method of any embodiment of the present invention.

[0016] Fourthly, embodiments of the present invention provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the voice dialogue interaction method of any embodiment of the present invention.

[0017] The beneficial effects of this invention are as follows: While maintaining the accuracy of the dialogue system (ensuring the accuracy of the dialogue result), it reduces the overall time consumption of the dialogue system and improves its real-time performance. Higher real-time performance of the dialogue interaction results in shorter response times for users interacting with the device, which not only enhances the competitiveness of voice interaction products but also reduces the amount of text content that needs to be synthesized via TTS, thereby increasing the concurrency of TTS services and improving economic efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a voice dialogue interaction method provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of synthesized audio stream invocation for a voice dialogue interaction method provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a synthesis service for a voice dialogue interaction method provided in an embodiment of the present invention;

[0022] Figure 4 This is a product launch illustration of a voice dialogue interaction method provided in an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of the structure of a voice dialogue interaction system provided in an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of an embodiment of an electronic device for voice dialogue interaction provided by an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 The diagram shown is a flowchart of a voice dialogue interaction method provided by an embodiment of the present invention, which includes the following steps:

[0027] S11: Receive user voice input from the client, generate a general response text for responding to the user voice, and a final response text that supplements the general response text with information keywords, wherein the final response text has a dialogue tag, and the dialogue tag is used to mark the general response text in the final response text;

[0028] S12: At least the final reply text is sent to the speech synthesis service. The speech synthesis service generates a synthesized audio playback address and sends it back to the client. It also asynchronously segments the final reply text based on the dialogue tags to obtain the sentence beginning text that needs to call the pre-synthesized audio and the remaining text that needs to be synthesized after removing the sentence beginning text.

[0029] S13: The speech synthesis service receives the audio stream acquisition request initiated by the client based on the synthesized speech playback address, and sends the pre-synthesized speech and the synthesized speech of the remaining text to the client in a streaming manner for voice dialogue interaction feedback.

[0030] In this embodiment, considering that the synthesized audio of a voice dialogue interaction system is usually generated by the dialogue module, the dialogue module can clearly know which texts are fixed and which texts need to be dynamically changed based on the lexicon or variable content. Therefore, the fixed content in the text can be used to generate pre-synthesized audio close to the length of a sentence. This not only avoids the problem of unnatural transitions in synthesis effects based entirely on word level, but also avoids the problem of low probability of hitting the cache based entirely on sentence level.

[0031] For step S11, this method can be applied to a cloud server to perform operations such as recognizing user voice from the intelligent voice client, semantic understanding, generating response content, and synthesizing response voice.

[0032] In everyday conversations, medical triage scenarios, or other situations, users express their needs to smart voice clients (such as smart speakers, smart screens, and other smart devices). For example, in an everyday conversation, a user might say "Play 'Too Softhearted'." The smart speaker at home picks up the user's voice input and sends it to a cloud server for processing. The cloud server's dialogue module recognizes and understands the semantics, generating a general response text for the user's feedback, as well as a final response text supplemented with keywords. For instance, if the user says "Play 'Too Softhearted'," which triggers the music skill, the general response text would be "Great music, coming right away. Playing #song name# for you." In conversations between users and smart devices, it was found that under the same voice skill, regardless of what the user says, the response text can have a fixed, consistent portion of the content. For example, if a user says, "Play 'Too Softhearted'", the reply can be, "Nice music, coming right away. Playing 'Too Softhearted' for you"; if a user says, "Play 'Hair Like Snow'", the reply can be, "Nice music, coming right away. Playing 'Hair Like Snow' for you"; if a user says, "Play 'You're Ruthless'", the reply can be, "Nice music, coming right away. Playing 'You're Ruthless' for you". In these dialogues, it can be seen that "Nice music, coming right away. Playing #songname# for you" is a fixed content (where #songname# refers to a semantic slot, which can contain various song names). "#songname#" is the missing keyword in the general reply text; if it's a navigation skill, the general reply text could be "Okay, navigating you to #destination name#", where the missing keyword is "#destination name#".

[0033] Continuing with the example of music skills, the final reply text, supplemented with keywords for general reply text, is "Nice music, coming right away. Playing 'Too Softhearted' for you." The final reply text includes a dialogue tag, marking the phrase "Nice music, coming right away. Playing 'Too Softhearted'" within "||Nice music, coming right away. Playing 'Too Softhearted'||".

[0034] For step S12, since the final reply text contains dialogue tags, only the final reply text can be sent to the speech synthesis service. During asynchronous segmentation, the final reply text can be segmented according to the dialogue tags.

[0035] In one implementation, the general reply text and the final reply text are sent to a speech synthesis service. In this implementation, since the general reply text is sent to the speech synthesis service together, the speech synthesis service can segment the text based on the difference between the general reply text and the final reply text.

[0036] Considering that less content transmitted results in less bandwidth usage, taking sending only the final reply text to the speech synthesis service as an example, to further reduce the latency of synthesized audio playback, the speech synthesis service immediately generates a synthesized audio playback address and sends it back to the client upon receiving the final reply text. Furthermore, the final reply text is asynchronously segmented based on dialogue tags to obtain the initial text of the sentence requiring the pre-synthesized speech ("Nice music, coming soon. Playing for you"), and the remaining text to be synthesized after removing the initial text ("Too soft-hearted").

[0037] Specifically, asynchronous segmentation of the final response text based on the general response text or the dialogue tags includes:

[0038] The final response text is divided into multiple clauses, and it is determined whether the first clause at the beginning of the sentence has a corresponding pre-synthesized speech.

[0039] In this implementation, if the transmitted message is "|| Nice music, coming soon. Playing for you||Too soft-hearted", it can be directly segmented into "Nice music, coming soon. Playing for you" and "Too soft-hearted" based on the dialogue tags. If the transmitted message is "|| Nice music, coming soon. Playing for you||Too soft-hearted" and "Nice music, coming soon. Playing for you", it can also be segmented into "Nice music, coming soon. Playing for you" and "Too soft-hearted" by comparing the two sentences.

[0040] However, considering the need for timely and rapid feedback to the user, it's crucial to determine whether the first clause among multiple clauses has a corresponding pre-synthesized speech. (Assuming that a synthesized speech for "Nice music, coming right away. Playing it for you" has already been pre-generated based on data collected during the user conversation), it's determined that the pre-synthesized speech will be used for "Nice music, coming right away. Playing it for you" in the reply to the user; the remaining text "Too soft-hearted" will be synthesized.

[0041] For step S13, since the speech synthesis service has already generated the synthesized audio playback address and sent it back to the client in step S12, such as... Figure 2 As shown, upon receiving the synthesized audio playback address, the client immediately initiates an audio stream retrieval request to the speech synthesis service. At this time, the speech synthesis service receives the client's audio stream retrieval request. Since the final reply text includes the pre-synthesized speech "Nice music, coming soon. Playing for you," it can directly send the pre-synthesized speech streaming data and also stream the synthesized speech to the client without waiting for the first frame of the TTS synthesized audio to be synthesized. The overall process is as follows. Figure 3 As shown.

[0042] Assuming the network latency for the dialogue service to return the playback address to the client is 100 milliseconds, and generating the first frame of the synthesized audio takes 300 milliseconds, before using this method, the client's total waiting time for synthesized audio is approximately 300 milliseconds. Since the TTS address is sent via an exception before synthesis, the total latency equals the time spent generating the first frame of the synthesized audio. After using this method, if the first clause of the NLG (natural language generation) returned by the dialogue is pre-synthesized audio, then the client's waiting time—approximately 5 milliseconds for determining if the first clause of the NLG is pre-synthesized audio plus the 100 milliseconds network latency for returning the playback address to the client—is 105 milliseconds, reducing the client's waiting time by nearly 75%.

[0043] As can be seen from this implementation method, this approach reduces the overall processing time of the dialogue system and improves its real-time performance while maintaining the accuracy of the dialogue system (ensuring the accuracy of the dialogue system). Higher real-time performance of the dialogue interaction results in shorter response times for users interacting with the device, which not only enhances the competitiveness of voice interaction products but also reduces the amount of text content that needs to be synthesized via TTS, thereby increasing the concurrency of TTS services and improving economic efficiency.

[0044] In one implementation, the pre-synthesized speech is generated from a generic response text that meets pre-synthesis conditions, wherein the pre-synthesis conditions include:

[0045] If the number of characters in the general reply text reaches a preset threshold, a pre-synthesized speech of the general reply text is generated to ensure that the speech synthesis service can synthesize the first frame of audio of the remaining text before the client finishes playing the pre-synthesized speech.

[0046] In this embodiment, the method involves two stages: in addition to the segmentation and synthesis part of the above-mentioned voice dialogue interaction, there is also a pre-synthesized speech development stage.

[0047] When developers publish voice products through a voice dialogue platform, the publishing program reads the list of all voice skills in the voice product and generates dialogue resources for the voice skills. Then, based on the dialogue resources of the skills and the timbre selected in the product's configuration list, the publishing program pre-synthesizes the dialogue response text that meets the pre-synthesis conditions. The process is as follows: Figure 4 As shown

[0048] For example, some commonly used responses are pre-synthesized during the publishing phase. Besides publishing, to enable the synthesis service to recognize the unchanging parts (clauses) in the synthesized text, this method submits the original information of the synthesized sentences to the synthesis service along with the final synthesized text. This allows the synthesis service to statistically analyze frequently used unchanging clauses based on the original information, thereby automatically using the generated audio as a pre-synthesis cache. Of course, this process can also be based on certain rules or even optional; for example, the synthesis service can cache all clauses generated by the dialogue system.

[0049] Considering real-world usage scenarios, not all generic response text is pre-synthesized. Simply put, given that this method provides the user with "pre-synthesized audio" + "currently synthesized audio," if the first frame of the currently synthesized audio hasn't been synthesized after the pre-synthesized audio finishes playing, audio stuttering will occur. Therefore, the pre-synthesis criteria need to be flexibly adjusted. For example, it can be determined whether the number of pre-synthesized text sub-items is not less than a certain quantity. The goal is to ensure that the network transmission time plus the length of the pre-synthesized audio is greater than the time it takes to generate the first frame of the synthesized audio. This is because if the network transmission time plus the time to play the pre-synthesized audio is too short, audio stuttering will occur if the synthesis of subsequent clauses hasn't yet generated the first frame of the audio.

[0050] As one implementation method, if there is no corresponding pre-synthesized speech, the speech of the multiple clauses segmented from the final response text is synthesized in order from first to last.

[0051] The speech synthesis service also receives the audio stream acquisition request initiated by the client based on the synthesized speech playback address, which further includes:

[0052] After the first frame of the final reply text is synthesized, the synthesized speech of the final reply text is sent to the client in a streaming manner for voice dialogue interaction feedback.

[0053] In this embodiment, if the sentence does not have a corresponding pre-synthesized speech, the speech of the multiple clauses of the final response text segmented in the normal manner is synthesized in the order from first to last.

[0054] For example, if a user asks "What's the weather like tomorrow?", the final reply text would be "Sunny, be careful to avoid the heat." In this case, "sunny" is a keyword in the "#weather#" semantic slot and does not have a corresponding pre-synthesized speech, so speech synthesis is performed using the ordinary method. After the first frame of audio is synthesized, the synthesized speech "Sunny, be careful to avoid the heat" is streamed to the client for voice dialogue interaction feedback.

[0055] like Figure 5The diagram shown is a structural schematic of a voice dialogue interaction system provided in an embodiment of the present invention. The system can execute the voice dialogue interaction method described in any of the above embodiments and is configured in a terminal.

[0056] This embodiment provides a voice dialogue interaction system 10, which includes: a reply text program module 11, a segmentation program module 12, and an interaction program module 13.

[0057] The response text module 11 is used to receive user voice input from the client, generate a general response text for responding to the user voice, and a final response text supplemented with keywords of information to the general response text. The final response text has dialogue tags, which are used to mark the general response text in the final response text. The segmentation module 12 is used to send the final response text to the speech synthesis service. The speech synthesis service generates a synthesized audio playback address and sends it back to the client. It also asynchronously segments the final response text based on the dialogue tags to obtain the sentence beginning text that needs to call the pre-synthesized audio and the remaining text that needs to be synthesized after removing the sentence beginning text. The interaction module 13 is used by the speech synthesis service to receive the audio stream acquisition request initiated by the client based on the synthesized audio playback address, and to stream the synthesized audio of the pre-synthesized audio and the remaining text to the client for voice dialogue interaction feedback.

[0058] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the voice dialogue interaction method in any of the above method embodiments;

[0059] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0060] The system receives user voice input from a client, generates a general response text for responding to the user voice, and a final response text that supplements the general response text with keywords. The final response text is labeled with dialogue tags, which are used to mark the general response text in the final response text.

[0061] The final response text is sent to the speech synthesis service at least once. The speech synthesis service generates a synthesized audio playback address and sends it back to the client. It also asynchronously segments the final response text based on the dialogue tags to obtain the first sentence text of the sentence that needs to call the pre-synthesized audio and the remaining text that needs to be synthesized after removing the first sentence text.

[0062] The speech synthesis service receives an audio stream acquisition request initiated by the client based on the synthesized speech playback address, and sends the pre-synthesized speech and the synthesized speech of the remaining text to the client in a streaming manner for voice dialogue interaction feedback.

[0063] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, the voice dialogue interaction method in any of the above method embodiments is executed.

[0064] Figure 6 This is a schematic diagram of the hardware structure of an electronic device for a voice dialogue interaction method provided in another embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0065] One or more processors 610 and memory 620, Figure 6 Taking a processor 610 as an example, the device for the voice dialogue interaction method may also include an input device 630 and an output device 640.

[0066] The processor 610, memory 620, input device 630, and output device 640 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0067] The memory 620, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the voice dialogue interaction method in the embodiments of this application. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 620, thereby implementing the voice dialogue interaction method of the above-described method embodiments.

[0068] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data, etc. Furthermore, the memory 620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include memory remotely located relative to the processor 610, and these remote memories can be connected to the mobile device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0069] Input device 630 can receive input numerical or character information. Output device 640 may include display devices such as a display screen.

[0070] The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, they execute the voice dialogue interaction method in any of the above method embodiments.

[0071] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0072] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0073] This invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the voice dialogue interaction method of any embodiment of this invention.

[0074] The electronic devices described in this application exist in various forms, including but not limited to:

[0075] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0076] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as tablet computers.

[0077] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0078] (4) Other electronic devices with data processing functions.

[0079] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice dialogue interaction method, comprising: receiving user voice input by a client, generating a general reply text for feeding back the user voice, and a final reply text supplemented with an information keyword to the general reply text, wherein the final reply text is provided with a dialogue label for marking the general reply text in the final reply text; sending at least the final reply text to a voice synthesis service, the voice synthesis service generating a synthesized voice playback address for feeding back to the client, and asynchronously segmenting the final reply text based on the dialogue label to obtain a first sentence text in the final reply text requiring a pre-synthesized voice and remaining text excluding the first sentence text requiring synthesis; the voice synthesis service receiving an audio stream acquisition request initiated by the client based on the synthesized voice playback address, and streaming the pre-synthesized voice and the synthesized voice of the remaining text to the client for voice dialogue interaction feedback.

2. The method of claim 1, wherein, the pre-synthesized voice is generated from the general reply text meeting a pre-synthesis condition, wherein the pre-synthesis condition comprises: if the number of words of the general reply text reaches a preset threshold, generating the pre-synthesized voice of the general reply text to ensure that the voice synthesis service can synthesize the first frame of audio of the remaining text before the client finishes playing the pre-synthesized voice.

3. The method of claim 1, wherein, the sending at least the final reply text to the voice synthesis service comprises: sending the general reply text and the final reply text to the voice synthesis service; the voice synthesis service generating a synthesized voice playback address for feeding back to the client, and asynchronously segmenting the final reply text based on the general reply text or the dialogue label.

4. The method of claim 3, wherein, the asynchronous segmentation of the final reply text based on the general reply text or the dialogue label comprises: segmenting the final reply text into multiple sub-sentences, determining whether the first sub-sentence as the first sentence in the multiple sub-sentences has a corresponding pre-synthesized voice; if the first sub-sentence has a corresponding pre-synthesized voice, obtaining the first sentence text in the final reply text requiring a pre-synthesized voice and the remaining text excluding the first sentence text requiring synthesis.

5. The method of claim 4, wherein, if the first sub-sentence does not have a corresponding pre-synthesized voice, performing voice synthesis on the multiple sub-sentences segmented from the final reply text in the order from first to last; the voice synthesis service receiving the audio stream acquisition request initiated by the client based on the synthesized voice playback address further comprises: after the first frame of audio of the final reply text is synthesized, streaming the synthesized voice of the final reply text to the client for voice dialogue interaction feedback. 6.A voice dialogue interaction system, comprising: a reply text program module configured to receive user voice input by a client, generate a general reply text for feeding back the user voice, and a final reply text supplemented with an information keyword to the general reply text, wherein the final reply text is provided with a dialogue label for marking the general reply text in the final reply text; The cutting program module is configured to send at least the final reply text to a voice synthesis service, and the voice synthesis service feeds back a synthesized voice playing address to the client, and cuts the final reply text based on the dialogue label asynchronously to obtain a first sentence text of the final reply text that needs to call a pre-synthesized voice and remaining text that needs to be synthesized except the first sentence text. The interaction program module is configured to receive an audio stream acquisition request initiated by the client based on the synthesized voice playing address, and stream the pre-synthesized voice and synthesized voice of the remaining text to the client for voice dialogue interaction feedback.

7. The system of claim 6, wherein, The pre-synthesized voice is generated by a general reply text that meets a pre-synthesis condition, wherein the pre-synthesis condition comprises: If the number of words of the general reply text reaches a preset threshold, the pre-synthesized voice of the general reply text is generated to ensure that the first frame audio of the remaining text can be synthesized by the voice synthesis service before the client finishes playing the pre-synthesized voice.

8. The system of claim 6, wherein, The cutting program module is configured to: send the general reply text and the final reply text to the voice synthesis service; The voice synthesis service feeds back a synthesized voice playing address to the client, and cuts the final reply text based on the general reply text or the dialogue label asynchronously.

9. The system of claim 8, wherein, The cutting program module is configured to: cut the final reply text into a plurality of clauses, and determine whether the first clause as the first sentence of the plurality of clauses has a corresponding pre-synthesized voice; If the corresponding pre-synthesized voice exists, the first sentence text of the final reply text that needs to call the pre-synthesized voice and the remaining text that needs to be synthesized except the first sentence text are obtained.

10. The system of claim 9, wherein, The cutting program module is configured to: If the corresponding pre-synthesized voice does not exist, the plurality of clauses of the final reply text are synthesized in the order from the first to the last; The interaction program module is configured to: After the first frame audio of the final reply text is synthesized, the synthesized voice of the final reply text is streamed to the client for voice dialogue interaction feedback.

11. An electronic device comprising: At least one processor and a memory connected to the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of any one of claims 1-5.

12. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of any one of claims 1-5.

Images