Voice interaction method and system based on response words, medium, product and terminal
By building a response word buffering model and packaging user's real-time voice information text, the problem of chat robots' response delay and improper response word processing in real-time voice interaction is solved, and rapid response and natural and accurate response word generation is achieved, which improves the User experience.
Patent Information
- Application Number
- CN202411168082.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-08
AI Technical Summary
During the real-time voice interaction, chatbots have problems such as delay in data processing response and improper prefix and word processing, which affects conversation fluency and user experience.
By building a response word buffering model, we will prioritize calling the response word buffering model to generate response word and broadcast it, package the user's real-time voice information text and response word generation task requests, avoid special token partitions, and ensure that response word and chat robots are generated The nature and accuracy of the content.
It achieves rapid response, improves interaction fluency and personification effect, reduces the problems generated by repeated responses, multiple responses and improper connections or misunderstandings, and enhances user satisfaction and loyalty.
Smart Images

Figure CN118918896A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, system, medium, product and terminal based on response words. Background Art
[0002] With the rapid development of artificial intelligence technology, chatbots (such as Llama-bot) have been widely used in customer service, intelligent assistants, online education and other fields as an important interface for human-computer interaction. These chatbots are based on large-scale pre-trained language models (such as the GPT series), which can understand and generate natural language and achieve smooth conversations with users. However, in actual applications, especially in real-time voice interaction scenarios, chatbots such as Llama-bot face many technical challenges, the most notable of which are data processing response delays and improper prefix and word processing.
[0003] In the process of real-time voice interaction, chatbots such as Llama-bot need to extract text information from the voice signal input by the user, and then understand and generate answers through complex computing models. In this process, each step of data processing may introduce delays. Especially in the case of network delays, excessive server load or increased model calculation complexity, the speed of the first sentence response is often significantly affected. The "response gap" caused by this response delay not only destroys the fluency of the conversation, but also makes users confused or dissatisfied, affecting the user experience.
[0004] In addition, prefix response words (such as "of course" and "received") play an important role in confirming, understanding or guiding the topic in the conversation. However, when processing these response words, chatbots such as Llama-bot are easily separated by special tokens, resulting in repeated responses, improper connection of multiple responses, or misunderstanding of generation, which makes the generated response content unnatural and inaccurate, increases the length and redundancy of the answer, destroys the coherence and logic of the conversation, makes the answer inconsistent with the user's expectations, reduces user satisfaction and information clarity, causes users to feel confused or dissatisfied, and affects the user experience. Summary of the invention
[0005] In view of the shortcomings of the prior art mentioned above, the purpose of the present application is to provide a voice interaction method, system, medium, product and terminal based on response words, which are used to solve the technical problems such as data processing response delay and improper processing of prefix response words in the voice interaction process of the chat robot in the prior art.
[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a voice interaction method based on response words, which is applied to DM, and the DM communication is connected to a chat robot. The voice interaction method includes: constructing a response word buffer model based on an acquired user consultation voice text data set and a reply response word data set; calling the constructed response word buffer model based on the acquired user real-time voice information text to generate response response words and broadcast them; packaging the user real-time voice information text and the response response words to generate task request information, and sending the task request information to the chat robot for the chat robot to generate a reply information text; receiving the reply information text and deleting duplicates from the reply information text; and broadcasting the reply information text after deleting duplicates.
[0007] In some embodiments of the first aspect of the present application, the method for constructing a response word buffer model includes: matching and annotating the acquired user consultation voice text data set and the reply response word data set according to the correspondence to form a training data set; the training data set includes multiple groups of data pairs; each group of data pairs includes a user consultation voice text and a target reply response word corresponding to the user consultation voice text; and training the training data set based on a machine learning model to construct a response word buffer model.
[0008] In some embodiments of the first aspect of the present application, the training data set is trained based on a machine learning model to construct a response word buffer model, including: training the machine learning model based on the training data set to obtain a primary response word buffer model, and generating primary response words corresponding to the user consultation voice text in each data pair according to the primary response word buffer model; according to the primary response words and target response words corresponding to the user consultation voice text in each data pair, and based on the defined contrast loss function, updating the parameters of the primary response word buffer model to obtain a final converged response word buffer model.
[0009] In some embodiments of the first aspect of the present application, the method for the chat robot to generate a reply information text includes: the chat robot parses the acquired task request information, and based on a preset prompt definition, generates a reply information text with the response word as the beginning word.
[0010] In some embodiments of the first aspect of the present application, the method of receiving the reply information text and deleting duplicates of the reply information text includes: based on a string matching algorithm, deleting duplicates of the beginning words of the reply information text and the reported response words.
[0011] In some embodiments of the first aspect of the present application, the string matching algorithm includes: a brute force matching algorithm, a KMP algorithm, a BMH algorithm, and a hash matching algorithm, or a combination of multiple thereof.
[0012] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a voice interaction system based on response words, including: DM; a chat robot, which is communicatively connected to the DM; wherein the DM constructs a response word buffer model based on the acquired user consultation voice text data set and the reply response word data set; calls the constructed response word buffer model based on the acquired user real-time voice information text to generate response response words and broadcast them; packages the user real-time voice information text and the response response words to generate task request information, and sends the task request information to the chat robot for the chat robot to generate a reply information text; receives the reply information text and deletes duplicates from the reply information text; and broadcasts the reply information text after deleting duplicates.
[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the voice interaction method based on response words as described above is implemented.
[0014] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the voice interaction method based on response words as described above.
[0015] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the voice interaction method based on response words as described above.
[0016] As described above, the voice interaction method, system, medium, product and terminal based on response words of the present application have the following beneficial effects: before sending a task request to Llama-bot, the response word buffer model is preferentially called to reply, and the response response words are generated and broadcast, thereby compensating for the problem of response delay, achieving rapid response of voice interaction scenario applications, improving interaction fluency and more realistic anthropomorphic effects; at the same time, ensuring the naturalness and accuracy of the connection between the response words and the content generated by Llama-bot. The task request initiated based on the packaged user real-time voice information text and the response words avoids being separated by special tokens, which helps the chat robot (Llama-bot) to more clearly identify the key points in the user's intentions and reduce interference with irrelevant information, so that the model can focus more on processing the parts directly related to the generation of response words, improve the accurate understanding of user requests, and achieve coherent generation and connection of Llama-bot processed texts. It greatly improves the accuracy of the correspondence between response words and generated reply texts, reduces the problems of repeated responses, improper connection of multiple responses or misunderstanding of generation, ensures that the connection between response words and Llama-bot generated content is more natural, accurate and smooth, enhances the interactive experience between users and models, and improves user satisfaction and loyalty. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 Shown is a flowchart of a voice interaction method based on response words in one embodiment of the present application.
[0018] Figure 2 Shown is a flowchart of constructing a response word buffer model in one embodiment of the present application.
[0019] Figure 3 Shown is a schematic block diagram of a voice interaction system based on response words in one embodiment of the present application.
[0020] Figure 4 Shown is a schematic diagram of the operation flow of a voice interaction system based on response words in one embodiment of the present application.
[0021] Figure 5 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION
[0022] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0023] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:
[0024] <1> DM (Dialog Management): Robot dialogue control module, which is mainly responsible for controlling and managing the process of human-computer dialogue to ensure the fluency and effectiveness of the dialogue.
[0025] <2> Prompt definition: It is a text template or instruction that has been determined at design time. It usually contains a series of guiding words or phrases that are designed to stimulate the GPT model to generate text in a specific direction and context.
[0026] <3> GPT (Generative Pre-Training) model: The generative pre-training model is a deep learning model based on the Transformer structure, mainly used for natural language processing tasks.
[0027] <4> Token: Usually generated by the system when a user logs in or performs some authorization operation, and contains security elements such as the user's identity information, permission information, and possible timestamps. It is a "token" or "credential" used for identity authentication and authorization, which ensures the security and legitimacy of the request.
[0028] During real-time voice interaction, chatbots such as Llama-bot need to extract text information from the voice signal input by the user, and then understand and generate answers through complex computing models. Due to network delays, excessive server load or increased model calculation complexity, the speed of the first sentence response is often significantly affected. The "response gap" caused by this response delay not only destroys the fluency of the conversation, but also makes users confused or dissatisfied, affecting the user experience.
[0029] In addition, when chatbots such as Llama-bot process prefix response words (such as "of course" and "received", etc.), they are easily separated by special tokens, causing the following problems:
[0030] (1) Repeated responses: Since the model fails to accurately identify and delete repeated responses, redundant information is added to the answer, such as “Of course, of course I can provide you with service.” This phenomenon not only increases the length of the answer, but also confuses the user, affecting the user experience.
[0031] (2) Improper connection between multiple responses: When generating responses, the model failed to strictly follow the response words provided by the user as the beginning, but instead generated new response words based on its own understanding, resulting in the reply sentence structure not being consistent with the user's expectations, destroying the coherence and logic of the conversation, reducing user satisfaction and information clarity, and affecting user experience. For example, the user used "received" as the response word, but the answer generated by the model started with "OK". When actually broadcasting, the two were spliced together, resulting in incoherent context.
[0032] (3) Misunderstanding generation: In some cases, the model may mistake a given response word as a user’s question, especially when the response word is similar to the question context or is ambiguous. This type of misunderstanding generation not only fails to meet the user’s actual needs, but also leads to more complex dialogue errors, affecting the user experience.
[0033] In order to solve the problems in the above-mentioned background technology, the present invention provides a voice interaction method, system, medium, product and terminal based on response words, aiming to solve the technical problems such as data processing response delay and improper processing of prefix response words existing in the chat robot during the voice interaction process in the prior art.
[0034] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 The flowchart of the voice interaction method based on the response word in the embodiment of the present invention is shown. In this embodiment, the voice interaction method based on the response word is applied to DM, and the DM communication is connected to the chat robot. The voice interaction method mainly includes the following steps:
[0035] S1: Build a response word buffer model based on the acquired user consultation voice text dataset and response word dataset.
[0036] In this embodiment, if Figure 2 As shown, it is a schematic diagram of the process of constructing a response word buffer model in an embodiment of the present invention. The method of constructing a response word buffer model includes:
[0037] S11: Match and annotate the acquired user consultation voice text dataset and the response word dataset to form a training dataset; the training dataset includes multiple groups of data pairs; each group of data pairs includes the user consultation voice text and the target response word corresponding to the user consultation voice text.
[0038] In this embodiment, a user consultation voice text dataset and a response word dataset used in a voice interaction scenario are collected. The user consultation voice text dataset includes multiple user consultation voice texts, such as "Can you help me with this task?", "Can you tell me?", etc. The response word dataset includes multiple target response words, such as: sure, got it, ok, great to hear, great question, i'm sorry, etc.
[0039] In this embodiment, the corresponding relationship between each user consultation voice text in the user consultation voice text data set and each target response word in the response word data set is matched and annotated to form a training data set containing multiple data pairs. Each data pair includes a user consultation voice text and a target response word corresponding to the user consultation voice text, thereby ensuring that each user consultation voice text can find the corresponding target response word, satisfying the fluency in the natural communication state and avoiding the "sudden feeling" of unsmooth response, that is, when the user says "can you help me?", the robot should reply "sure" instead of "great question".
[0040] S12: Training the training data set based on the machine learning model to build a corresponding word buffer model, including the following steps:
[0041] S121: Train a machine learning model based on the training data set to obtain a primary response word buffer model, and generate primary reply response words corresponding to the user consultation voice text in each data pair according to the primary response word buffer model.
[0042] S122: According to the primary response words and the target response words corresponding to the user consultation voice text in each group of data pairs, and based on the defined contrast loss function, update the parameters of the primary response word buffer model to obtain a final converged response word buffer model.
[0043] In this embodiment, the training data set is used to train a machine learning model, and the model learns the matching relationship from the user consultation voice text (input) to the target response word (output) to obtain a primary response word buffer model. The primary response word buffer model has a certain generation capability and can generate corresponding primary response words based on the user consultation voice text.
[0044] In this embodiment, in order to further improve the performance of the model, iterative optimization is required. In each iteration, the primary model is used to predict each data pair in the training data set to generate corresponding primary response words. These primary response words are then compared with the target response words, and the difference between the two (i.e., loss) is calculated based on the defined contrast loss function. Based on this loss value, the parameters of the model are updated through the back propagation algorithm to reduce errors in future predictions. This process is repeated for multiple iterations until the performance of the model on the training data becomes stable, i.e., the model converges. After sufficient iterative training, the primary response word buffer model gradually optimizes its parameters, and eventually obtains a final response word buffer model with stable performance that can accurately generate response words. This final model can be applied to actual scenarios to generate corresponding response words based on the user's consultation voice text.
[0045] In this embodiment, the machine learning model includes but is not limited to: a sequence-to-sequence (Seq2Seq) model, a Transformer model, a generative adversarial network model, etc.
[0046] In this embodiment, the sequence-to-sequence (Seq2Seq) model is a deep learning model for processing the generation of another sequence from one sequence. This model has a wide range of applications in the fields of machine translation, text summarization, dialogue systems, speech recognition, and image description generation. The sequence-to-sequence (Seq2Seq) model usually consists of two parts: an encoder and a decoder. The data pairs (user consultation voice text-target response words) are fed into the model in batches. The encoder processes the input sequence and generates an internal representation. The decoder generates primary response words based on the output of the encoder and the previously generated response words. Calculate the loss between the generated primary response words and the target response words. Use the back propagation algorithm to update the parameters of the encoder and decoder. Repeat the above steps until the performance of the model on the validation set is no longer significantly improved or reaches the preset training rounds, thereby obtaining a final converged response word buffer model.
[0047] In this embodiment, the Transformer model captures dependencies in the sequence through a self-attention mechanism and performs better when processing long-distance dependencies. It has achieved remarkable success in multiple natural language processing tasks, including machine translation, text summarization, and dialogue generation. Using the standard Transformer encoder-decoder architecture, the encoder is responsible for processing the text sequence of the user's consultation, and the decoder generates a sequence of responses based on the output of the encoder and the previous generation results. Add a word embedding layer to the input text and the target reply response word to convert the text into a numerical vector that the model can understand. At the same time, position encoding can also be added to provide the position information of the word in the sequence. The encoder consists of multiple Transformer encoder layers, each of which contains a self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the encoder to consider the contextual information of the entire sequence when processing the text. The decoder also consists of multiple Transformer decoder layers, but each decoder layer also contains an additional encoder-decoder attention mechanism so that the decoder can pay attention to the output of the encoder. The output layer of the decoder is usually a linear layer that converts the output of the decoder into a score of the vocabulary size, and then converts it to a probability distribution through a softmax function to select the most likely next response word. The data pairs (user consultation voice text-target reply response words) are fed into the model in batches for training. In each batch, the encoder processes the input user consultation voice text sequence and generates a series of internal representations (context vectors). The decoder generates response words one by one based on the output of the encoder and the previously generated response words (using techniques such as teacher forcing or planned sampling). Calculate the loss between the generated response word sequence and the target reply response word sequence. Use the backpropagation algorithm and optimizer (such as Adam) to update the parameters of the Transformer model. Repeat the above steps until the preset training rounds are reached or other stopping conditions are met, thereby obtaining the final converged response word buffer model.
[0048] In this embodiment, the generative adversarial network model includes a generator and a discriminator, and the generator is used to receive the user consultation voice text and generate a response word sequence. The discriminator is used to distinguish the authenticity between the generated response word sequence and the target reply response word sequence, and give a score (indicating the probability that it is a true sequence). Using this score as a reward signal, the parameters of the generator are updated by the policy gradient algorithm. The cross entropy loss function is used to train the discriminator to minimize the probability that the true sequence is misjudged as false and the probability that the false sequence is misjudged as true. The generator and the discriminator are trained alternately until a certain convergence condition is reached (such as the performance is no longer significantly improved or the preset training rounds are reached). The trained generator model is deployed in practical applications to generate response words based on the user consultation voice text.
[0049] It is worth mentioning that, through machine learning, the matched user consultation voice text and target reply word combination are trained to complete the establishment of the response word buffer model, which has the characteristics of small model and fast response.
[0050] S2: Based on the acquired real-time voice information text of the user, the constructed response word buffer model is called to generate and broadcast the response words.
[0051] In this embodiment, when DM (robot dialogue control module) receives real-time voice information text from the user, it will give priority to calling the response word buffer model for reply response, generate response words and broadcast them, thereby compensating for the response delay problem by executing the response word buffer model response before sending a task request to Llama-bot, achieving rapid response of voice interaction scenario applications, improving interaction fluency and more realistic anthropomorphic effects.
[0052] S3: Packing the user's real-time voice information text and the response words to generate task request information, and sending the task request information to the chat robot so that the chat robot can generate a reply information text.
[0053] In this embodiment, DM (robot dialogue control module) packages the user's real-time voice information text and the response words, and initiates a task request to the chat robot (Llama-bot).
[0054] In this embodiment, the method for the chat robot to generate a reply information text includes: the chat robot parses the acquired task request information, and based on a preset prompt definition, generates a reply information text with the response word as the beginning word.
[0055] In this embodiment, after the chatbot (Llama-bot) receives the packaged task request, it confirms to use the response word as the beginning word of the reply information text based on the preset prompt definition, and then generates the reply information text based on the response word as the beginning word to complete the reply work of the request task. According to the preset prompt definition, the GPT model in the chatbot (Llama-bot) responds to the user's real-time voice information text based on the response word in the request task as the beginning word. Exemplarily, the prompt definition can be written as: "Hello" (response word) is used as the beginning word to reply "I want to consult about loan matters" (user's real-time voice information text).
[0056] In this embodiment, the chatbot (Llama-bot) is a generative robot based on a large language model, which includes a GPT model. The chatbot (Llama-bot) generates new and original natural language text as a response based on an input task request using the large language model.
[0057] It is worth noting that the user's real-time voice information text and the response words are packaged and processed, thereby structuring or pre-defining the input data to avoid being separated by special tokens, which helps the GPT model in the chat robot (Llama-bot) to more clearly identify the key points in the user's intentions and reduce interference with irrelevant information, so that the model can focus more on processing the parts directly related to the generation of response words, improve the accurate understanding of user requests, and achieve coherent generation and connection of Llama-bot processed texts. It greatly improves the accuracy of the correspondence between response words and generated reply texts, reduces the problems of repeated responses, improper connection of multiple responses, or misunderstanding of generation, and makes the response words generated by the model more accurate, natural and fluent, enhances the interactive experience between users and models, and improves user satisfaction and loyalty.
[0058] S4: Receive the reply information text and delete duplicate items from the reply information text.
[0059] S5: Announce the reply information text after the duplicate items are deleted.
[0060] In this embodiment, the method of receiving the reply information text and deduplicating the reply information text includes: based on a string matching algorithm, deduplicating the beginning words of the reply information text and the reported response words.
[0061] In this embodiment, after the reply message text is generated, it is returned to DM. The beginning words of the reply message text are repeated with the response words that have been broadcast. To avoid repeated broadcasting, DM deletes duplicate items between the reply message text and the response words that have been broadcast through a string matching algorithm. After the duplicate items in the reply message text are deleted, the response words that have been broadcast are connected and continued to be broadcast (the response time can generally be controlled within 70 to 150ms) to achieve the purpose of smooth connection.
[0062] In this embodiment, the string matching algorithm includes: a brute force matching algorithm, a KMP algorithm, a BMH algorithm, and a hash matching algorithm, or a combination of multiple thereof.
[0063] In this embodiment, the brute force matching algorithm is also called the naive matching algorithm, which is a simple and intuitive string matching method. Its basic principle is to start from the first character of the text string (also called the main string or the target string), and compare it with the pattern string (also called the substring or the string to be matched) character by character. If the match fails, the pattern string is moved one position to the right and then compared with the next character of the text string until a matching substring is found or the text string is traversed.
[0064] In this embodiment, the KMP algorithm (The Knuth-Morris-Pratt Algorithm) is an efficient string matching algorithm, which is mainly used to find the position of a shorter string (called a pattern string) in a longer string (called a main string), and its time complexity is O(m+n), where m and n are the lengths of the main string and the pattern string, respectively. The core of the KMP algorithm is to use the information after the matching failure to minimize the number of matches between the pattern string and the main string to achieve the purpose of fast matching. In specific implementation, it is implemented through a function called next, which itself contains local matching information of the pattern string. When the matching fails, the algorithm will slide the pattern string to the appropriate position based on the information provided by the next function.
[0065] In this embodiment, the BMH algorithm (Boyer-Moore-Horspool) reduces unnecessary traversals by focusing on the last character (i.e., "bad character") when the match fails, thereby improving the efficiency of string matching. The core of the BMH algorithm is: when searching for a pattern string (Pattern) in the main string (Text), if an unmatched character is encountered, the distance the pattern string needs to slide to the right is determined based on the last position of the "bad character" in the pattern string (excluding the last character of the pattern string). This strategy reduces invalid comparisons after a match fails and improves the efficiency of the algorithm.
[0066] In this embodiment, the hash matching algorithm maps a string into a number (ie, a hash value) through a hash function, and compares the hash values to determine whether the strings are equal or similar.
[0067] The voice interaction method based on response words of the present invention preferentially calls the response word buffer model for reply response before sending a task request to Llama-bot, generates and broadcasts the response words, thereby compensating for the problem of response delay, achieving rapid response of voice interaction scenario applications, improving interaction fluency and more realistic anthropomorphic effects; at the same time, ensuring the naturalness and accuracy of the connection between the response words and the content generated by Llama-bot. The task request initiated based on the packaged user real-time voice information text and the response words avoids being separated by special tokens, which helps the chat robot (Llama-bot) to more clearly identify the key points in the user's intentions and reduce interference with irrelevant information, so that the model can focus more on processing the parts directly related to the generation of response words, improve the accurate understanding of user requests, and achieve coherent generation and connection of Llama-bot processed texts. It greatly improves the accuracy of the correspondence between response words and generated reply texts, reduces the problems of repeated responses, improper connection of multiple responses or misunderstanding of generation, ensures that the connection between response words and Llama-bot generated content is more natural, accurate and smooth, enhances the interactive experience between users and models, and improves user satisfaction and loyalty.
[0068] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0069] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.
[0070] Figure 3 is a schematic block diagram of a voice interaction system based on response words provided in an embodiment of the present application. Figure 3 As shown, the speech interaction system 300 based on the response word includes:
[0071] DM 301;
[0072] A chat robot 302, which is in communication connection with the DM 301;
[0073] Among them, the DM 301 constructs a response word buffer model based on the acquired user consultation voice text data set and the response word data set; calls the constructed response word buffer model based on the acquired user real-time voice information text to generate response words and broadcast them; packages the user real-time voice information text and the response words to generate task request information, and sends the task request information to the chat robot 302 for the chat robot 302 to generate a reply information text; receives the reply information text and deletes duplicates from the reply information text; and broadcasts the reply information text after deleting duplicates.
[0074] like Figure 4 As shown, a schematic diagram of the operation flow of the voice interaction system based on the response word in the embodiment of the present application is shown, including:
[0075] S401: The DM receives the user's real-time voice information text, and calls the constructed response word buffer model to generate and broadcast the response word.
[0076] S402: The DM packages the user's real-time voice information text and the response words to generate task request information, and sends the task request information to the chat robot.
[0077] S403: The chat robot generates a reply message text starting with the response word, and returns the reply message text to the DM.
[0078] S404: The DM deletes duplicate items from the reply information text, and broadcasts the reply information text after the duplicate items are deleted.
[0079] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.
[0080] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.
[0081] Figure 5 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 5As shown, the electronic terminal 500 includes: at least one processor 501, a memory 502, at least one network interface 503 and a user interface 505. The various components in the device are coupled together through a bus system 504. It can be understood that the bus system 504 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 In the specification, various buses are labeled as bus systems.
[0082] The user interface 505 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.
[0083] It is understood that the memory 502 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.
[0084] The memory 502 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 500. Examples of these data include: any executable program used to operate on the electronic terminal 500, such as an operating system 5021 and an application 5022; the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 5022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The voice interaction method based on the response word provided in the embodiment of the present invention can be included in the application 5022.
[0085] The method disclosed in the above embodiment of the present invention can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 501 or the instruction in the form of software. The above processor 501 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 501 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 501 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0086] In an exemplary embodiment, the electronic terminal 500 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.
[0087] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes Figure 1 to Figure 2 A method according to any one of the embodiments shown.
[0088] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 1 to Figure 2 A method according to any one of the embodiments shown.
[0089] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).
[0090] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0092] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0095] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrated. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs), etc.).
[0096] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.
[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0098] In summary, the present application provides a voice interaction method, system, medium, product and terminal based on response words. Before sending a task request to Llama-bot, the response word buffer model is preferentially called to reply, and the response response words are generated and broadcast, thereby compensating for the problem of response delay, achieving rapid response of voice interaction scenario applications, improving interaction fluency and more realistic anthropomorphic effects; at the same time, ensuring the naturalness and accuracy of the connection between the response words and the content generated by Llama-bot. The task request initiated based on the packaged user real-time voice information text and the response words avoids being separated by special tokens, helps the chat robot (Llama-bot) to more clearly identify the key points in the user's intention, reduces interference with irrelevant information, and enables the model to focus more on processing the parts directly related to the generation of response words, improves the accurate understanding of user requests, and achieves the coherent generation and connection of Llama-bot processed texts. It greatly improves the accuracy of the response words corresponding to the generated reply text, reduces the problems of repeated responses, improper connection of multiple responses, or misunderstanding of generation, ensures that the connection between the response words and the content generated by Llama-bot is more natural, accurate and smooth, enhances the interactive experience between the user and the model, and improves user satisfaction and loyalty. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.
[0099] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A voice interaction method based on response words, characterized in that: Applied to DM, the DM communication is connected to a chat robot, and the voice interaction method includes: Build a response word buffer model based on the acquired user consultation voice text dataset and response word dataset; According to the acquired real-time voice information text of the user, the constructed response word buffer model is called to generate the response word and broadcast it; Packing the user's real-time voice information text and the response words to generate task request information, and sending the task request information to the chat robot so that the chat robot can generate a reply information text; Receiving the reply information text and deleting duplicate items from the reply information text; Announces the reply message text after duplicates are removed.
2. The voice interaction method based on response words according to claim 1 is characterized in that: Ways to build a response word buffer model include: The obtained user consultation voice text data set and the answer word data set are matched and annotated according to the corresponding relationship to form a training data set; the training data set includes multiple groups of data pairs; each group of data pairs includes the user consultation voice text and the target answer word corresponding to the user consultation voice text; The training data set is trained based on a machine learning model to construct a response word buffer model.
3. The voice interaction method based on response words according to claim 2 is characterized in that: The method of training the training data set based on the machine learning model to build the corresponding word buffer model includes: Training a machine learning model based on the training data set to obtain a primary response word buffer model, and generating primary reply response words corresponding to the user consultation voice text in each data pair according to the primary response word buffer model; According to the primary response words and the target response words corresponding to the user consultation voice text in each group of data pairs, and based on the defined contrast loss function, the parameters of the primary response word buffer model are updated to obtain the final converged response word buffer model.
4. The voice interaction method based on response words according to claim 1, characterized in that: The method for the chat robot to generate a reply information text includes: the chat robot parses the acquired task request information, and based on a preset prompt definition, generates a reply information text with the response word as the beginning word.
5. The voice interaction method based on response words according to claim 4 is characterized in that: The method of receiving the reply information text and deleting duplicates of the reply information text includes: based on a string matching algorithm, deleting duplicates of the beginning words of the reply information text and the reported response words.
6. The voice interaction method based on response words according to claim 5 is characterized in that: The string matching algorithm includes: a brute force matching algorithm, a KMP algorithm, a BMH algorithm and a hash matching algorithm, or a combination of multiple thereof.
7. A voice interaction system based on response words, characterized in that: include: DM; A chat robot, connected to the DM for communication; The DM constructs a response word buffer model based on the acquired user consultation voice text data set and the response word data set; and calls the constructed response word buffer model based on the acquired user real-time voice information text to generate and broadcast the response word; The user's real-time voice information text and the response words are packaged to generate task request information, and the task request information is sent to the chat robot so that the chat robot can generate reply information text; the reply information text is received and duplicates are deleted from the reply information text; and the reply information text after duplicates are deleted is broadcasted.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the voice interaction method based on response words as described in any one of claims 1 to 6 is implemented.
9. A computer program product, characterized in that The computer program product includes computer program codes, and when the computer program codes are executed on a computer, the computer is enabled to implement the speech interaction method based on response words as described in any one of claims 1 to 6.
10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the voice interaction method based on response words according to any one of claims 1 to 6.
Citation Information
Cited By
Multi-modal data conversation robot interaction method, configuration method, terminal, medium and product
CN120336486A
