Multi-Agent-based voice interaction method

Abstract

The invention provides a Multi-Agent-based voice interaction method, which comprises the following steps of S1, waking up a voice assistant, collecting voice data input by a user, converting the collected voice data into text input through an STT voice-to-text module, and storing the text input into a historical dialogue database; through a weighted intention classification algorithm using an intention classification large model and a preset vector library, accurate distribution of user tasks is significantly improved. Meanwhile, the intelligent home and equipment control are set as independent agents, the burden of an intention classification large model is relieved, and the intelligence and accuracy of the system are improved. The design ensures that a system link is simple and efficient, provides highly personalized services, adapts to various devices and application scenes, and improves user experience.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and more specifically, to a multi-agent based voice interaction method.

[0002] With the increasing popularity of smart devices and artificial intelligence technology, voice interaction assistants have gradually become an indispensable auxiliary tool in our daily lives. These assistants can not only accurately identify and respond to user needs, but also provide diversified services such as smart home control, music playback, knowledge questions and answers, real-time weather query and news information, greatly improving the convenience and comfort of life. They enrich the way users obtain information and enhance the overall experience of daily life.

[0003] However, the voice assistants currently on the market generally rely on a single intelligent processing unit to perform tasks and parse instructions, and this design has obvious limitations. First, when dealing with complex or multi-level conversations, these systems are usually only able to perform simple rule matching and command recognition, and it is difficult to deeply analyze the user's deep intentions. Especially when handling multiple tasks at the same time, the performance of existing systems is often unsatisfactory. In addition, traditional rule-based voice assistants are insufficient in understanding semantics and user emotions, making users often feel that they are just interacting with mechanical programs and lack the necessary anthropomorphic experience.

[0004] Therefore, developing a Multi-Agent based voice interaction method has become an important technical requirement today. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a Multi-Agent based voice interaction method which can not only enhance the user's interactive experience but also expand the application scenarios of voice assistants in view of the shortcomings of the above technical solutions.

[0006] The present invention provides a multi-agent-based voice interaction method, the method comprising the following steps:

[0007] S1, wake up the voice assistant, collect the voice data input by the user, convert the collected voice data into text input through the STT voice-to-text module, and store the text input in the historical conversation database;

[0008] S2, extract the historical conversation data related to the corresponding user and the conversation in the historical conversation database, and transmit the text and historical conversation data in the historical conversation data to the intention classification model Agent_Classifer(). The intention classification model Agent_Classifer() classifies the text and historical conversation data into smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes, calculates the possibility of each scene, and determines which scene has the highest possibility;

[0009] S3, according to the possibility of the scenario, select appropriate knowledge supplement methods, external tool calls, smart home task dispatch verification and processing, and device operation task dispatch verification and processing, and input the supplemented knowledge input or rejected text input into the question-answering big model. The question-answering big model generates the final text output in a streaming manner, and sends the text output, the designated home device instructions called, and the designated operation instructions for device operation to the TTS text-to-speech module in real time to generate a voice reply.

[0010] In the Multi-Agent based voice interaction method of the present invention, step S2 comprises the following steps:

[0011] S21, reconstructing the text input into multiple rounds of text input text, extracting the historical conversation data history related to the corresponding user and the conversation, and storing the text input text in the historical conversation database History(text); wherein the historical data related to the corresponding user and the conversation is expressed as History(text)=history+text, i.e., the historical conversation data history plus the current round of text input text;

[0012] S22, fine-tune the intent classification model Agent_Classifer(), and call the fine-tuned intent classification model Agent_Classifer() to perform intent classification on the reconstructed multi-round text input History(text), obtain smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes, and calculate the possibilities of smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes; wherein, the calculation formula of the possibility vector of the smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes is as follows:

[0013]

[0014] R1 represents the possibility vector of smart home scenario, device operation scenario, tool call scenario and question-answering scenario, SH represents smart home scenario, DC represents device operation scenario, Tools represents tool call scenario, and QA represents question-answering scenario; Respectively represent the probability rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios;

[0015] S23, converting the text input text into a vector Input_Vector, and storing it in a preset vector library Categpry_Vector, and calculating the similarity between the text input text and the preset vector libraries Categpry_Vectors of the smart home scene, device operation scene, tool call scene and question-and-answer scene, and obtaining the possibility ranking R2. The calculation formula of the possibility ranking is as follows:

[0016]

[0017] in, They respectively represent the possibility rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios.

[0018] In the Multi-Agent based voice interaction method of the present invention, the step S2 further includes the following steps:

[0019] S24, weighted average the possibility vectors R1 of smart home scenes, device operation scenes, tool call scenes and question-answering scenes and the possibility rankings R2 of smart home scenes, device operation scenes, tool call scenes and question-answering scenes to obtain the final classification scene R final , and according to the final classification scene R final The probability of judging which scene is the highest; the final classification scene R final The calculation formula is

[0020]

[0021] Among them, α and β represent weights.

[0022] In the Multi-Agent based voice interaction method of the present invention, step S3 comprises the following steps:

[0023] S31, if the final classification scene R final For the question-answering scenario, The weighted ranking value of the question-answering scenario is smaller than that of all other scenarios, which means that the question-answering scenario has the highest probability. final For question-answering scenarios and If Rag = = 1, then enable RAG mode for knowledge supplement; if the final classification scene R final For question-answering scenarios and If Rag= = 0, then the knowledge is supplemented by multi-engine crawler search. When the knowledge supplement is completed, a set of multi-round text input History (text) and supplementary knowledge Knowledge Final_Input will be output, where Final_Input = (History (text), Knowledge). Rag Indicates the mark bit.

[0024] In the Multi-Agent based voice interaction method of the present invention, the step S3 further includes the following steps:

[0025] S32, if the final classification scene R final Call the scene for the tool, that is If the weighted ranking value of the tool call scenario is less than that of all other scenarios, then the tool call scenario is most likely to be the tool call scenario; and the specified tool is called to obtain supplementary information. When the tool call is completed, a supplementary information Supplement_Info is returned, and finally a set Final_Input containing multiple rounds of text input History (text) and supplementary information Supplement_Info is generated.

[0026] In the Multi-Agent based voice interaction method of the present invention, the step S3 further includes the following steps:

[0027] S33, if the final classification scene R final For smart home scenarios, If the weighted ranking value of the tool call scenario is less than that of all other scenarios, it is most likely to be a smart home scenario. Determine whether it belongs to the smart home category. If it does, analyze the home devices that the user needs to call and the functions to be executed, and convert the user's needs into smart home call statements and execute them, encapsulated as SHAgent(). If it does not belong to the smart home category, reject it.

[0028] In the Multi-Agent based voice interaction method of the present invention, the step S3 further includes the following steps:

[0029] S34, if the final classification scene R final Manipulate the scene for the device, i.e. in, If the weighted ranking value of the device operation scenario is less than that of all other scenarios, the device operation scenario is most likely to be the scenario. Determine whether it belongs to the device operation category. If it does, analyze the device that the user needs to call and the function to be executed, and convert the user's needs into device call statements and execute them. If it does not belong to the device operation category, reject it.

[0030] In the Multi-Agent based voice interaction method described in the present invention; in the step S3, the input of the knowledge supplemented from step S31 to step S32 or the rejected multiple rounds of text input are input into the question-answering big model, the question-answering big model stream-generates the final text output, and sends the text output to the TTS text-to-speech module in real time to generate a voice reply.

[0031] The multi-agent-based voice interaction method of the present invention significantly improves the accurate distribution of user tasks by using the intention classification model and the weighted intention classification algorithm of the preset vector library. At the same time, the smart home and device operation are established as independent intelligent entities, which reduces the burden of the intention classification model and improves the intelligence and accuracy of the system. This design ensures that the system link is simple and efficient, while providing highly personalized services, adapting to a variety of devices and application scenarios, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of the voice interaction method based on Multi-Agent of the present invention;

[0033] Figure 2 It is a flow chart of an embodiment of a voice interaction method based on Multi-Agent of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] like Figure 1-2 As shown, Figure 1 1 is a flow chart of an embodiment of a multi-agent based voice interaction method of the present invention. A multi-agent based voice interaction method is provided, comprising the following steps:

[0037] In step S1, the voice assistant is awakened, the voice data input by the user is collected, and the collected voice data is converted into text input through the STT voice-to-text module, and the text input is stored in the historical conversation database;

[0038] In step S2, historical conversation data related to the corresponding user and conversation in the historical conversation database is extracted, and the text in the historical conversation data and the historical conversation data are transmitted to the intention classification model Agent_Classifer(). The intention classification model Agent_Classifer() performs intent classification on the text and the historical conversation data to obtain smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes, calculates the possibility of each scene, and determines which scene has the highest possibility;

[0039] In step S3, appropriate knowledge supplement methods, external tool calls, smart home task dispatch verification and processing, and device operation task dispatch verification and processing are selected according to the possibility of the scenario, and the supplemented knowledge input or rejected text input is input into the question-answering big model. The question-answering big model generates the final text output in a streaming manner, and sends the text output, the designated home device instructions called, and the designated operation instructions for device operation to the TTS text-to-speech module in real time to generate a voice reply.

[0040] In this embodiment, step S2 includes the following steps:

[0041] In step S21, the text input is reconstructed into multiple rounds of text input text, the historical conversation data history related to the corresponding user and the conversation is extracted, and the text input text is stored in the historical conversation database History(text); wherein the historical data related to the corresponding user and the conversation is expressed as History(text)=history+text, i.e., the historical conversation data history plus the current round of text input text;

[0042] In step S22, the intent classification model Agent_Classifer() is fine-tuned, and the fine-tuned intent classification model Agent_Classifer() is called to perform intent classification on the reconstructed multi-round text input History(text), and the smart home scene, device operation scene, tool call scene and question-and-answer scene are obtained, and the possibility of the smart home scene, device operation scene, tool call scene and question-and-answer scene is calculated; wherein, the calculation formula of the possibility vector of the smart home scene, device operation scene, tool call scene and question-and-answer scene is as follows:

[0043]

[0044] R1 represents the possibility vector of smart home scenario, device operation scenario, tool call scenario and question-answering scenario, SH represents smart home scenario, DC represents device operation scenario, Tools represents tool call scenario, and QA represents question-answering scenario; Respectively represent the probability rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios;

[0045] Specifically, the value of each dimension represents the probability ranking of the scenario. That is, the big model determines that the subsequent tasks are most likely to be delivered to the smart home module for processing. For example, if the probability of this round of input belonging to the smart home ranks second, then

[0046] In step S23, the text input text is converted into a vector Input_Vector and stored in a preset vector library Categpry_Vector, and the similarity between the text input text and the preset vector library Categpry_Vectors of the smart home scene, device operation scene, tool call scene and question-and-answer scene is calculated to obtain the possibility ranking R2. The calculation formula of the possibility ranking is as follows:

[0047]

[0048] in, They respectively represent the possibility rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios.

[0049] In this embodiment, step S2 further includes the following steps:

[0050] In step S24, the possibility vectors R1 of smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes and the possibility rankings R2 of smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes are weighted averaged to obtain the final classification scene R final , and according to the final classification scene R final The probability of judging which scene is the highest; the final classification scene R final The calculation formula is

[0051]

[0052] Among them, α and β represent weights, which can be preset based on actual experience or dynamically adjusted based on classification performance. For example, α and β are preset to 0.8 and 0.2 based on experience, R final =0.8·R1+0.2·R2=[0.8·1+0.2·1, 0.8·4+0.2·4, 0.8·2+0.2·3, 0.8·3+0.2·

[0053] 2]=[1,4,2.2,2.8].

[0054] In this embodiment, step S3 includes the following steps:

[0055] In step S31, if the final classification scene R final For the question-answering scenario, The weighted ranking value of the question-answering scenario is smaller than that of all other scenarios, which means that the question-answering scenario has the highest probability. final For question-answering scenarios and If Rag = = 1, then enable RAG mode for knowledge supplement; if the final classification scene R final For question-answering scenarios and If Rag = = 0, then the knowledge is supplemented by multi-engine crawler search. When the knowledge supplement is completed, a set of multi-round text input Histoty(text) and supplementary knowledge Knowledge Final_Input will be output, where Final_Input = (History(text), Knowledge). Rag Indicates the mark bit.

[0056] In this embodiment, the step S3 further includes the following steps:

[0057] In step S32, if the final classification scene R final Call the scene for the tool, i.e. If the weighted ranking value of the tool call scenario is less than that of all other scenarios, then the tool call scenario is most likely to be the tool call scenario; and the specified tool is called to obtain supplementary information. When the tool call is completed, a supplementary information Supplement_Info is returned, and finally a set Final_Input containing multiple rounds of text input History (text) and supplementary information Supplement_Info is generated.

[0058] In this embodiment, the step S3 further includes the following steps:

[0059] In step S33, if the final classification scene R final For smart home scenarios, If the weighted ranking value of the tool call scenario is less than that of all other scenarios, it is most likely to be a smart home scenario. Determine whether it belongs to the smart home category. If it does, analyze the home devices that the user needs to call and the functions to be executed, and convert the user's needs into smart home call statements and execute them, encapsulated as SHAgent(). If it does not belong to the smart home category, reject it.

[0060] In this embodiment, the step S3 further includes the following steps:

[0061] In step S34, if the final classification scene R final Manipulate the scene for the device, i.e. in, If the weighted ranking value of the device operation scenario is less than that of all other scenarios, the device operation scenario is most likely to be the scenario. Determine whether it belongs to the device operation category. If it does, analyze the device that the user needs to call and the function to be executed, and convert the user's needs into device call statements and execute them. If it does not belong to the device operation category, reject it.

[0062] In this embodiment, in step S3, the input of knowledge supplemented from step S31 to step S32 or multiple rounds of rejected text input are input into the question-answering big model, and the question-answering big model stream-generates the final text output, and sends the text output to the TTS text-to-speech module in real time to generate a voice reply.

[0063] This application integrates multiple dedicated intelligent processing agents (Agents) and uses optimized intent classification algorithms to greatly improve the accuracy of task distribution in complex applications with multiple scenarios and multiple devices. The multi-agent design not only effectively reduces the burden of the intent recognition module, but also enhances the intelligent performance of the system and the accuracy of understanding user intent. Through an efficient collaboration mechanism, the system achieves faster and more accurate speech understanding and response. In addition, combined with RAG knowledge base retrieval, multi-engine crawlers and tool call modules, the system makes full use of rich historical data and real-time information to enhance the relevance and accuracy of the interaction. This system is built on a powerful and flexible multi-agent model platform with excellent adaptability and scalability, suitable for a variety of hardware and software environments. Despite the increase in the number of agents, the reasonable architecture design avoids a significant increase in link complexity, ensuring that users have an efficient and natural interactive experience. The system can be quickly reorganized and adjusted according to specific application requirements, making the product highly portable and market adaptable, and is widely used in intelligent interaction scenarios such as car assistants, smart home control, and personal life assistants to meet the diverse needs of users.

[0064] Specifically, the STT speech-to-text module is used to convert the user's speech input into text for subsequent processing, and the intent classification model is used to analyze the user's text input and classify it into different intent categories. The fine-tuned large language model (LLM), namely the intent recognition model, combines the preset vector library and outputs the final intent classification scenario by weighted average. And according to the intent recognition result, select and obtain supplementary knowledge methods. Supplementary knowledge methods include RAG (retrieval enhanced generation) and multi-engine crawler search to ensure efficient knowledge expansion. Tool call is used to parse and execute the tool call statement generated in the intent classification model, complete the call of a specific tool and obtain the corresponding supplementary information. Smart home is used to determine whether the task distributed by the intent classification model belongs to the scope of smart home control, and perform the corresponding home device operation. Device operation is used to analyze the user's instructions and operate the specified device, such as a mobile phone or computer, to complete the call of a specific function. The question-answering model is used to receive and process all multi-round text input and supplementary information, generate the final text output, and deliver it to the TTS text-to-speech module in real time for speech generation. And generate speech to reply to the user. The RAG knowledge base is used to store and retrieve knowledge data generated by the system, and supports the knowledge expansion function of the system. The historical dialogue database is used to store the user's historical dialogue records to support multi-round dialogue and context understanding.

[0065] When the system carrier continuously monitors the environment in a loop, it transmits the monitored content to the wake-up recognition module for analysis. Once the wake-up word set by the user is recognized, the system will issue a voice interaction prompt to guide the user to input and start the Multi-Agent core.

[0066] Collect the user's voice input and filter the blank part of the input according to the preset blank threshold. Then, call the STT (Speech-to-Text) voice-to-text module to convert the user's voice input into text input. text = STT (Filter (Speech_Input, Blank_Threshold))

[0067] Speech_Input is the collected single-round speech input of the user, and Blank_Threshold is the set blank threshold. Blank parts such as pauses in the input are filtered according to the threshold.

[0068] In this embodiment, when the length of the user's single-round voice input Speech_Input is 5 seconds, the set Blank_Threshold is 200 milliseconds. According to the threshold, the blank parts of 200 milliseconds and above in the user's input voice are filtered. After recognition, a total of 3 speech blanks with a length of more than 200 milliseconds were detected, namely 300 milliseconds, 250 milliseconds and 450 milliseconds. After blank filtering, the effective length of the user's input voice is shortened to 4s=4000ms=5s-250ms-300ms-450ms seconds. The STT speech-to-text module is used to convert the filtered voice, and the final text input by the user is: "It was really annoying that it rained all day yesterday. The weather is good today. Please help me open the curtains and let in some fresh air."

[0069] Right now

[0070] text=

[0071] {user: "It rained all day yesterday, it was really annoying. The weather is good today. Please help me open the curtains to let in some fresh air."}.

[0072] The aforementioned text input is stored in the historical conversation database and submitted together with the historical conversation data to the intent classification model for intent classification to obtain smart home scenarios, device operation scenarios, tool call scenarios and question-and-answer scenarios.

[0073] In this embodiment, the user is in the first round of dialogue, and there is no historical dialogue, that is, history = [].

[0074] History(text)= []+ {user: "It rained all day yesterday, it was really annoying. Today the weather is nice. Please help me open the curtains to get some fresh air."} = [{user: "It rained all day yesterday, it was really annoying. Today the weather is nice. Please help me open the curtains to get some fresh air."}].

[0077] In step S31, when the intention classification model is calculating the category, it will determine in advance which knowledge supplement method to choose, and use If Rag The flag indicates whether to use the RAG method. Rag = = 1, the RAG method is enabled for knowledge supplementation. Otherwise, the knowledge will be supplemented through multi-engine crawler search. In addition, when the RAG method cannot be recalled, it will also switch to the multi-engine crawler search method. After the knowledge supplement is completed, a set of multi-round text input History (text) and supplementary knowledge Knowledge will be output Final_Input: Final_Input = (History (text), Knowledge)

[0078] In this embodiment, due to The conditions for enabling the knowledge supplement module are not met, so the knowledge supplement module is dormant in this round.

[0079] In step S32, when the intent classification model calculates the category, a tool call statement tool_call_message will be generated in advance. In the tool call, this statement will be parsed, that is, the tool call is performed through Tool_Fetcher(). The tool call is responsible for parsing the tool specified by the tool call statement and the passed parameters, and returns a supplementary information Supplement_Info after the tool call is completed. Finally, a set Final_Input containing multiple rounds of text input History (text) and supplementary information Supplement_Info is generated.

[0080] In this embodiment, due to The tool call module activation conditions are not met, so the tool call module is dormant in this round.

[0081] In step S33, the smart home will determine whether the task dispatch of the intent classification model is reasonable, reject unreasonable dispatches, further analyze reasonable task dispatches, and call designated home devices. The smart home module itself is an agent with a large model as the core, so it does not rely on the analysis of the intent classification model. It can analyze multiple rounds of text input History (text) by itself to determine whether it belongs to the category of smart home.

[0082] If it belongs to the category of smart home, it will analyze the home devices and functions that the user needs to call, and convert the user's needs into smart home call statements and execute them. Its process is similar to that of the Tools module and is encapsulated as SHAgent(). If it does not belong to the category of smart home, it will be rejected.

[0083] There are two ways to reject: for time-sensitive scenarios, you can directly reject it to the question-answering model in step S3 to quickly reply to the user; for relatively time-insensitive scenarios, reject it to the intent classification model for redirection. The system can only redirect once at most. If the redirection fails, it will be rejected to the question-answering model.

[0084] In this embodiment, due to The smart home module is enabled. After verification by the smart home big model, the current user intention belongs to the smart home processing category History(text)∈SmartHome. The big model will analyze multiple rounds of text input History(text) and provide structured feedback on the smart home that the user needs to operate and the operations that need to be performed: furniture_call_message = {"device":"curtain","action":"open",}

[0085] A script will then analyze the structured feedback from the large model and perform corresponding operations on the specified home.

[0086] The device operator will determine whether the task assignment of the intent classification model is reasonable, reject unreasonable assignments, further analyze reasonable task assignments, and operate the device to complete the specified operation.

[0087] Specifically, device control is also an agent with a large model as its core. Therefore, it does not rely on the analysis of the intent classification large model. It can analyze multiple rounds of text input History (text) by itself to determine whether it belongs to the device control category DeviceControl.

[0088] If it belongs to the category of device manipulation, the device (such as mobile phone, computer) and the function to be executed are analyzed, and the user's needs are converted into device call statements and executed. The process is similar to the Tools module and encapsulated as DCAgent(). The rejection mechanism is the same as that of smart home.

[0089] In this embodiment, due to The conditions for enabling device operation are not met, so this round of device operation is dormant.

[0090] The beneficial effects of a multi-agent-based voice interaction method provided by an embodiment of the present invention are at least:

[0091] 1. The present invention is based on a general framework of Multi-Agent voice assistants, which can adapt to a variety of carrier platforms. The flexibility of the framework allows it to be easily deployed on different devices. In addition, after removing the first and last voice processing links, the framework can still be used as a powerful text processing system, providing more extensive support for various applications.

[0092] 2 The present invention introduces an innovative intent classification algorithm, which significantly improves the accuracy of intent recognition by weighted averaging the similarity between the classification results of the large model and the preset vector library. The application of this algorithm ensures that user tasks can be distributed and processed more accurately, thereby improving the overall efficiency of the system and user experience.

[0093] 3 The present invention sets up the smart home module as an independent agent, so that the system can independently judge and execute tasks related to smart home. Smart home combines the intention classification model, can independently analyze the user's multi-round dialogue input, accurately identify user needs, intelligently schedule home appliances, and provide highly personalized services.

[0094] 4. The present invention also integrates a device control module and sets it up as an independent agent, which enhances the system's device control capabilities. This module can quickly identify and execute control tasks involving multiple devices (such as mobile phones, computers, etc.) through independent analysis of user input. This function not only enriches the system's interactive methods, but also greatly improves the convenience and efficiency of user operations.

[0095] 5. The present invention reduces the pressure on the intention classification model by setting up the smart home and device control modules as independent agents, and improves the accuracy and intelligence of understanding and processing user intentions. At the same time, this architecture can verify whether the division of the intention classification model is correct, so that the intention classification model only needs to deal with the division of large areas, while the task execution in each small area (such as the smart home module) is analyzed and executed by the corresponding module, which enhances the intelligence and accuracy of the system.

[0096] 6. The present invention improves the accuracy of user intention recognition without increasing the complexity of the system link. The length of each link is almost the same and relatively short, ensuring timely response to the user.

[0097] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0099] Therefore, the above is only a preferred specific implementation mode of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A Multi-Agent based voice interaction method, characterized in that: The method comprises the following steps: S1, wake up the voice assistant, collect the voice data input by the user, convert the collected voice data into text input through the STT voice-to-text module, and store the text input in the historical conversation database; S2, extract the historical conversation data related to the corresponding user and the conversation in the historical conversation database, and transmit the text and historical conversation data in the historical conversation data to the intention classification model Agent_Classifer(). The intention classification model Agent_Classifer() classifies the text and historical conversation data into smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes, calculates the possibility of each scene, and determines which scene has the highest possibility; S3, according to the possibility of the scenario, select appropriate knowledge supplement methods, external tool calls, smart home task dispatch verification and processing, and device operation task dispatch verification and processing, and input the supplemented knowledge input or rejected text input into the question-answering big model. The question-answering big model generates the final text output in a streaming manner, and sends the text output, the designated home device instructions called, and the designated operation instructions for device operation to the TTS text-to-speech module in real time to generate a voice reply.

2. The multi-agent based voice interaction method according to claim 1, characterized in that: The step S2 comprises the following steps: S21, reconstructing the text input into multiple rounds of text input text, extracting the historical conversation data history related to the corresponding user and the conversation, and storing the text input text in the historical conversation database History(text); wherein the historical data related to the corresponding user and the conversation is expressed as History(text)=history+text, i.e., the historical conversation data history plus the current round of text input text; S22, fine-tune the intent classification model Agent_Classifer(), and call the fine-tuned intent classification model Agent_Classifer() to perform intent classification on the reconstructed multi-round text input History(text), obtain smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes, and calculate the possibilities of smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes; wherein, the calculation formula of the possibility vector of the smart home scenes, device operation scenes, tool call scenes and question-and-answer scenes is as follows: R1 represents the possibility vector of smart home scenario, device operation scenario, tool call scenario and question-answering scenario, SH represents smart home scenario, DC represents device operation scenario, Tools represents tool call scenario, and QA represents question-answering scenario; Respectively represent the probability rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios; S23, converting the text input text into a vector Input_Vector, and storing it in a preset vector library Categpry_Vector, and calculating the similarity between the text input text and the preset vector libraries Categpry_Vectors of the smart home scene, device operation scene, tool call scene and question-and-answer scene, and obtaining the possibility ranking R2. The calculation formula of the possibility ranking is as follows: in, They respectively represent the possibility rankings of smart home scenarios, device operation scenarios, tool calling scenarios, and question-and-answer scenarios.

3. The Multi-Agent based voice interaction method according to claim 2, characterized in that: The step S2 further comprises the following steps: S24, weighted average the possibility vectors R1 of smart home scenes, device operation scenes, tool call scenes and question-answering scenes and the possibility rankings R2 of smart home scenes, device operation scenes, tool call scenes and question-answering scenes to obtain the final classification scene R final , and according to the final classification scene R final The probability of judging which scene is the highest; the final classification scene R final The calculation formula is Among them, α and β represent weights.

4. The Multi-Agent based voice interaction method according to claim 3, characterized in that: The step S3 comprises the following steps: S31, if the final classification scene R final For the question-answering scenario, The weighted ranking value of the question-answering scenario is smaller than that of all other scenarios, which means that the question-answering scenario has the highest probability. final For question-answering scenarios and If Rag = = 1, then enable RAG mode for knowledge supplement; if the final classification scene R final For question-answering scenarios and If Rag = = 0, then the knowledge is supplemented by multi-engine crawler search. When the knowledge supplement is completed, a set of multi-round text input History (text) and supplementary knowledge Knowledge Final_Input will be output, where Final_Input = (History (text), Knowledge). Rag Indicates the mark bit.

5. The Multi-Agent based voice interaction method according to claim 4, characterized in that: The step S3 further comprises the following steps: S32, if the final classification scene R final Call the scene for the tool, i.e. If the weighted ranking value of the tool call scenario is less than that of all other scenarios, then the tool call scenario is most likely to be the tool call scenario; and the specified tool is called to obtain supplementary information. When the tool call is completed, a supplementary information Supplement_Info is returned, and finally a set Final_Input containing multiple rounds of text input History (text) and supplementary information Supplement_Info is generated.

6. The multi-agent based voice interaction method according to claim 5, characterized in that: The step S3 further comprises the following steps: S33, if the final classification scene R final For smart home scenarios, If the weighted ranking value of the tool call scenario is less than that of all other scenarios, it is most likely to be a smart home scenario. Determine whether it belongs to the smart home category. If it does, analyze the home devices that the user needs to call and the functions to be executed, and convert the user's needs into smart home call statements and execute them, encapsulated as SHAgent(). If it does not belong to the smart home category, reject it.

7. The Multi-Agent based voice interaction method according to claim 6, characterized in that: The step S3 further comprises the following steps: S34, if the final classification scene R final Manipulate the scene for the device, i.e. in, If the weighted ranking value of the device operation scenario is less than that of all other scenarios, the device operation scenario has the highest possibility; determine whether it belongs to the device operation category. If it does, analyze the device that the user needs to call and the function to be executed, and convert the user's needs into device call statements and execute them. If it does not belong to the device operation category, reject it.

8. The Multi-Agent based voice interaction method according to claim 7, characterized in that: In step S3, the knowledge input supplemented from step S31 to step S32 or the rejected multiple rounds of text input are input into the question-answering model, and the question-answering model generates the final text output in a streaming manner, and sends the text output to the TTS text-to-speech module in real time to generate a voice reply.

Images