A collaborative voice interaction system and method

Abstract

The present invention discloses a collaboration-oriented voice interaction system and method, which includes: a data acquisition and transmission module, which is used to obtain voice information collected by each user's voice interaction device, add corresponding collaboration tag information, and send it to a voice recognition module; a voice recognition module, which is used to receive voice information with collaboration tag information and recognize it as text information; a collaboration dialogue module, which is used to receive text information from the voice recognition module, and collaborate on the dialogue management of voice interaction between multiple people according to the text information and collaboration tag information, and generate natural language; a speech synthesis module, which is used to perform speech synthesis according to the natural language generated by the collaboration dialogue module, and output the synthesized speech through the voice interaction device of the corresponding collaboration personnel. The voice interaction system coordinates multiple people to complete a collaborative task together through multiple groups of dialogues with multiple people, making the entire collaborative interaction process more intelligent and efficient, and ultimately improving the office efficiency of the collaborative work environment.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a collaboration-oriented speech interaction system and method. Background Art

[0002] In recent years, human-computer voice interaction systems have developed rapidly in various fields including medical, smart home, and in-car voice. As an important way of human-computer interaction, voice interaction is being used more and more widely. However, most of the common voice interaction systems on the market are single-person voice interaction systems, and collaborative voice interaction systems for multi-person scenarios are still rare in real life.

[0003] The conversations in the voice interaction process are mainly divided into three categories: small talk, question-answer, and multi-round task-based conversations. Currently, the collaborative voice interaction systems mainly process multi-round task-based conversations, which are often processed by the intent recognition plus semantic slot extraction method.

[0004] This is mainly because the purpose of collaborative dialogue is generally to hope that the interactive system can complete some tasks or operations according to the user's own intentions. Task-based dialogue using intent recognition and semantic slot extraction is a widely used method for processing multi-round task-based dialogue, but this method still does not solve the problem of multi-person dialogue.

[0005] Compared with general voice interaction, the dialogue process of voice interaction for collaborative scenarios is more complex, with the characteristics of multiple people and multiple rounds.

[0006] The existing collaborative voice interaction system that uses the technical solution of intent recognition plus semantic slot extraction can only be used to handle multi-round task-based conversations with multiple intentions of a single person, but cannot handle multi-person scenarios, and cannot solve the situation where multiple groups of multi-round conversations coordinate with each other to handle a collaborative task. For example, in real life, the task of meeting reservation requires asking other users whether they want to attend the meeting after the meeting is created. Such collaborative tasks require multiple rounds of conversations with multiple people to complete. In the existing collaborative voice interaction system, this multi-person conversation operation is difficult to achieve.

[0007] In addition, the existing collaborative voice interaction system also lacks a dedicated collaborative knowledge graph to store and access collaborative related information. In the process of human-computer dialogue in the collaborative field, a lot of collaborative knowledge is often involved. In order to make the collaborative voice interaction system more natural and intelligent, it is very necessary to build a collaborative knowledge graph and integrate it into the collaborative field dialogue process.

[0008] The existence of the above two problems limits the functions and application scenarios of collaborative voice interaction systems. Therefore, practitioners in the same field are in urgent need of solving the above problems. Summary of the invention

[0009] The purpose of the present invention is to provide a collaborative voice interaction system and method that at least partially solves the above-mentioned technical problems, can better handle various complex voice interaction scenarios in a collaborative environment, complete task-based multi-round conversations in multi-person scenarios, and further expand the application scope of voice interaction in a collaborative environment.

[0010] To achieve the above object, the technical solution adopted by the present invention is:

[0011] In a first aspect, the present invention provides a collaboration-oriented voice interaction system, comprising: a data acquisition and transmission module, a voice recognition module, a collaboration dialogue module and a voice synthesis module;

[0012] The data collection and transmission module is used to obtain the voice information collected by each user voice interaction device, add corresponding collaboration tag information, and send it to the voice recognition module;

[0013] The speech recognition module is used to receive speech information with collaborative tag information and recognize it as text information;

[0014] The collaborative dialogue module is used to receive text information from the voice recognition module, and to coordinate the dialogue management of voice interaction between multiple people based on the text information and the collaborative tag information to generate natural language;

[0015] The speech synthesis module is used to perform speech synthesis based on the natural language generated by the collaborative dialogue module, and output the synthesized speech through the speech interaction device of the corresponding collaborative personnel.

[0016] Furthermore, the data acquisition and transmission module includes:

[0017] The acquisition module is used to obtain the recording permissions of each collaborative voice interaction device client and the audio source data in BLOB format;

[0018] A transcoding module, used for transcoding the audio source data in the BLOB format into a base64 format;

[0019] A sending module, used to send the base64 format data from the client to the server via the WebSocket protocol;

[0020] The marking module is used to transcode the base64 format data into wav format on the server side and mark the audio with collaborative tag information; the collaborative tag information includes: recording timestamp and user ID.

[0021] Furthermore, the collaborative dialogue module includes:

[0022] The natural language understanding module is used to obtain the dialogue intent and entity information in the sentence after word segmentation, vectorization, and neural network layer of the received text information; and transmit it to the collaborative dialogue management module together with the encoded collaborative tag information;

[0023] Collaborative dialogue management module, used to monitor dialogue status and select dialogue strategies;

[0024] The natural language generation module is used to obtain structured data based on the dialogue state information and dialogue strategy selection; and fill the structured data into the reply template to obtain natural language understandable to humans, and send it to the speech synthesis module.

[0025] Furthermore, the collaborative dialogue management module includes:

[0026] The dialogue state tracking submodule is used to receive entity information from the natural language understanding module and complete the dialogue slot filling according to preset rules;

[0027] The dialogue strategy learning submodule determines the next collaborative dialogue member that meets the preset conditions based on the rule-based atomic dialogue intention and collaborative member selector, and selects the action based on the transformer's action classifier.

[0028] Furthermore, the collaborative dialogue management module further includes: a dialogue knowledge base;

[0029] The dialogue knowledge base is used to communicate with the dialogue state tracking submodule and the dialogue strategy learning submodule respectively through an API interface to provide collaborative knowledge graph information and general domain knowledge information required to complete the dialogue.

[0030] Furthermore, the collaborative knowledge graph information is constructed in the following manner:

[0031] Analyze the entities, attributes and relationship information in the collaborative dialogue task through the natural language understanding module;

[0032] Decompose the collaborative dialogue task to obtain the basic elements that constitute the collaborative task, and provide a basic template for the design of the collaborative task;

[0033] Focusing on collaborative dialogue tasks, collaborative knowledge graph information is constructed in a top-down manner.

[0034] Furthermore, the dialogue state tracking submodule is also used to save the Action information in the dialogue history, the historical dialogue intention information and the collaborative tag information of each round of dialogue.

[0035] In a second aspect, the present invention also provides a collaborative voice interaction method, which uses the collaborative voice interaction system described in any of the above embodiments to implement dialogue management of voice interactions between multiple people and coordinate the entire collaborative voice interaction process.

[0036] Further, the method comprises the following steps:

[0037] S1. The data collection and transmission module obtains the voice information collected by each user's voice interaction device, adds the corresponding collaborative tag information, and sends it to the voice recognition module;

[0038] S2, the speech recognition module receives the speech information with the collaborative tag information and recognizes it as text information;

[0039] S3, the collaborative dialogue module receives the text information of the speech recognition module, and collaborates with multiple people in voice interaction dialogue management according to the text information and the collaborative tag information to generate natural language;

[0040] S4. The speech synthesis module performs speech synthesis according to the natural language generated by the collaborative dialogue module, and outputs the synthesized speech through the speech interaction device of the corresponding collaborative personnel.

[0041] Furthermore, the step S3 includes the following sub-steps:

[0042] S31, the collaborative dialogue module receives the text information from the speech recognition module, and analyzes the collaborative dialogue tasks existing in the collaborative scenario according to the text information;

[0043] S32, decomposing the collaborative dialogue task to obtain basic elements constituting the collaborative dialogue task;

[0044] S33, executing the preset system operation of the atomic dialogue, and updating the collaborative dialogue state information to the dialogue state tracking submodule;

[0045] S34, according to the current state, searching for the next collaborative dialogue member that meets the preset conditions;

[0046] S35, analyzing the task to be completed by the next collaborative dialogue member, calling up the voice interaction device corresponding to the member, and initiating the collaborative dialogue task;

[0047] S36: After all the collaborative dialogue members meeting the preset conditions are traversed, the collaborative dialogue ends and the collaborative dialogue status information is cleared.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The embodiment of the present invention provides a collaboration-oriented voice interaction system, which coordinates multiple people to complete a collaborative task together through multiple groups of dialogues with multiple people, making the entire collaborative interaction process more intelligent and efficient, and ultimately improving the office efficiency of the collaborative work environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 An architecture diagram of a collaborative voice interaction system provided for an embodiment of the present invention.

[0051] Figure 2 This is a flow chart of the data acquisition and transmission module of the voice interaction system according to an embodiment of the present invention.

[0052] Figure 3 Schematic diagram of the collaborative dialogue module structure of the voice interaction system of an embodiment of the present invention.

[0053] Figure 4 A schematic diagram of a collaborative knowledge graph of a voice interaction system according to an embodiment of the present invention.

[0054] Figure 5 A schematic diagram of the collaborative dialogue management process according to an embodiment of the present invention.

[0055] Figure 6 This is an example diagram of a conversation involving a multi-person collaboration scenario according to an embodiment of the present invention.

[0056] Figure 7 The figure is a flow chart of a collaborative voice interaction method according to an embodiment of the present invention.

[0057] Figure 8 Flow chart of the collaborative dialogue module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0059] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0060] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] Embodiment 1:

[0062] Reference Figure 1 As shown, a collaborative voice interaction system provided by the present invention includes: a data acquisition and transmission module, a voice recognition module, a collaborative dialogue module and a voice synthesis module;

[0063] Among them, the data acquisition and transmission module is used to obtain the voice information collected by each user's voice interaction device, add the corresponding collaborative tag information, and send it to the voice recognition module; the voice recognition module is used to receive the voice information with collaborative tag information and identify it as text information; the collaborative dialogue module is used to receive the text information of the voice recognition module, and according to the text information and the collaborative tag information, collaborate on the dialogue management of voice interaction between multiple people to generate natural language; the speech synthesis module is used to perform speech synthesis according to the natural language generated by the collaborative dialogue module, and output the synthesized speech through the voice interaction device of the corresponding collaborative personnel.

[0064] Collaboration members interact with the system in the form of voice through their respective client devices to complete various collaborative tasks. The speech recognition module and speech synthesis module are implemented using a mature end-to-end framework, which will not be described here.

[0065] The overall process of the data acquisition and transmission module is as follows: Figure 2 As shown, the function is to obtain the voice information collected by each user voice interaction device, add timestamp, user ID and other label information to the audio data, and send it to the voice recognition module. It includes:

[0066] The acquisition module is used to obtain the recording permissions of each collaborative voice interaction device client and the audio source data in BLOB format;

[0067] A transcoding module, used for transcoding the audio source data in the BLOB format into a base64 format;

[0068] A sending module, used to send the base64 format data from the client to the server via the WebSocket protocol;

[0069] The marking module is used to transcode the base64 format data into wav format on the server side and mark the audio with collaborative tag information; the collaborative tag information includes: recording timestamp and user ID.

[0070] like Figure 3 As shown in the figure, the collaborative dialogue module is the core of the entire collaborative voice interaction system. Its role in the collaborative dialogue process is to monitor the dialogue status and control the entire collaborative human-computer dialogue process. On this basis, it executes appropriate strategies to make the dialogue proceed efficiently towards the task goal and finally complete the collaborative dialogue task specified by the collaborative members.

[0071] The collaborative dialogue module splits the tasks into three parts: natural language understanding sub-module, collaborative dialogue management sub-module and natural language generation sub-module.

[0072] The function of the natural language understanding module is to convert natural language into structured language that the system can understand. First, it receives the natural language text data and the collaborative tag information describing the sentence, including user ID, timestamp, etc., from the data acquisition and transmission module. For the natural language text data, after word segmentation, vectorization, and neural network layer, the dialogue intent and entity information in the sentence are obtained, and together with the encoded collaborative tag information, it is passed to the collaborative dialogue management submodule.

[0073] The collaborative dialogue management module monitors the dialogue state and selects dialogue strategies. According to the different tasks it needs to complete, it can be further divided into a dialogue state tracking submodule, a dialogue strategy learning submodule and a dialogue knowledge base.

[0074] The dialogue state tracking submodule receives entity information from the natural language understanding module and uses it to complete the dialogue slot filling. In addition, action information in the dialogue history, historical dialogue intention information, and collaborative tag information for each round of dialogue are also stored in it for the dialogue strategy learning submodule to generate reasonable dialogue decisions for the next round of dialogue.

[0075] In a single-person conversation, the dialogue strategy learning submodule only needs to decide the next action to be executed, and an action classifier can solve the problem. However, in a multi-person collaborative conversation, because the number of people is added, it is necessary to first determine who the action recipient is before making an action decision. Therefore, a rule-based atomic dialogue intention and collaborative member selector are added to first solve the problem of who to talk to, and then select the action.

[0076] Finally, the dialogue knowledge base submodule provides the collaborative knowledge information and general domain knowledge information required to complete the dialogue in the form of API according to the needs of the collaborative dialogue management module, helping the system to better understand the user's intention and complete the entire dialogue.

[0077] The natural language generation module fills the output structured data into the reply template to obtain natural language understandable to humans and sends it to the speech synthesis module.

[0078] like Figure 4 The collaborative knowledge graph shown is Figure 3 The part of the dialogue knowledge base shown in the figure is constructed in a top-down manner based on the task-centric idea. The collaborative task entity is the center of the collaborative knowledge graph, on which five entities are extended: task subject, task time, task behavior, task location, and task participant. Among them, task participants include two types: individual personnel and small groups. The relationship between the entities is shown in the figure. The box shows a part of the attribute information contained in each entity, which can be expanded on this basis according to the actual needs. Provide the collaborative knowledge information and general domain knowledge information required to complete the dialogue in the form of an API according to the needs of the collaborative dialogue management module. Assist the system to better understand the user's intentions and complete the entire dialogue.

[0079] The above-mentioned knowledge graph for the collaboration field provides domain-related knowledge support for collaboration-oriented voice interaction systems and devices.

[0080] Knowledge graph, also known as semantic network, is a form of knowledge representation consisting of entities, concepts, attributes, relationships, values, etc. Domain knowledge graph evolved from general knowledge graph. For example, collaborative domain knowledge graph needs to use its own domain-related knowledge to improve the functionality and efficiency of collaborative dialogue.

[0081] The current construction of domain knowledge graphs is concentrated in certain popular fields, such as finance, medical care, e-commerce, etc. However, in some other fields such as collaboration, the construction of knowledge graphs is less common.

[0082] The embodiment of the present invention solves the problem of lack of knowledge graph in the field of collaboration in task-based dialogue. Based on the actual collaboration scenario, a top-down approach is adopted to first construct the collaborative knowledge graph model layer according to the seven-step method proposed by the Stanford University School of Medicine, and then fill the structured collaboration-related data into the knowledge graph data layer. The constructed collaborative knowledge graph is as follows: Figure 4 Finally, the knowledge graph provides collaborative information support for collaborative dialogue in the form of API.

[0083] like Figure 5The figure shows a schematic diagram of the collaborative dialogue management process. First, user A wakes up the interactive device and initiates a collaborative task. The system first determines whether the intention is a collaborative intention. If it is a collaborative intention, the user first fills in various information required for the collaborative task, such as task time, location, and participating members, according to the system's dialogue guidance. Then the system operation specified by the user is executed, and the collaborative dialogue state information update is sent to the dialogue state tracking submodule of the collaborative dialogue management module. The dialogue state tracking submodule receives the information and updates its own collaborative state information. Then the dialogue strategy selection submodule searches for the next collaborative dialogue member and analyzes the atomic dialogue task that the member needs to perform based on the current state. The system wakes up the voice assistant of the member and initiates the collaborative task. If all collaborative members are traversed and no member that needs to perform the collaborative task is found, the collaborative dialogue ends and the collaborative dialogue state information is cleared; if it is a non-collaborative single-person intention, the collaborative dialogue module uses the information in the general domain knowledge graph to collaboratively complete the dialogue, and there is no need to make decisions on dialogue intentions and dialogue members.

[0084] like Figure 6 As shown in the figure, taking the task of booking a meeting as an example, a complete collaborative dialogue process in a multi-person scenario is demonstrated. Complex multi-person multi-round collaborative voice interaction tasks can be completed. First, the group manager G1 Manager initiates the task of booking a meeting. After the steps of intent initiation, slot filling, and information confirmation, the new meeting is successfully added to the conference management system, and a round of atomic dialogue is completed. After the intention is completed, the collaborative dialogue management module updates the dialogue status information. The atomic dialogue intent and the collaborative member selector select the collaborative participant G1Member1 and the meeting confirmation intent according to the pre-set rules and the collaborative knowledge information in the collaborative knowledge graph, and start a new round of collaborative atomic dialogue. Finally, after completing the dialogue with the last participant G1 Memeber n, all participants have received the meeting information and confirmed it, and the entire collaborative dialogue ends.

[0085] A collaborative voice interaction system provided by an embodiment of the present invention creatively solves the problem of dialogue management in multi-person collaborative scenarios. On the basis of the traditional single-person multi-round task-based dialogue, a rule-based atomic dialogue intention and a collaborative member selector are added for the selection of collaborative dialogue members and atomic collaboration intentions. Each group of single-person task-based dialogues is regarded as an atomic dialogue in a multi-person collaborative dialogue process, and multiple single-person atomic dialogues are carried out in a certain order, and finally constitute the entire collaborative dialogue process to complete the collaborative dialogue task specified by the user. In addition to the rule-based decision-making method, atomic dialogue strategy selection methods such as finite state machine-based methods and reinforcement learning-based methods can also be used.

[0086] This collaborative voice interaction system, combined with the collaborative domain knowledge graph, can better handle various complex voice interaction scenarios in collaborative environments, complete task-based multi-round conversations in multi-person scenarios, and expand the application of voice interaction from single-person scenarios to multi-person collaborative scenarios, further expanding the scope of application of voice interaction in collaborative environments. The collaborative voice interaction system coordinates multiple groups of conversations with multiple people to complete a collaborative task together, making the entire collaborative interaction process more intelligent and efficient, and ultimately improving the office efficiency of the collaborative work environment.

[0087] Embodiment 2:

[0088] The present invention also provides a collaborative voice interaction method, using the collaborative voice interaction system of Example 1 to achieve dialogue management of voice interaction between multiple people and coordinate the entire collaborative voice interaction process. Figure 7 As shown, the method comprises the following steps:

[0089] S1. The data collection and transmission module obtains the voice information collected by each user's voice interaction device, adds the corresponding collaborative tag information, and sends it to the voice recognition module;

[0090] S2, the speech recognition module receives the speech information with the collaborative tag information and recognizes it as text information;

[0091] S3, the collaborative dialogue module receives the text information of the speech recognition module, and collaborates with multiple people in voice interaction dialogue management according to the text information and the collaborative tag information to generate natural language;

[0092] S4. The speech synthesis module performs speech synthesis according to the natural language generated by the collaborative dialogue module, and outputs the synthesized speech through the speech interaction device of the corresponding collaborative personnel.

[0093] Among them, Figure 8 As shown, step S3 includes:

[0094] Step S3 includes the following sub-steps:

[0095] S31, the collaborative dialogue module receives the text information from the speech recognition module, and analyzes the collaborative dialogue tasks existing in the collaborative scenario according to the text information;

[0096] S32, decomposing the collaborative dialogue task to obtain basic elements constituting the collaborative dialogue task;

[0097] S33, executing the preset system operation of the atomic dialogue, and updating the collaborative dialogue state information to the dialogue state tracking submodule;

[0098] S34, according to the current state, searching for the next collaborative dialogue member that meets the preset conditions;

[0099] S35, analyzing the task to be completed by the next collaborative dialogue member, calling up the voice interaction device corresponding to the member, and initiating the collaborative dialogue task;

[0100] S36: After all the collaborative dialogue members meeting the preset conditions are traversed, the collaborative dialogue ends and the collaborative dialogue status information is cleared.

[0101] In this embodiment, by first analyzing the collaborative dialogue tasks existing in the collaborative scenario, analyzing the entities, attributes and relationships existing in the collaborative field, and constructing a collaborative information knowledge graph, this is used as the data basis for the collaborative voice interaction system; then the collaborative tasks are decomposed to obtain the basic elements that constitute the collaborative tasks: task start time, task end time, task location, participants, etc., to provide a basic template for the design of collaborative tasks; finally, a collaborative dialogue management module is added on top of the traditional dialogue management module for single-person task-based dialogues, which is used for dialogue management of voice interactions between multiple people, and expands human-computer voice interaction from single-person scenarios to multi-person collaborative scenarios. It can coordinate the entire collaborative voice interaction process well, making the entire collaborative interaction process more intelligent and efficient, and ultimately improving the office efficiency of the collaborative work environment.

[0102] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A collaborative voice interaction system, characterized in that: include: Data collection and transmission module, speech recognition module, collaborative dialogue module and speech synthesis module; The data collection and transmission module is used to obtain the voice information collected by each user voice interaction device, add corresponding collaboration tag information, and send it to the voice recognition module; The speech recognition module is used to receive speech information with collaborative tag information and recognize it as text information; The collaborative dialogue module is used to receive text information from the voice recognition module, and to coordinate the dialogue management of voice interaction between multiple people based on the text information and the collaborative tag information to generate natural language; The speech synthesis module is used to perform speech synthesis according to the natural language generated by the collaborative dialogue module, and output the synthesized speech through the speech interaction device of the corresponding collaborative personnel; The collaborative dialogue module includes: The natural language understanding module is used to obtain the dialogue intent and entity information in the sentence after word segmentation, vectorization, and neural network layer of the received text information; and transmit it to the collaborative dialogue management module together with the encoded collaborative tag information; The collaborative dialogue management module is used to monitor the dialogue state and select dialogue strategies. Specifically, it includes: a dialogue state tracking submodule, which receives entity information from the natural language understanding module and completes the dialogue slot filling according to preset rules; a dialogue strategy learning submodule, which determines the next collaborative dialogue member that meets the preset conditions based on the rule-based atomic dialogue intention and collaborative member selector; and selects the action based on the transformer action classifier; A natural language generation module is used to obtain structured data according to the dialogue state information and dialogue strategy selection; and fill the structured data into the reply template to obtain natural language understandable to humans, and send it to the speech synthesis module; Also included: conversation knowledge base; The dialogue knowledge base is used to communicate with the dialogue state tracking submodule and the dialogue strategy learning submodule through an API interface to provide collaborative knowledge graph information and general domain knowledge information required to complete the dialogue; The collaborative knowledge graph information is constructed in the following manner: Analyze the entities, attributes and relationship information in the collaborative dialogue task through the natural language understanding module; Decompose the collaborative dialogue task to obtain the basic elements that constitute the collaborative task, and provide a basic template for the design of the collaborative task; Focusing on collaborative dialogue tasks, collaborative knowledge graph information is constructed in a top-down manner.

2. A collaborative voice interaction system according to claim 1, characterized in that: The data acquisition and transmission module comprises: The acquisition module is used to obtain the recording permissions of each collaborative voice interaction device client and the audio source data in BLOB format; A transcoding module, used for transcoding the audio source data in the BLOB format into a base64 format; A sending module, used to send the base64 format data from the client to the server via the WebSocket protocol; The marking module is used to transcode the base64 format data into wav format on the server side and mark the audio with collaborative tag information; the collaborative tag information includes: recording timestamp and user ID.

3. A collaborative voice interaction system according to claim 1, characterized in that: The dialogue state tracking submodule is also used to save the Action information in the dialogue history, the historical dialogue intention information and the collaborative tag information of each round of dialogue.

4. A collaborative voice interaction method, characterized in that: By using the collaborative voice interaction system according to any one of claims 1 to 3, dialogue management of voice interaction between multiple people is implemented to coordinate the entire collaborative voice interaction process.

5. A collaborative voice interaction method according to claim 4, characterized in that: The method comprises the following steps: S1. The data collection and transmission module obtains the voice information collected by each user's voice interaction device, adds the corresponding collaborative tag information, and sends it to the voice recognition module; S2, the speech recognition module receives the speech information with the collaborative tag information and recognizes it as text information; S3, the collaborative dialogue module receives the text information of the speech recognition module, and collaborates with multiple people in voice interaction dialogue management according to the text information and the collaborative tag information to generate natural language; S4. The speech synthesis module performs speech synthesis according to the natural language generated by the collaborative dialogue module, and outputs the synthesized speech through the speech interaction device of the corresponding collaborative personnel.

6. A collaborative voice interaction method according to claim 5, characterized in that: The step S3 comprises the following sub-steps: S31, the collaborative dialogue module receives the text information from the speech recognition module, and analyzes the collaborative dialogue tasks existing in the collaborative scenario according to the text information; S32, decomposing the collaborative dialogue task to obtain basic elements constituting the collaborative dialogue task; S33, executing the preset system operation of the atomic dialogue, and updating the collaborative dialogue state information to the dialogue state tracking submodule; S34, according to the current state, searching for the next collaborative dialogue member that meets the preset conditions; S35, analyzing the task to be completed by the next collaborative dialogue member, calling up the voice interaction device corresponding to the member, and initiating the collaborative dialogue task; S36: After all the collaborative dialogue members meeting the preset conditions are traversed, the collaborative dialogue ends and the collaborative dialogue status information is cleared.

Images