Multimodal agent conference information interaction method and device, equipment and medium

Abstract

The multi-modal agent conference information interaction method, device and equipment and the medium provided by the embodiment of the invention comprise the following steps: in response to a received conference request sent by a target object, determining a target agent group according to the conference request; sending a conference joining request to each target agent, so that the target object establishes conference communication with each target agent; according to the first label identifier of each target agent and the conference process, sequentially outputting a control instruction to each target agent, and obtaining response data of each target agent after receiving the control instruction; and the conference report is generated according to the response data of each target agent, so that the accuracy of the finally generated decision suggestion is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology and related technical fields, and in particular to a method, device, equipment and medium suitable for multimodal intelligent agent conference information interaction. Background Art

[0002] With the rapid development of artificial intelligence technology, multimodal AI agent systems have gradually become a research hotspot. Traditional AI systems are mainly based on text interaction and have difficulty processing input and output of multiple modalities such as voice, image, and video. At the same time, most existing AI systems use asynchronous communication methods and cannot achieve real-time and smooth human-computer interaction experience.

[0003] WebRTC (Web Real-Time Communication) is a technology that supports web browsers for real-time voice or video conversations. It allows web applications to directly connect between browsers to achieve functions such as video conferencing and file transfer without the need for an intermediary. Due to the widespread use of this real-time communication technology, it has now been extended to other client applications that support WebRTC technology. However, WebRTC technology is currently mainly used for communication between people, and its application in combination with multimodal AI agents has not yet been fully explored. Summary of the invention

[0004] The embodiments described in this article provide a multimodal intelligent agent conference information interaction method, device, equipment and medium to solve the problems existing in the prior art.

[0005] In a first aspect, according to the content of the present disclosure, a multimodal agent conference information interaction method is provided, comprising: In response to receiving a meeting request sent by a target object, determining a target agent group according to the meeting request, wherein the target object includes a user and an agent, the meeting request includes a meeting topic and the number of participants, the target agent group includes a plurality of target agents, and the number of the target agents is the same as the number of participants; Sending a conference joining request to each of the target intelligent entities, so that the target object establishes conference communication with each of the target intelligent entities; Output control instructions to each of the target agents in sequence according to the first tag identifiers of each of the target agents and the conference process, and obtain response data of each of the target agents after receiving the control instructions; A meeting report is generated according to the response data of each target intelligent agent, wherein the meeting report includes a meeting summary, decision recommendations and action recommendations.

[0006] In some embodiments of the present disclosure, in response to receiving a meeting request sent by a target object, determining a target agent group according to the meeting request includes: In response to receiving a meeting request sent by a target object, determining a target professional field to which the meeting request belongs according to the meeting theme; Acquire the second tag identifier of each agent, the response efficiency of each agent, and the state information of each agent, wherein the state information includes an idle state and a busy state; The target intelligent agent is determined according to the matching degree between the label content corresponding to the second label identification of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent.

[0007] In some embodiments of the present disclosure, determining the target intelligent agent according to the matching degree between the tag content corresponding to the second tag identifier of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent includes: Sequentially obtain a matching value corresponding to the matching degree between the tag content corresponding to the second tag identifier of each of the agents and the target professional field, an efficiency value corresponding to the response efficiency of each agent, and a state value corresponding to the state information of each agent; Determine the scoring parameters of each of the agents according to the matching value corresponding to the tag content corresponding to the second tag identifier of each of the agents and the matching degree of the target professional field, the efficiency value corresponding to the response efficiency of each of the agents, and the state value corresponding to the state information of each of the agents; After the scoring parameters of each of the agents are sorted in forward order, a target number of agents are selected to form a target agent group, wherein the target number is the same as the number of participants.

[0008] In some embodiments of the present disclosure, outputting control instructions to each target agent in sequence according to the first tag identifier of each target agent and the conference process, and obtaining response data of each target agent after receiving the control instruction, includes: Determine the role of each target intelligent agent in the meeting according to the first tag identifier of each target intelligent agent and the proportional coefficient corresponding to each first tag identifier, wherein the role includes at least a host, an observer and a speaker; According to the role of each target intelligent agent in the meeting, the target intelligent agent whose role is the host is controlled to output control instructions to the other target intelligent agents except the host in sequence according to the meeting process, so that each target intelligent agent whose role is the speaker generates a meeting answer corresponding to the meeting question after receiving the control instruction, and each target intelligent agent whose role is the observer generates a meeting minutes corresponding to the meeting answer after receiving the control instruction, wherein the meeting process includes the speaking order of the target intelligent agent whose role is the speaker and the meeting question corresponding to the target intelligent agent whose role is the speaker; Acquire response data of each target agent after receiving the control instruction, wherein the response data includes meeting answers and meeting minutes.

[0009] In some embodiments of the present disclosure, generating a conference report according to the response data of each target agent includes: Obtaining the meeting answers of the target intelligent agent whose role in the meeting is a speaker to each of the meeting questions, and the meeting minutes of the target intelligent agent whose role in the meeting is an observer to each of the meeting questions; Generate the meeting report according to the meeting process, the meeting answers and the meeting minutes.

[0010] In some embodiments of the present disclosure, before determining the target agent group according to the meeting request in response to receiving the meeting request sent by the target object, the method further includes: In response to receiving the communication request of each intelligent agent, receiving the registration information of each intelligent agent, and storing the registration information of each intelligent agent, wherein the registration information includes device condition information and IP address.

[0011] In some embodiments of the present disclosure, the method further includes: Acquire the conference data of each target intelligent agent, and update the database of each target intelligent agent.

[0012] In a second aspect, according to the present disclosure, a multimodal agent conference information interaction device is provided, comprising: A target agent determination module, configured to determine a target agent group according to a meeting request received from a target object, wherein the target object includes a user and an agent, the meeting request includes a meeting topic and the number of participants, the target agent group includes a plurality of target agents, and the number of the target agents is the same as the number of participants; A communication establishment module, used for sending a conference joining request to each of the target intelligent entities, so that the target object establishes conference communication with each of the target intelligent entities; A control module, used to output control instructions to each of the target agents in sequence according to the first tag identification and the conference process of each of the target agents, and obtain response data of each of the target agents after receiving the control instructions; A meeting report generation module is used to generate a meeting report based on the response data of each target intelligent agent, wherein the meeting report includes a meeting summary, decision recommendations and action recommendations.

[0013] In a third aspect, according to the present disclosure, there is provided a computer device, including: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement any method as described in the first aspect.

[0014] In a fourth aspect, according to the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any method as described in the first aspect. The multimodal agent conference information interaction method, device, equipment and medium provided by the disclosed embodiment, in response to receiving a conference request sent by the target object, determines the target agent group according to the conference request; sends a request to join the meeting to each target agent, so that the target object and each target agent establish conference communication; according to the first tag identification and conference process of each target agent, outputs control instructions to each target agent in sequence, and obtains the response data of each target agent after receiving the control instruction; generates a conference report according to the response data of each target agent. Since the target agents are selected based on the conference theme, the target agents participating in this conference have a close correlation with the content discussed in this conference, thereby improving the accuracy of the decision recommendations finally generated. In addition, in the process of outputting control instructions to the target agents, based on the first tag identification of each target agent, the role of each target agent in the conference is determined, so that each agent plays the greatest role in the conference, further ensuring the accuracy of the generated decision recommendations.

[0015] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure, wherein: Figure 1 It is a flowchart of a multi-modal intelligent agent conference information interaction method provided by an embodiment of the present disclosure; Figure 2 It is a structural diagram of a multi-modal intelligent agent conference information interaction device provided by an embodiment of the present disclosure; Figure 3 It is a structural diagram of a computer device provided in an embodiment of the present disclosure.

[0017] In the drawings, reference numerals with the same last two digits correspond to the same elements. It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0019] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by a person skilled in the art to which the subject matter of the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, a statement that two or more parts are "connected" or "coupled" together shall mean that the parts are joined together directly or through one or more intermediate components.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiments" in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists, A and B exist at the same time, and B exists. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0022] Furthermore, in all embodiments of the present disclosure, terms such as “first” and “second” are used only to distinguish one component (or a part of a component) from another component (or another part of a component).

[0023] In the description of the present application, unless otherwise specified, “plurality” means more than two (including two), and similarly, “plurality groups” means more than two (including two).

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0025] Based on the problems existing in the prior art, the present disclosure provides a multi-modal intelligent agent conference information interaction method. Figure 1 is a flow chart of a multi-modal agent conference information interaction method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the multimodal agent conference information interaction method includes: S110. In response to receiving a meeting request sent by a target object, determine a target agent group.

[0026] The target objects include users and agents, the meeting request includes the meeting topic and the number of participants, the target agent group includes multiple target agents, and the number of target agents is the same as the number of participants.

[0027] Among them, the agent is a multimodal agent. A multimodal agent refers to an intelligent system or entity that can process and understand multiple different types of data, which may include but are not limited to text, images, audio, video, etc. The key feature of a multimodal agent is that it can integrate different types of information to obtain a more comprehensive and in-depth understanding, thereby making more intelligent decisions or responses.

[0028] In a specific implementation, the signaling server stores registration information of multiple agents, the registration information of the agents includes device condition information, communication channels, and IP addresses, etc. The agents register the registration information to the signaling server through the WebRTC communication protocol.

[0029] That is, before executing step S110, the signaling server receives registration information sent by multiple agents through the WebRTC communication protocol, and stores the received registration information of each agent. Therefore, the signaling server stores the registration information of multiple agents, and when the signaling server receives a meeting request sent by the target object, according to the meeting topic included in the meeting request, multiple target agents with the same number as the participants are selected from the signaling server to form a target agent group.

[0030] As a specific implementation method, in response to receiving a meeting request sent by a target object, the target professional field to which the meeting request belongs is determined according to the meeting theme; the second tag identifier of each intelligent agent, the response efficiency of each intelligent agent and the status information of each intelligent agent are obtained, wherein the second tag identifier is used to identify the professional field to which the intelligent agent belongs, and the status information includes an idle state and a busy state; the target intelligent agent is determined according to the matching degree between the tag content corresponding to the second tag identifier of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent.

[0031] Among them, the target intelligent subject is determined according to the matching degree between the label content corresponding to the second label identification of each intelligent subject and the target professional field, the response efficiency of each intelligent subject and the status information of each intelligent subject, including: sequentially obtaining the matching value corresponding to the matching degree between the label content corresponding to the second label identification of each intelligent subject and the target professional field, the efficiency value corresponding to the response efficiency of each intelligent subject and the status value corresponding to the status information of each intelligent subject; the scoring parameter of each intelligent subject is determined according to the matching value corresponding to the matching degree between the label content corresponding to the second label identification of each intelligent subject and the target professional field, the efficiency value corresponding to the response efficiency of each intelligent subject and the status value corresponding to the status information of each intelligent subject; after the scoring parameters of each intelligent subject are sorted in forward order, the first target number of intelligent subjects are selected to form a target intelligent subject group, wherein the target number is the same as the number of participants.

[0032] Specifically, when the signaling server receives a meeting request sent by the target object, it first determines the target professional field to which the meeting belongs based on the meeting theme included in the meeting request, and then obtains the second label identification of each intelligent agent, the response efficiency of each intelligent agent and the status information of each intelligent agent, and then determines the matching value corresponding to the matching degree between the label content corresponding to the second label identification of each intelligent agent and the target professional field, the efficiency value corresponding to the response efficiency of each intelligent agent and the status value corresponding to the status information of each intelligent agent, and then obtains the scoring parameters of each intelligent agent through the matching value, efficiency value and status value, and finally determines the target intelligent agent through the scoring parameters.

[0033] Exemplarily, the agents stored in the signaling server include agent 1, agent 2, agent 3, ..., agent N, the second label identifier corresponding to agent 1 includes label identifier 1, label identifier 2 and label identifier 3, the efficiency value corresponding to the response efficiency of agent 1 is a%, and the status value corresponding to the status information of agent 1 is 1, the second label identifier corresponding to agent 2 includes label identifier 1, label identifier 3 and label identifier 4, the efficiency value corresponding to the response efficiency of agent 2 is b%, and the status value corresponding to the status information of agent 1 is 1, ..., the second label identifier corresponding to agent N includes label identifier 1, label identifier 2 and label identifier i, the efficiency value corresponding to the response efficiency of agent N is c%, and the status value corresponding to the status information of agent N is 1. At this time, first, the label content corresponding to the label identifier of agent 1 is matched with the target professional field, that is, if the second label identifier of agent 1 includes label identifier 1, label identifier 2 and label identifier 3, the label content corresponding to label identifier 1 is content 1, the label content corresponding to label identifier 2 is content 2, and the label content corresponding to label identifier 3 is content 3. Content 1 is matched with the target professional field in turn to obtain a first matching value, content 2 is matched with the target professional field to obtain a second matching value, and content 3 is matched with the target professional field to obtain a third matching value. The first matching value, the second matching value and the third matching value are then averaged to obtain the matching value of the label content corresponding to the second label identifier of agent 1 and the target professional field, and so on, to obtain the matching value of the label content corresponding to the second label identifier of other agents and the target professional field; then, the matching value, efficiency value and state value of agent 1 are weighted and summed to obtain the scoring parameters of each agent.

[0034] Exemplarily, the scoring parameters of Agent 1 satisfy: , in, is the weighted value of the matching value, is the weighted value of the efficiency value, is the weighted value of the state value, is the first matching value between content 1 and the target professional field, is the second matching value between content 2 and the target professional field, It is the third matching value between content 3 and the target professional field.

[0035] If the state information of the agent is an idle state, the state value corresponding to the state information of the agent is 1; if the state information of the agent is a busy state, the state value corresponding to the state information of the agent is 0.

[0036] After obtaining the scoring parameters of each agent, the scoring parameters of each agent are sorted in forward order, that is, after sorting the agent scoring parameters from large to small, the agents with the top target number in the forward sorting are selected to form a target agent group, ensuring that the target agents included in the determined target agent group can have a better reference basis for the meeting answers generated based on the meeting questions in the meeting constructed by the target object.

[0037] S120: Send a conference joining request to each target intelligent agent, so that the target object can establish conference communication with each target intelligent agent.

[0038] After screening out the target intelligent agents, the signaling server sends a conference joining request to each target intelligent agent so that each target intelligent agent enters the conference created by the target object, and the target intelligent agent can conduct conference communication with each target object in the conference created by the target object.

[0039] In addition, the target object may also request other users to attend the meeting, which is not specifically limited in the embodiments of the present disclosure.

[0040] S130, outputting control instructions to each target intelligent agent in sequence according to the first tag identification of each target intelligent agent and the conference process, and acquiring response data of each target intelligent agent after receiving the control instructions.

[0041] After the target agents are screened out in step S110 and each target agent is required to join the meeting in step S120, the first tag of each target agent can be used to determine the role of each target agent in the meeting, and according to the meeting process and the role of each target agent in the meeting, control instructions are output to each target agent to obtain response data of each target agent, so as to facilitate the generation of a meeting report based on the response data of each target agent in step S140.

[0042] As a specific implementation method, after the signaling server screens out the target intelligent entities, the role of each target intelligent entity in the current meeting can be determined by obtaining the first tag identifier of each target intelligent entity, wherein the first tag identifier of the target intelligent entity is used to identify the role of the target intelligent entity in the historical meeting. Then, according to the meeting process, control instructions are sent to the target intelligent entities playing different roles in the meeting in turn to obtain the response data of each target intelligent entity after receiving the control instructions.

[0043] Specifically, according to the first label identification of each target intelligent agent and the meeting process, control instructions are output to each target intelligent agent in sequence, and response data of each target intelligent agent after receiving the control instructions is obtained, including: according to the first label identification of each target intelligent agent and the proportional coefficient corresponding to each first label identification, the role of each target intelligent agent in the meeting is determined, wherein the roles include at least a host, an observer and a speaker; according to the role of each target intelligent agent in the meeting, the target intelligent agent whose role is the host is controlled to output control instructions to other target intelligent agents except the host in sequence according to the meeting process, so that each target intelligent agent whose role is the speaker generates meeting answers corresponding to meeting questions after receiving the control instructions, and each target intelligent agent whose role is the observer generates meeting minutes corresponding to the meeting answers after receiving the control instructions, wherein the meeting process includes the speaking order of the target intelligent agent whose role is the speaker and the meeting questions corresponding to the target intelligent agent whose role is the speaker; the response data of each target intelligent agent after receiving the control instructions is obtained, and the response data includes meeting answers and meeting minutes.

[0044] Specifically, in a meeting, the signaling server can set the intelligent agent configured by the signaling server itself as the host of the meeting, and the signaling server can also set any target intelligent agent in the target intelligent agent group as the host of the meeting. In the embodiment of the present disclosure, a specific example is given of the signaling server setting any target intelligent agent in the target intelligent agent group as the host of the meeting.

[0045] After all target agents enter the meeting in step S120, the signaling server obtains the role of each target agent in the historical meeting, that is, the role of each target agent in the historical meeting is determined based on the first label identification of the target agent. The first label identification of each target agent includes a host label identification, an observer label identification and a speaker label identification. The content corresponding to the first label identification is the proportion coefficient of each target agent playing different roles. The higher the proportion coefficient, the more times the target agent plays the role in the historical meeting. After determining the proportion coefficient corresponding to each first label identification of each target agent, the role of each target agent in the meeting is determined according to the proportion coefficient corresponding to each first label identification of each target agent. For example, if the target agent group includes target agent 1, target agent 2, target agent 3 and target agent 4, the proportion corresponding to the host label identification of target agent 1 is The coefficient is 0.7, the proportional coefficient corresponding to the speaker label is 0.1, and the proportional coefficient corresponding to the observer label is 0.2. The proportional coefficient corresponding to the host label of target agent 2 is 0.1, the proportional coefficient corresponding to the speaker label is 0.6, and the proportional coefficient corresponding to the observer label is 0.3. The proportional coefficient corresponding to the host label of target agent 3 is 0.7, the proportional coefficient corresponding to the speaker label is 0.1, and the proportional coefficient corresponding to the observer label is 0.2. The proportional coefficient corresponding to the host label of target agent 4 is 0.3, the proportional coefficient corresponding to the speaker label is 0.2, and the proportional coefficient corresponding to the observer label is 0.5. Then the role of target agent 1 in the meeting is selected as the host, the roles of target agent 2 and target agent 3 in the meeting are selected as speakers, and the role of target agent 4 in the meeting is selected as the observer.

[0046] After determining the role of each target agent in the meeting, first, the target agent with the role of the host is controlled to output control instructions to each target agent with the role of the speaker in sequence according to the meeting process, so that the target agent with the role of the speaker outputs the meeting answer according to the meeting question after receiving the control instruction, and outputs control instructions to each target agent with the role of the observer in sequence according to the meeting process, so that the target agent with the role of the observer outputs the meeting minutes according to the meeting answer after receiving the control instruction, that is, the control signal received by the target agent with the role of the host is sent by the signaling server, and the control signal received by the target agent with the role of the observer and the speaker is sent by the signaling server to control the target agent with the role of the host. Since the meeting process includes the speaking order of the target agent with the role of the speaker and the meeting question corresponding to the target agent with the role of the speaker, the target agent with the role of the host includes the order of output to each target agent with the role of the speaker and the meeting question corresponding to each target agent with the role of the speaker in the control signal received from the signaling server. And the target agent with the role of the observer will output the meeting minutes of the corresponding question after each target agent with the role of the speaker finishes speaking. At this time, the response data obtained by the signaling server includes the meeting answers of the person whose role is the speaker and the meeting minutes of the person whose role is the observer.

[0047] In addition, it should be noted that in addition to the host, observer and speaker, the meeting roles may also include more roles, such as expert consultants, data analysts, creative inspirers, etc., and the embodiments of the present disclosure do not specifically limit this.

[0048] S140. Generate a meeting report based on the response data of each target intelligent agent.

[0049] Among them, the meeting report includes meeting summary, decision-making recommendations and action recommendations.

[0050] In a specific implementation, a meeting report is generated based on the response data of each target intelligent agent, including: obtaining the meeting answers of the target intelligent agent whose role in the meeting is a speaker to each meeting question, and the meeting minutes of the target intelligent agent whose role in the meeting is an observer to each meeting question; and generating a meeting report based on the meeting process, meeting answers and meeting minutes.

[0051] After obtaining the meeting answers for the speaker and the meeting minutes for the observer, the signaling server associates each meeting question with the corresponding meeting answer and meeting minutes, and then generates a specific meeting report based on the meeting answers and meeting minutes corresponding to each meeting question after the meeting.

[0052] It should be noted that, in the above-mentioned embodiments, the signaling server is exemplarily shown to determine the role of each target intelligent agent in the meeting, the speaking order of each target intelligent agent in the meeting, and the meeting questions. In addition, the signaling server can also output a control signal to the target intelligent agent whose role is the host, and control the speaking time of the target intelligent agent whose role is the speaker through the target intelligent agent whose role is the host, thereby ensuring the efficiency of the meeting, and can dynamically adjust the target intelligent agent and observer corresponding to the role of the speaker according to the meeting answer of the target intelligent agent whose role is the speaker, as well as adjust the number of target intelligent agents for each role, etc.

[0053] When the roles of the meeting include not only the host, observer and speaker, but also expert consultants, data analysts and creative inspirers, the generated meeting report can be analyzed from multiple dimensions, thus improving the practicality of the generated meeting report.

[0054] The multimodal agent conference information interaction method provided by the disclosed embodiment responds to receiving a conference request sent by the target object, determines the target agent group according to the conference request; sends a request to join the meeting to each target agent, so that the target object and each target agent establish conference communication; outputs control instructions to each target agent in sequence according to the first tag identification and conference process of each target agent, and obtains the response data of each target agent after receiving the control instruction; generates a conference report according to the response data of each target agent. Since the target agents are selected based on the conference theme, the target agents participating in this conference have a close correlation with the content discussed in this conference, thereby improving the accuracy of the decision recommendations finally generated. In addition, in the process of outputting control instructions to the target agents, based on the first tag identification of each target agent, the role of each target agent in the conference is determined, so that each agent plays the greatest role in the conference, further ensuring the accuracy of the generated decision recommendations.

[0055] On the basis of the above-mentioned embodiments, the method provided by the embodiments of the present disclosure further includes: acquiring the conference data of each target intelligent agent, and updating the database of each target intelligent agent.

[0056] The conference data includes conference roles, answered conference questions, and conference answers given to conference questions, etc., which are not specifically limited in the embodiments of the present disclosure.

[0057] In addition, in the disclosed embodiment, the signaling server stores the meeting process, meeting answers, meeting minutes and meeting reports of each meeting, and learns the historical meeting data through a machine learning algorithm, thereby updating the screening criteria for the signaling server to screen the target intelligent entities.

[0058] Based on the above embodiments, the present disclosure also provides a multi-modal intelligent agent conference information interaction device, Figure 2 is a schematic diagram of the structure of a multi-modal intelligent agent conference information interaction device provided by an embodiment of the present disclosure, such as Figure 2 As shown, the multimodal intelligent agent conference information interaction device includes: The target agent determination module 210 is used to determine a target agent group according to the meeting request in response to receiving a meeting request sent by a target object, wherein the target object includes a user and an agent, the meeting request includes a meeting topic and the number of participants, the target agent group includes multiple target agents, and the number of target agents is the same as the number of participants; The communication establishment module 220 is used to send a conference joining request to each target intelligent entity so that the target object can establish conference communication with each target intelligent entity; The control module 230 is used to output control instructions to each target agent in sequence according to the first tag identification of each target agent and the conference process, and obtain response data of each target agent after receiving the control instruction; The meeting report generation module 240 is used to generate a meeting report based on the response data of each target intelligent agent, wherein the meeting report includes a meeting summary, decision recommendations and action recommendations.

[0059] The multimodal intelligent agent conference information interaction device provided by the embodiment of the present disclosure responds to receiving a conference request sent by the target object, determines the target intelligent agent group according to the conference request; sends a request to join the meeting to each target intelligent agent, so that the target object and each target intelligent agent establish conference communication; outputs control instructions to each target intelligent agent in sequence according to the first tag identification and conference process of each target intelligent agent, and obtains the response data of each target intelligent agent after receiving the control instruction; generates a conference report according to the response data of each target intelligent agent. Since the target intelligent agent is selected based on the conference theme, the target intelligent agent participating in this conference has a close correlation with the content discussed in this conference, thereby improving the accuracy of the decision-making suggestions finally generated. In addition, in the process of outputting control instructions to the target intelligent agent, based on the first tag identification of each target intelligent agent, the role of each target intelligent agent in the conference is determined, so that each intelligent agent plays the greatest role in the conference, further ensuring the accuracy of the generated decision-making suggestions.

[0060] In a specific implementation, in response to receiving a meeting request sent by a target object, determining a target agent group according to the meeting request includes: In response to receiving a meeting request sent by a target object, determining a target professional field to which the meeting request belongs according to a meeting theme; Acquire the second tag identifier of each agent, the response efficiency of each agent, and the status information of each agent, wherein the second tag identifier is used to identify the professional field to which the agent belongs, and the status information includes an idle state and a busy state; The target intelligent agent is determined according to the matching degree between the label content corresponding to the second label identification of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent.

[0061] In a specific implementation, the target agent is determined according to the matching degree between the tag content corresponding to the second tag identifier of each agent and the target professional field, the response efficiency of each agent, and the status information of each agent, including: Sequentially obtain a matching value corresponding to the matching degree between the label content corresponding to the second label identifier of each agent and the target professional field, an efficiency value corresponding to the response efficiency of each agent, and a state value corresponding to the state information of each agent; Determine the scoring parameters of each agent according to the matching value corresponding to the matching degree between the label content corresponding to the second label identifier of each agent and the target professional field, the efficiency value corresponding to the response efficiency of each agent, and the state value corresponding to the state information of each agent; After sorting the scoring parameters of each agent in forward order, select the target number of agents to form a target agent group, where the target number is the same as the number of participants.

[0062] In a specific implementation, according to the first tag identification of each target agent and the conference process, the control instruction is output to each target agent in sequence, and the response data of each target agent after receiving the control instruction is obtained, including: Determine the role of each target agent in the meeting according to the first tag identifier of each target agent and the proportional coefficient corresponding to each first tag identifier, wherein the role includes at least a host, an observer and a speaker; According to the role of each target agent in the meeting, the target agent whose role is the host is controlled to output control instructions to the other target agents except the host in sequence according to the meeting process, so that each target agent whose role is the speaker generates a meeting answer corresponding to the meeting question after receiving the control instruction, and each target agent whose role is the observer generates a meeting minutes corresponding to the meeting answer after receiving the control instruction, wherein the meeting process includes the speaking order of the target agent whose role is the speaker and the meeting question corresponding to the target agent whose role is the speaker; The response data of each target agent after receiving the control instruction is obtained, wherein the response data includes the meeting answers and meeting minutes.

[0063] In a specific implementation, a meeting report is generated based on the response data of each target agent, including: Obtain the meeting answers of the target agent whose role in the meeting is a speaker to each meeting question, and the meeting minutes of the target agent whose role in the meeting is an observer to each meeting question; Generate meeting reports based on meeting procedures, meeting answers, and meeting minutes.

[0064] In a specific implementation, in response to receiving a meeting request sent by a target object, before determining a target agent group according to the meeting request, the method further includes: In response to receiving the communication request of each intelligent agent, receiving the registration information of each intelligent agent, and storing the registration information of each intelligent agent, wherein the registration information includes device condition information and IP address.

[0065] In a specific embodiment, the method further comprises: Obtain the meeting data of each target agent and update the database of each target agent.

[0066] The present application also provides a computer device. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0067] The computer device includes a memory 510 and a processor 520 that are connected to each other through a system bus. It should be noted that the figure only shows a computer device with components 510-520, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field-programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0068] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0069] The memory 510 includes at least one type of readable storage medium, and the readable storage medium includes a non-volatile memory or a volatile memory, such as a flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc., and the RAM may include a static RAM or a dynamic RAM. In some embodiments, the memory 510 may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory 510 may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, or a flash card (Flash Card), etc., equipped on the computer device. Of course, the memory 510 may also include both an internal storage unit of the computer device and an external storage device thereof. In this embodiment, the memory 510 is generally used to store an operating system and various application software installed on the computer device, such as the program code of the above method, etc. In addition, the memory 510 may also be used to temporarily store various data that have been output or are to be output.

[0070] The processor 520 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 510 is used to store program codes or instructions, the program code includes computer operation instructions, and the processor 520 is used to execute the program codes or instructions stored in the memory 510 or process data, such as running the program code of the above method.

[0071] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus system can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0072] Another embodiment of the present application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads a computer-readable program code stored in the computer-readable medium, so that the processor can execute the functional actions specified in each step or a combination of steps in the above method; and generate a device for implementing the functional actions specified in each block or a combination of blocks in the block diagram.

[0073] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above methods stored in the memory.

[0074] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0075] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] Each functional unit or module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0077] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0078] Unless the context clearly indicates otherwise, the singular form of the words used herein and in the appended claims includes the plural and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the words "comprise" and "include" are to be interpreted as inclusive rather than exclusive. Likewise, the terms "include" and "or" should be interpreted as inclusive unless such interpretation is expressly prohibited herein. Where the term "example" is used herein, particularly when it is located after a group of terms, the "example" is merely exemplary and illustrative and should not be considered exclusive or comprehensive.

[0079] Further aspects and scopes of adaptability become apparent from the description provided herein. It should be understood that various aspects of the present application can be implemented individually or in combination with one or more other aspects. It should also be understood that the description and specific embodiments herein are intended for purposes of illustration only and are not intended to limit the scope of the present application.

[0080] Several embodiments of the present disclosure are described in detail above, but it is obvious that those skilled in the art can make various modifications and variations to the embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. The protection scope of the present disclosure is defined by the attached claims.

Claims

1. A multimodal agent conference information interaction method, characterized in that: include: In response to receiving a meeting request sent by a target object, determining a target agent group according to the meeting request, wherein the target object includes a user and an agent, the meeting request includes a meeting topic and the number of participants, the target agent group includes a plurality of target agents, and the number of the target agents is the same as the number of participants; Sending a conference joining request to each of the target intelligent entities, so that the target object establishes conference communication with each of the target intelligent entities; Output control instructions to each of the target agents in sequence according to the first tag identifiers of each of the target agents and the conference process, and obtain response data of each of the target agents after receiving the control instructions; A meeting report is generated according to the response data of each target intelligent agent, wherein the meeting report includes a meeting summary, decision recommendations and action recommendations.

2. The method according to claim 1, characterized in that The step of determining the target agent group according to the meeting request in response to receiving the meeting request sent by the target object includes: In response to receiving a meeting request sent by a target object, determining a target professional field to which the meeting request belongs according to the meeting theme; Acquire the second tag identifier of each agent, the response efficiency of each agent, and the state information of each agent, wherein the state information includes an idle state and a busy state; The target intelligent agent is determined according to the matching degree between the label content corresponding to the second label identification of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent.

3. The method according to claim 2, characterized in that The step of determining the target intelligent agent according to the matching degree between the tag content corresponding to the second tag identifier of each intelligent agent and the target professional field, the response efficiency of each intelligent agent and the status information of each intelligent agent comprises: Sequentially obtain a matching value corresponding to the matching degree between the tag content corresponding to the second tag identifier of each of the agents and the target professional field, an efficiency value corresponding to the response efficiency of each agent, and a state value corresponding to the state information of each agent; Determine the scoring parameters of each of the agents according to the matching value corresponding to the tag content corresponding to the second tag identifier of each of the agents and the matching degree of the target professional field, the efficiency value corresponding to the response efficiency of each of the agents, and the state value corresponding to the state information of each of the agents; After the scoring parameters of each of the agents are sorted in forward order, a target number of agents are selected to form a target agent group, wherein the target number is the same as the number of participants.

4. The method according to claim 1, characterized in that: Outputting control instructions to each target agent in sequence according to the first tag identifier of each target agent and the conference process, and obtaining response data of each target agent after receiving the control instruction, includes: Determine the role of each target intelligent agent in the meeting according to the first tag identifier of each target intelligent agent and the proportional coefficient corresponding to each first tag identifier, wherein the role includes at least a host, an observer and a speaker; According to the role of each target intelligent agent in the meeting, the target intelligent agent whose role is the host is controlled to output control instructions to the other target intelligent agents except the host in sequence according to the meeting process, so that each target intelligent agent whose role is the speaker generates a meeting answer corresponding to the meeting question after receiving the control instruction, and each target intelligent agent whose role is the observer generates a meeting minutes corresponding to the meeting answer after receiving the control instruction, wherein the meeting process includes the speaking order of the target intelligent agent whose role is the speaker and the meeting question corresponding to the target intelligent agent whose role is the speaker; Acquire response data of each target agent after receiving the control instruction, wherein the response data includes meeting answers and meeting minutes.

5. The method according to claim 1, characterized in that The generating of the conference report according to the response data of each target intelligent agent includes: Obtaining the meeting answers of the target intelligent agent whose role in the meeting is a speaker to each of the meeting questions, and the meeting minutes of the target intelligent agent whose role in the meeting is an observer to each of the meeting questions; Generate the meeting report according to the meeting process, the meeting answers and the meeting minutes.

6. The method according to claim 1, characterized in that In response to receiving a meeting request sent by a target object, before determining a target agent group according to the meeting request, the method further includes: In response to receiving the communication request of each intelligent agent, receiving the registration information of each intelligent agent, and storing the registration information of each intelligent agent, wherein the registration information includes device condition information and IP address.

7. The method according to claim 1, characterized in that The method further comprises: Acquire the conference data of each target intelligent agent, and update the database of each target intelligent agent.

8. A multi-modal intelligent agent conference information interaction device, characterized in that: include: A target agent determination module, configured to determine a target agent group according to a meeting request received from a target object, wherein the target object includes a user and an agent, the meeting request includes a meeting topic and the number of participants, the target agent group includes a plurality of target agents, and the number of the target agents is the same as the number of participants; A communication establishment module, used for sending a conference joining request to each of the target intelligent entities, so that the target object establishes conference communication with each of the target intelligent entities; A control module, used to output control instructions to each of the target agents in sequence according to the first tag identification and the conference process of each of the target agents, and obtain response data of each of the target agents after receiving the control instructions; A meeting report generation module is used to generate a meeting report based on the response data of each target intelligent agent, wherein the meeting report includes a meeting summary, decision recommendations and action recommendations.

9. A computer device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images