Conference management system and method integrating large language model and multi-agent collaboration

Abstract

The embodiment of the invention provides a conference management system and method fusing a large language model and multi-agent collaboration. The system comprises a multi-modal central language large model and a multi-agent platform, and the multi-modal central language large model comprises a multi-modal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module and an agent scheduling module; the multi-agent platform comprises a data searching and issuing agent, a agenda management agent, a voice transcription agent and a knowledge base updating agent. According to the embodiment of the invention, the conference process is divided into a plurality of independent intelligent agents, and then the intelligent agents are scheduled and dynamically arranged in a unified manner by the central big model. Strategies can be flexibly adjusted according to real-time situations, and intelligence and high adaptability of the whole process are guaranteed. The conference process and the emotional state are monitored. And the central big model sends an intervention instruction. The active conference management can be realized, a targeted intervention means is provided, and the conference efficiency and the communication effect can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice, and particularly to a conference management system and method that integrates large language models and multi-agent collaboration. Background Art

[0002] In order to improve the efficiency of meeting recording and management, existing technologies use AI voice transcription and automatic meeting minutes, and also use large language models to provide instant in-meeting suggestions or summary refinement, specifically including aspects such as meeting management, meeting content generation, and meeting summary.

[0003] Regarding meeting management, existing technologies submit meeting reservation-related information online by the meeting initiator, use deep learning algorithms to perform semantic understanding and context semantic association feature analysis on the reservation-related information of each meeting, and automatically arrange the meeting agenda based on this. In this way, intelligent scheduling can be performed according to the schedules of participants, avoiding time conflicts and improving meeting efficiency.

[0004] Regarding meeting content generation, existing technologies establish a task execution chain between large language models, determine a large model group according to the meeting task execution chain, divide multiple large model groups into several collaboration groups according to the established paradigm, and evaluate the content of each large model group in the collaboration group by an evaluator large language model to generate optimized content.

[0005] Regarding meeting summary, existing technologies obtain the natural language to be processed in the meeting, input the natural language to be processed into a pre-trained large language model, and the main control agent performs intention recognition, process reasoning, and process decomposition on the natural language to be processed to obtain reasoning results and decomposition processes. The main control agent performs intention reasoning based on the processing results to obtain and output the meeting reasoning conclusion.

[0006] In the process of implementing the present invention, the inventors found that there are at least the following problems in the related technologies: Regarding meeting management, it mainly focuses on the collection of meeting information, context semantic analysis, and automatic scheduling of time conflicts, and does not involve deep-level collaboration such as multi-Agent real-time intervention, transcription, and minutes generation during the actual progress of the meeting. Its focus is on the automated processing of "reservation information", and it does not disclose how to systematically manage and precipitate meeting minutes, decisions, to-do items, etc. after the meeting, nor does it reflect the technical means of continuous iteration and application based on the knowledge base.

[0007] Regarding the generation of meeting content, it focuses more on the content generation and evaluation iteration among multiple large language models, and does not specifically design for the requirements of agenda management, real-time audio transcription, and post-meeting task tracking in the scenario of online meetings. Although multi-model collaboration is discussed, no implementation of a complete closed-loop such as pre-meeting agenda planning, in-meeting process control, post-meeting knowledge base update, or task distribution is disclosed. Therefore, it cannot well solve the problems of "multi-role collaboration" and "intervention while in progress" in the actual meeting scenario.

[0008] Regarding the meeting summary, it belongs to a more general task decomposition and multi-Agent collaborative processing framework, mainly focusing on the process of intent recognition, sub-task decomposition, and result summarization. However, for the requirements of timely agenda adjustment, real-time minutes generation, and dynamic reminder in online meetings, there are no relevant technical points or process details. There is no functional description of continuous precipitation for in-meeting emotion analysis, timeout reminder, automatic minutes, or docking with the enterprise knowledge base, lacking special attention to the in-meeting / post-meeting scenarios. Summary of the Invention

[0009] In order to at least solve the problem that existing meeting management focuses more on aspects such as "pre-meeting scheduling", "optimization of multi-language model generation", or "general multi-Agent collaboration", and has not formed an intelligent management mechanism for the entire meeting process (pre-meeting, in-meeting, and post-meeting), and also lacks specific functions such as "real-time orchestration of the central large model for multi-Agents" and "automatic minutes and knowledge base iteration".

[0010] In a first aspect, an embodiment of the present invention provides a meeting management system integrating a large language model and multi-Agent collaboration, including: A multi-modal central language large model and a multi-Agent platform, where The multi-modal central language large model includes: a multi-modal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module, and an agent scheduling module, The multi-modal interaction module is used to perform multi-modal processing on the input content from the device side and the feedback from the multi-Agent platform, and obtain the meeting agenda determined by speech recognition and natural language understanding of the input content, and the meeting content determined by text generation and text-to-speech of the feedback; The task reasoning module is used to extract meeting minutes and meeting to-dos based on the meeting content and the meeting agenda; The task planning module is used to optimize the meeting time sequence according to the meeting to-dos; The agent scheduling module is used to parallelly call corresponding agents from the multi-Agent platform split based on the meeting process according to the meeting to-dos; The full-duplex interaction interface is used for full-duplex interaction in conference management of the input content from the device side and the feedback content of the multi-agent platform, so as to assist in the generation, modification and distribution of the meeting minutes, and the display and distribution of the feedback content.

[0011] In a second aspect, an embodiment of the present invention provides a conference management method based on a conference management system, including: In the pre-conference stage, the multi-modal central language large model responds to the input content sent by the conference host through the device side, and identifies the conference agenda through the task reasoning module; Optimize the time sequence of the conference agenda through the task planning module, and feedback the optimization result to the device side; The agent scheduling module responds to the confirmation of the feedback by the conference host through the device side, schedules the data search and distribution agent from the multi-agent platform to collect data related to the conference agenda from the knowledge base, and distributes it to all participants.

[0012] In a third aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the conference management method of the conference management system based on the cooperation of the fusion large language model and multi-agent in any embodiment of the present invention.

[0013] In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the conference management method of the conference management system based on the cooperation of the fusion large language model and multi-agent in any embodiment of the present invention are implemented.

[0014] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the conference management method of the conference management system based on the cooperation of the fusion large language model and multi-agent in any embodiment of the present invention are implemented.

[0015] The beneficial effects of the embodiments of the present invention are as follows: The conference process is split into multiple independent agents (agenda management Agent, sentiment analysis Agent, transcription Agent, knowledge base update Agent), and then the central large model with context understanding and multi-round reasoning capabilities uniformly schedules and dynamically arranges each Agent. In the same conference scenario, different Agents can complete their respective professional tasks in parallel and efficiently, and the system can flexibly adjust strategies according to real-time situations (such as topic timeout, mood fluctuations) to ensure the intelligence and high adaptability of the whole process. Brief Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic structural diagram of a conference management system integrating a large language model and multi-agent collaboration provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the user interface of a conference management system integrating a large language model and multi-agent collaboration provided by an embodiment of the present invention; Figure 3 It is a schematic flowchart of a conference management method based on a conference management system provided by an embodiment of the present invention; Figure 4 It is a schematic flowchart of a conference management method based on a conference management system provided by another embodiment of the present invention; Figure 5 It is a schematic flowchart of a conference management method based on a conference management system provided by yet another embodiment of the present invention; Figure 6 It is a system interaction flowchart of a conference management method based on a conference management system provided by an embodiment of the present invention; Figure 7 It is a schematic structural diagram of an embodiment of an electronic device for conference management based on a conference management system provided by an embodiment of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0019] As Figure 1 shown, it is a schematic structural diagram of a conference management system integrating a large language model and multi-agent collaboration provided by an embodiment of the present invention. The conference management system 10 integrating a large language model and multi-agent collaboration includes: a multimodal central language large model 11 and a multi-agent platform 12, where The multimodal central language large model 11 includes: a multimodal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module, and an agent scheduling module. The multimodal interaction module is used to perform multimodal processing on the input content from the device side and the feedback from the multi-agent platform, and obtain the meeting agenda determined by speech recognition and natural language understanding of the input content, and the meeting content determined by text generation and text-to-speech of the feedback. The task reasoning module is used to extract meeting minutes and meeting to-dos based on the meeting content and the meeting agenda. The task planning module is used to optimize the meeting time sequence according to the meeting to-dos. The agent scheduling module is used to parallelly call corresponding agents from the multi-agent platform split based on the meeting process according to the meeting to-dos. The full-duplex interaction interface is used to perform full-duplex interaction for meeting management on the input content from the device side and the feedback content from the multi-agent platform, so as to assist in the generation, modification, and distribution of the meeting minutes, and the display and distribution of the feedback content.

[0020] Considering that on the basis of meeting management, meeting content generation, and meeting summary in the prior art, the improvements in the prior art usually solve defects or enhance the user experience by the following several methods: 1. Through single-function optimization, such as strengthening speech transcription accuracy or automatic minutes generation, but this approach is often limited to a certain link and lacks an intelligent solution for the entire process. 2. Through traditional AI algorithms for multimodal data integration to improve recognition accuracy, but still rely on rules or simple conditional judgments, lacking dynamic multi-Agent collaboration and real-time intervention capabilities. 3. Through the integration with existing tools and services to achieve partial functions, such as meeting material pushing or translation. However, this integration is usually "point-to-point" and lacks in-depth system collaboration and intelligent scheduling.

[0021] This method discovers that the meeting management in the prior art lacks intelligent management of the entire meeting process, realizes the limitations of single-point optimization and shallow integration. Based on the central large model and the multi-Agent collaboration architecture, this system redesigned the intelligent management of the entire process from pre-meeting agenda optimization to in-meeting real-time intervention, and then to post-meeting task tracking and knowledge base update, so as to have higher forward-looking and differentiation. A meeting management system integrating large language models and multi-agent collaboration is constructed, and this system can be configured in cloud services for users to perform meeting management.

[0022] In the multi-modal central language large model, the multi-modal interaction module enables the multi-modal central language large model to recognize the input of the participants in the meeting and generate text or voice feedback of the output of the multi-agent platform to the participants. Specifically, to enable it to have interaction capabilities, the multi-modal interaction module is constructed from the following four aspects: 1. ASR (Automatic Speech Recognition). For example, an automatic speech recognition model based on the end-to-end Transformer architecture can be used, or an automatic speech recognition model that combines local feature extraction of CNN (Convolutional Neural Networks) can be used.

[0023] 2. NLU (Natural Language Understanding). For example, a Transformer model based on BERT can be used. In collaboration with ASR, on the basis of ASR, it performs natural language understanding on the recognized meeting content. For example, when it recognizes the user input: Help me organize a meeting for the First International Development Department, in the afternoon, or Wednesday morning is also okay. Through natural language understanding, it recognizes that the participants are "all members of the First International Development Department" and the meeting time is "afternoon or Wednesday morning". By the above method, the meeting time, location, process, etc. can be determined, and then the meeting agenda can be created.

[0024] 3. Text generation. Text generation models based on GAN (Generative Adversarial Networks) and VAE (Variational Auto encoders) can be used. Through ASR, the meeting information in the meeting is recognized to obtain the meeting context, and based on this, the text generation of the meeting is performed to obtain the meeting content.

[0025] 4. TTS (Text To Speech). It can perform text-to-speech generation on the content fed back by the multi-agent platform and feed the generated voice back to the participants.

[0026] Through the above method, the multi-modal interaction capabilities of the central language large model are realized.

[0027] The task reasoning module, task planning module, and agent scheduling module are responsible for task reasoning, task planning, and agent scheduling of multiple agents, and overall coordinate the data exchange and working order between different Agents. The specific steps are described in combination with the multi-agent platform.

[0028] A full-duplex interaction interface for full-duplex interaction in meeting management of input content from the device side and feedback content of the multi-agent platform, so that when the central large model interacts with the participants' voices, mutual interruption can be carried out, more efficiently helping the communication of the meeting, and also assisting the meeting management system to complete the meeting minutes, modification, display and distribution of feedback content faster.

[0029] As an implementation, the multi-agent platform includes: a data search and distribution agent, an agenda management agent, a voice transcription agent, and a knowledge base update agent, where, The data search and distribution agent is used to receive the meeting agenda from the multi-modal central language large model, retrieve data related to the meeting agenda, and feedback it to the device side through the multi-modal central language large model; The agenda management agent is used to monitor the duration and content of the meeting in real time, give active intervention prompts based on the duration and content of the meeting, and feedback the active intervention prompts to the device side through the multi-modal central language large model. The active intervention prompts include: timeout prompts, meeting conflict suggestion prompts; The voice transcription agent is used to perform real-time transcription on the meeting content transmitted through the multi-modal central language large model to obtain a transcribed text, and feedback the transcribed text to the multi-modal central language large model; The knowledge base update agent is used to store the meeting minutes and feedback content, and distribute them to the device side through the multi-modal central language large model.

[0030] In this implementation, the data search and distribution agent can be called the data collection / distribution Agent, which automatically retrieves or organizes data and documents related to the agenda before or during the meeting and distributes them to the participants.

[0031] The agenda management agent can be called the agenda management Agent, which monitors the meeting process, judges whether the agenda is overdue, whether the discussion goes off-topic, and gives intervention prompts when necessary.

[0032] The voice transcription agent can be called the meeting transcription Agent, which uses the ASR function to transcribe the meeting voice into text in real time and provides the transcription result to the central large model and other Agents.

[0033] The knowledge base update agent can be called the knowledge base update Agent, which stores data such as the generated meeting minutes, decision points, and to-do items in the enterprise knowledge base or document management system to provide a basis for subsequent retrieval and review.

[0034] To facilitate users in using the conference management system that integrates large language models and multi-agent collaboration, the conference management system further includes a user interface provided for the device side, where the user interface includes: A main viewing content area for displaying conference speakers or document sharing content through different display modes; A tag bar for providing auxiliary function tags for users, where the auxiliary function tags include: an agenda management tag and a real-time transcription tag; the agenda management tag includes: agenda display, overtime warning, topic expansion, and real-time suggestions; the real-time transcription tag includes: transcribed text, keyword highlighting, and positioning.

[0035] A toolbar for providing the user with functions including audio / video switch, screen sharing, recording, raising hand operation, and AI collaboration button; A status bar for displaying global status prompts such as the conference title and the number of meeting minutes; A suggestion panel for displaying the proactive intervention prompts.

[0036] In this embodiment, as Figure 2 shown in the block diagram of the user interface of the conference management system, the design is inclined to be modern, simple, and have high contrast, facilitating users to quickly obtain key information during the conference. The overall interface includes the following layout areas: (1) Main video screen / shared content area: Occupying most of the central space of the interface, it is used to present the video screen of the current conference speaker or shared content (PPT, document, screen, etc.); when there are multiple people's videos side by side, it can automatically switch to the "main speaker mode" or "gallery mode".

[0037] (2) Sidebar Tabs: Placing entry tags (Tabs) for important auxiliary functions such as "agenda management" and "real-time transcription"; users can switch between different Tabs in the sidebar to view the real-time agenda, transcribed text, message conversations, or system suggestions.

[0038] (3) Bottom toolbar: Placing common conference controls (operation buttons such as audio / video switch, screen sharing, recording, raising hand, etc.), and an "open / close AI collaboration assistant" button can also be added for users to choose whether to display the AI suggestion panel.

[0039] (4) Top status bar: Displaying the current conference title, countdown / progress bar (linked with agenda management), and global status prompts such as the number of recorded minutes / to-do items.

[0040] (5) AI suggestion panel card: According to the progress of the conference, the system pops up suggestion cards (such as "Do you need to vote?" and "Do you need to share materials?") for situations such as disputes.

[0041] Specifically, the agenda management tags include: Agenda display: Displays the defined meeting agenda in list or card form, showing the estimated discussion duration, remaining time, and automatic minutes editing pop-up window under each topic.

[0042] Timeout warning: If the topic time limit is exceeded, the topic entry is highlighted and buttons such as "Next Topic" or "Extend Duration" are provided.

[0043] Topic expansion: Supports clicking on a topic to view details, including main discussion points, relevant material links, etc.

[0044] Real-time suggestion: According to the real-time monitoring of the meeting, if the system detects repeated invalid disputes (controversial conflicts) or intense tone conflicts (emotional conflicts), the system suggests that the host appropriately guide in the form of a prompt card.

[0045] The real-time transcription tags include: Transcription text area: Scrolls and displays the real-time transcription results in chronological order; different speakers can be distinguished by color or avatar tags.

[0046] Keyword highlighting: Automatically detects and highlights words related to the agenda theme or business keywords for easy reference.

[0047] Search / Locate: Provides a search bar where users can enter keywords to locate relevant content in the real-time transcription.

[0048] It can be seen from this implementation that the meeting management system can solve the problems of common functional fragmentation, information silos, lack of real-time intervention, and multi-modal data processing in existing meeting management through the collaborative work of the central large model and multiple agents, realizing a complete meeting management closed-loop. Through full-process management, real-time dynamic intervention, and knowledge base precipitation, it can greatly improve the efficiency and decision-making quality of meetings.

[0049] Such as Figure 3 Shown is a flowchart of a meeting management method based on a meeting management system provided by an embodiment of the present invention, including the following steps: S11: In the pre-meeting stage, the multi-modal central language large model responds to the input content sent by the meeting host through the device end, and identifies the meeting agenda through the task reasoning module; S12: Optimize the time sequence of the meeting agenda through the task planning module, and feedback the optimization result to the device end; S13: The agent scheduling module responds to the confirmation of the feedback by the meeting host through the device end, schedules the data search and distribution agent from the multi-agent platform to collect data related to the meeting agenda from the knowledge base, and distributes it to all participants.

[0050] In this embodiment, based on the constructed conference management system that integrates large language models and multi-agent collaboration, this method uses this conference management system to manage conferences.

[0051] For step S11, large companies usually have multiple conference rooms for employees from different departments to hold meetings. For example, on one floor of a large company, there are Domestic Development Department 1, Domestic Development Department 2, International Development Department 1, International Development Department 2, International Development Department 3, Public Affairs Department, Financial Management Department, and two to three conference rooms. Due to the large number of departments, organizing a meeting in this environment does not mean that you can immediately start the meeting by gathering the participants. It is possible that other departments also need to hold meetings at the same time, or it may be necessary to vacate the conference room for customers coming to handle business. In the startup company environment, there are usually multiple different companies on one floor. To facilitate these companies to hold meetings, public conference rooms are also provided on the floor for different companies to use. When organizing a meeting in this environment, it is also necessary to consider whether others will occupy the conference room. Therefore, this method needs to be managed in the pre-meeting stage of the conference management system that integrates large language models and multi-agent collaboration.

[0052] The meeting host inputs the meeting content, specifically the meeting creation and agenda, in natural language or text form through the user interface of the conference management system on the device side. The multi-modal central language large model responds to the input of the meeting host, recognizes the text content of the input meeting creation and agenda through the multi-modal interaction module, and then recognizes the content of the meeting agenda from the text content through the task reasoning module. For example, the meeting theme, participants, meeting location, meeting time, and meeting duration.

[0053] For step S12, in the pre-meeting stage, the task planning module generates an initialization in chronological order based on the content of the meeting agenda, and initiates interactive modification, optimization, and confirmation with the meeting host through the full-duplex interaction interface to complete the "setting" optimization of the meeting agenda. Furthermore, considering the efficient utilization of the conference room and enabling each meeting to start as soon as possible, the task planning module can be used to optimize the meeting duration. For example, the conference room is occupied by other meetings from 1 pm to 2 pm and from 3 pm to 5 pm. And the theoretical duration of this meeting is about 1 hour and 20 minutes. If optimized in the conventional order, the meeting will be postponed to the next day. This method can use the task planning module to optimize the meeting duration. Judge whether there is room for optimization in some agendas of the meeting agenda to optimize the duration of this meeting. For example, optimize 1 hour and 20 minutes to within 1 hour. Return the optimization result to the host for confirmation. In this way, the meeting host can make an appropriate choice. Ensure the effective utilization of the conference room and the efficiency of the meeting itself.

[0054] For step S13, after the meeting is confirmed to be created, the document search and distribution Agent is called to retrieve relevant documents from the internal knowledge base or external resources according to the agenda theme, and after being reviewed by the central large model, they are sent to the host and other participants for pre-meeting preparation. In this way, the meeting management in the pre-meeting stage is realized.

[0055] As Figure 4 shown in the flowchart of a meeting management method provided by another embodiment of the present invention, which includes the following steps: S21: In response to the meeting host's click on the start meeting button in the user interface of the device end, the multi-modal central language large model enters the in-meeting stage, and the agent scheduling module calls the voice transcription agent and the agenda management agent for real-time processing and monitoring; S22: The voice transcription agent performs real-time transcription processing on the input of the participants and feeds the transcribed text obtained through transcription back to the multi-modal central language large model; S23: The agenda management agent monitors the meeting duration and the input of the participants in real time, judges the emotions of the participants, determines the conflict type according to the emotions and the input, generates corresponding proactive intervention prompts, and feeds the proactive intervention prompts back to the device end through the multi-modal central language large model, and prompts through the suggestion panel of the device end.

[0056] In this embodiment, this method takes into account using the meeting management system for meeting management in the in-meeting stage.

[0057] For step S21, the meeting host clicks "Start Meeting" in the user interface of the device end. At this time, the agent scheduling module is activated, and the meeting transcription Agent, the agenda management Agent, etc. are created and scheduled to enter the real-time monitoring and processing state.

[0058] For step S22, the meeting transcription Agent performs real-time transcription on the speeches of the participants and feeds the transcription results back to the central large model and the user interfaces of the host / participants for easy viewing and marking of key points at any time.

[0059] For step S23, during the meeting, the agenda management Agent monitors and optimizes the meeting progress duration (the actual execution order and time consumption of the agenda) and whether emotional / disputatious conflicts occur. If it times out, it feeds back information to the central large model and displays it to the user side. That is to say, the optimization of the agenda management Agent is determined through intervention judgment; if a conflict is detected, for example, by performing emotion detection on the speech, the agenda management Agent identifies the conflict type, and the central large model can issue a "suggested discussion" or "priority processing" prompt, which is prompted through the suggestion panel of the device-side user interface. So that the host can make timely adjustments according to the prompt to complete the "execution" optimization of the meeting agenda.

[0060] It can be seen from this implementation that during the meeting, the meeting process and emotional state can be monitored through the agenda management Agent. Once abnormal situations such as timeouts or conflict emotions are detected, the central large model issues intervention instructions (such as reminding the host, adjusting the agenda, providing materials). This can achieve "active" rather than "passive" meeting management. On the one hand, it avoids the traditional single time reminder or condition trigger method, and on the other hand, it provides targeted and personalized intervention means, which can improve the meeting efficiency and communication effect.

[0061] As Figure 5 shown is a flowchart of a meeting management method based on a meeting management system provided by another embodiment of the present invention, including the following steps: S31: During the in-meeting stage, the multi-modal central language large model responds to the click of the end meeting button by the meeting host in the user interface of the device side, enters the post-meeting stage, uses the task reasoning module to generate the original meeting minutes from the transcribed text transcribed by the speech transcription agent, and displays them to the participants in real time through the user interface; S32: During the real-time display process, use the full-duplex interaction interface to receive the modification content input by the participants, interrupt the generation of the original meeting minutes based on the modification content, and regenerate the final meeting minutes; S33: Schedule the knowledge base update agent through the agent scheduling module to store the modification content and the final meeting minutes, and call the data search and distribution agent to send the final meeting minutes to all participants.

[0062] In this implementation, this method takes into account using the meeting management system for meeting management in the post-meeting stage.

[0063] For step S31, the meeting host clicks the "End Meeting" button in the user interface on the device side, triggering the generation of meeting minutes, which changes the progress from the in-meeting stage to the post-meeting stage. The task inference module performs aggregation and semantic analysis on the transcribed text transcribed by the speech transcription agent to generate a structured draft of the meeting minutes, i.e., the meeting minutes, including the discussion content of each topic, the main conclusions, and to-do items, etc., and displays it to the participants in real time through the user interface.

[0064] For step S32, during the real-time generation of the meeting minutes, the generated content is continuously displayed to the participants in real time through the user interface. The host or participants can view and revise the draft meeting minutes. If there are any errors, the full-duplex interaction interface in the multimodal central language large model can receive the modified content input by the participants in real time, and then regenerate the final meeting minutes in combination with the revision opinions.

[0065] For step S33, the knowledge base update Agent stores the key content of the meeting minutes (such as project decisions, risk assessments, technical solutions, etc.) in the knowledge base or the enterprise document management system to form traceable and searchable materials. The final meeting minutes can be automatically distributed to all participants by calling the data search and distribution agent using email or IM tools for subsequent follow-up.

[0066] It can be seen from this implementation method that after the meeting ends, important decisions, discussion content, and to-do items will be automatically formed into structured meeting minutes, and the enterprise knowledge base will be persistently stored by the knowledge base update Agent. In this way, the meeting results are deeply integrated with enterprise document management and knowledge precipitation, enabling subsequent meetings or projects to directly call previous discussion results, decision-making references, and historical documents, reducing repeated discussions and improving decision-making accuracy, and forming a knowledge reuse and self-evolution cycle for the future.

[0067] Regarding the overall interaction process of the above pre-meeting stage, in-meeting stage, and post-meeting stage as Figure 6 shown. It can be seen that the meeting process is split into multiple independent agents (agenda management Agent, sentiment analysis Agent, transcription Agent, knowledge base update Agent), and then the central large model with context understanding and multi-round reasoning capabilities uniformly schedules and dynamically arranges each Agent. In the same meeting scenario, different agents can complete their respective professional tasks in parallel and efficiently, and the system can flexibly adjust strategies according to real-time situations (such as topic timeout, mood fluctuations) to ensure the intelligence and high adaptability of the entire process.

[0068] The embodiment of the present invention also provides a non-volatile computer storage medium, which stores computer-executable instructions, and the computer-executable instructions can execute the meeting management method of the meeting management system based on the fusion of large language models and multi-agent collaboration in any of the above method embodiments; As an implementation, the non - volatile computer storage medium of the present invention stores computer - executable instructions, and the computer - executable instructions are set as follows: In the pre - meeting stage, the multi - modal central language large - model responds to the input content sent by the meeting host through the device end, and identifies the meeting agenda through the task reasoning module; The task planning module optimizes the time sequence of the meeting agenda and feeds the optimization result back to the device end; The intelligent agent scheduling module responds to the confirmation of the feedback by the meeting host through the device end, schedules the data search and distribution intelligent agent from the multi - intelligent agent platform to collect data related to the meeting agenda from the knowledge base, and distributes it to all participants.

[0069] The multi - modal central language large - model responds to the click of the start - meeting button in the user interface of the device by the meeting host and enters the in - meeting stage. The intelligent agent scheduling module calls the voice transcription intelligent agent and the agenda management intelligent agent for real - time processing and monitoring; The voice transcription intelligent agent performs real - time transcription processing on the input of the participants and feeds the transcribed text obtained through transcription back to the multi - modal central language large - model; The agenda management intelligent agent monitors the meeting duration and the input of the participants in real - time, judges the mood of the participants, determines the conflict type according to the mood and the input, generates a corresponding active intervention prompt, and feeds the active intervention prompt back to the device end through the multi - modal central language large - model, and prompts through the suggestion panel of the device end.

[0070] In the in - meeting stage, the multi - modal central language large - model responds to the click of the end - meeting button in the user interface of the device by the meeting host and enters the post - meeting stage. It uses the task reasoning module to generate the original meeting minutes from the transcribed text transcribed by the voice transcription intelligent agent and displays them to the participants in real - time through the user interface; During the real - time display process, it uses the full - duplex interaction interface to receive the input modification content of the participants, interrupts the generation of the original meeting minutes based on the modification content, and regenerates the final meeting minutes; The intelligent agent scheduling module schedules the knowledge - base update intelligent agent to store the modification content and the final meeting minutes, and calls the data search and distribution intelligent agent to send the final meeting minutes to all participants.

[0071] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the method in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, they perform the meeting management method of the meeting management system based on the fusion of large language models and multi-agent collaboration in any of the above method embodiments.

[0072] Figure 7 It is a schematic diagram of the hardware structure of an electronic device for the meeting management method of the meeting management system based on the fusion of large language models and multi-agent collaboration provided in another embodiment of the present application, as Figure 7 shown. The device includes: One or more processors 710 and a memory 720, Figure 7 Taking one processor 710 as an example. The device for the meeting management method of the fusion of large language models and multi-agent collaboration may further include: an input device 730 and an output device 740.

[0073] The processor 710, the memory 720, the input device 730, and the output device 740 can be connected through a bus or other means, Figure 7 Taking connection through a bus as an example.

[0074] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the meeting management method of the fusion of large language models and multi-agent collaboration in the embodiments of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 720, that is, implements the meeting management method of the meeting management system based on the fusion of large language models and multi-agent collaboration in the above method embodiments.

[0075] The memory 720 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data, etc. In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 720 may optionally include a memory remotely set relative to the processor 710, and these remote memories can be connected to the mobile device through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0076] The input device 730 can receive input digital or character information. The output device 740 can include display devices such as a display screen.

[0077] The one or more modules are stored in the memory 720 and, when executed by the one or more processors 710, perform the meeting management method of the meeting management system based on the integration of the large language model and multi-agent collaboration in any of the above method embodiments.

[0078] The above product can execute the method provided in the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided in the embodiments of the present application.

[0079] The non-volatile computer-readable storage medium may include a storage program area and a storage data area. Among them, the storage program area can store an operating system and application programs required for at least one function; the storage data area can store data created according to the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the device through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0080] The embodiments of the present invention also provide an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the meeting management method of the integration of the large language model and multi-agent collaboration in any embodiment of the present invention.

[0081] The electronic devices in the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by having mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones, multimedia phones, functional phones, and low-end phones, etc.

[0082] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have the characteristic of mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, etc., such as tablet computers.

[0083] (3) Portable entertainment devices: Such devices can display and play multimedia content. This type of device includes: audio and video players, handheld game consoles, e-books, as well as smart toys and portable in-vehicle navigation devices.

[0084] (4) Other electronic devices with data processing functions.

[0085] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising" and "including" not only include those elements, but also other elements not explicitly listed, or elements inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the said element.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A conference management system integrating a large language model and multi-agent collaboration, comprising: Multimodal central language large model and multi-agent platform, including: The multimodal central language model includes: a multimodal interaction module, a full-duplex interaction interface, a task reasoning module, a task planning module, and an intelligent agent scheduling module. The multimodal interaction module is used to perform multimodal processing on the input content from the device end and the feedback from the multi-agent platform, and obtain the meeting agenda determined by speech recognition and natural speech understanding for the input content, and the meeting content determined by text generation and text-to-speech for the feedback; The task reasoning module is used to extract meeting minutes and meeting agents according to the meeting content and the meeting agenda; The task planning module is used to optimize the meeting time sequence according to the meeting agent; The agent scheduling module is used to call the corresponding agents in parallel from the multi-agent platform based on the splitting of the conference process according to the conference agent; The full-duplex interaction interface is used for full-duplex interaction of conference management for input content from the device side and feedback content from the multi-agent platform, so as to assist in the generation, modification and distribution of the meeting minutes, and the display and distribution of the feedback content.

2. The system according to claim 1, wherein: The multi-agent platform includes: a data search agent, an agenda management agent, a speech transcription agent, and a knowledge base update agent, wherein: The data search agent is used to receive the meeting agenda from the multimodal central language model, retrieve data related to the meeting agenda, and feed it back to the device end through the multimodal central language model; The agenda management agent is used to monitor the duration and content of the meeting in real time, and to actively intervene based on the duration and content of the meeting. The active intervention prompt is fed back to the device end through the multimodal central language model. The active intervention prompt includes: timeout prompt and meeting conflict suggestion prompt; The speech transcription agent is used to transcribe the conference content transmitted via the multimodal central language model in real time to obtain a transcribed text, and feed the transcribed text back to the multimodal central language model; The knowledge base updating agent is used to store meeting minutes and feedback content, and distribute them to the device end through the multimodal central language model.

3. The system according to claim 1, wherein: The conference management system further includes a user interface provided for the device end, wherein the user interface includes: The main content area is used to display conference speakers or document sharing content through different display modes; The tag bar is used to provide auxiliary function tags for users, wherein the auxiliary function tags include: agenda management tags and real-time transcription tags; Toolbar, used to provide the user with buttons including audio and video switch, screen sharing, recording, hand raising operation, and AI collaboration; Status bar, used to display the global status prompt of the meeting title and number of meeting minutes; A suggestion panel is used to display the proactive intervention prompt.

4. The system according to claim 3, wherein: The agenda management tags include: agenda display, timeout warning, topic development and real-time suggestions; the real-time transcription tags include: transcription text, keyword highlighting and positioning.

5. A conference management method based on a conference management system, comprising: In the pre-meeting stage, the multimodal central language model responds to the input content sent by the meeting host through the device end and identifies the meeting agenda through the task reasoning module; Optimizing the time sequence of the meeting agenda through a task planning module, and feeding back the optimization result to the device end; The agent scheduling module responds to the confirmation of the feedback by the conference host through the device end, schedules the data search agent in the multi-agent platform to collect data related to the meeting agenda from the knowledge base, and distributes it to all participants.

6. The method according to claim 5, wherein: The method further comprises: The multimodal central language model responds to the conference host clicking the start meeting button in the user interface of the device to enter the mid-conference stage. The agent scheduling module calls the speech transcription agent and the agenda management agent for real-time processing and monitoring. The speech transcription agent performs real-time transcription processing on the input of the participants, and feeds back the transcribed text obtained through transcription to the multimodal central language model; The agenda management agent monitors the meeting duration and the input of the participants in real time, judges the emotions of the participants, determines the conflict type based on the emotions and the input, generates corresponding active intervention prompts, and feeds back the active intervention prompts to the device side through the multimodal central language model, and prompts through the suggestion panel on the device side.

7. The method according to claim 6, wherein: The method further comprises: In the mid-meeting stage, the multimodal central language model responds to the conference host clicking the end-meeting button in the user interface of the device end, enters the post-meeting stage, and uses the task reasoning module to generate the original meeting minutes from the transcribed text transcribed by the speech transcription agent, and displays it to the participants in real time through the user interface; In the process of real-time presentation, a full-duplex interactive interface is used to receive the modification content input by the conference participant, and the generation of the original conference minutes is interrupted based on the modification content, and the final conference minutes are regenerated; The agent scheduling module schedules the knowledge base updating agent to store the modified content and the final meeting minutes, and calls the data search and sending agent to send the final meeting minutes to all participants.

8. A storage medium having a computer program product stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 5 to 7 are implemented.

9. A computer program product having instructions embedded on a storage medium, wherein the instructions implement the steps of the method according to any one of claims 5 to 7.

10. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 5 to 7.

Images