Intelligent conference management system, method and equipment based on large model and medium

Abstract

The invention provides an intelligent conference management system, method and device based on a large model and a medium, and belongs to the technical field of computers. The system comprises a pre-meeting module used for automatically generating and sending a meeting invitation through an integrated enterprise schedule system, using telephone notification and IVR interaction to confirm a meeting participation form, and counting a meeting participation condition and a meeting arrival state; the in-conference module is used for collecting multi-source audios based on a microphone array in the conference process, transferring conference content in real time and generating subtitles in combination with a voice recognition technology, synchronously recording videos, recording process information of the conference, and performing structured analysis; and the post-conference module is used for extracting key indication information and backlogs in the process information of the conference, generating a standardized conference summary, automatically distributing tasks, tracking the progress and supporting semantic retrieval of historical conference records. According to the invention, a large model technology is utilized, and technologies of cloud computing, big data, artificial intelligence and the like are combined, so that automatic management of the whole conference process is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and more particularly relates to an intelligent conference management system, method, device and medium based on a large model. Background Art

[0002] As businesses grow and their operations become increasingly complex, meetings, a crucial means of communication and collaboration, continue to increase in number. However, traditional meeting management methods are no longer able to meet the efficient, accurate, and convenient management needs of modern enterprises. Specifically, traditional meeting management methods present numerous problems before, during, and after meetings: During the pre-meeting phase, meeting invitations must be sent manually one by one, which is not only time-consuming and labor-intensive but also prone to omissions, resulting in some attendees not receiving the meeting notification in a timely manner. The attendee confirmation process is also cumbersome, often requiring organizers to communicate repeatedly via email and phone, increasing the workload. Furthermore, pre-meeting reminders are easily overlooked, especially in busy work environments. Participants may miss the meeting due to negligence, disrupting the smooth progress of the meeting.

[0003] During meetings, minutes are often taken manually, a method that is not only inefficient but also prone to missing important details. As discussions deepen, manual recording often struggles to keep pace, resulting in incomplete minutes. This not only hinders the accurate communication of meeting content but also makes subsequent compilation of the minutes difficult.

[0004] After the meeting, compiling meeting minutes takes a significant amount of time. Due to manual organization, the format is difficult to standardize, making subsequent review and archiving inconvenient. Tracking of to-do items is also untimely, often relying on the initiative and responsibility of participants, which can easily lead to work delays. Furthermore, searching historical meeting records is difficult, making it difficult to quickly access key information from past meetings, impacting the efficiency of corporate knowledge management and decision-making. Summary of the Invention

[0005] In response to the above problems, the purpose of the present invention is to provide an intelligent conference management system, method, equipment and medium based on big models, which utilizes big model technology and combines cloud computing, big data, artificial intelligence and other technologies to realize automated management of the entire conference process.

[0006] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions: In a first aspect, an embodiment of the present application provides an intelligent conference management system based on a large model, comprising: The pre-meeting module is used to automatically generate and send meeting invitations through integration with the enterprise calendar system, confirm the form of participation through telephone notifications and IVR interaction, and conduct real-time statistics on participation and attendance status to prepare for the meeting; The in-meeting module is used to collect multi-source audio using a microphone array during a meeting, transcribe the meeting content in real time and generate subtitles using speech recognition technology, simultaneously record the video, record the meeting process information, and perform structured analysis; The post-meeting module uses natural language processing technology to extract key instructions and to-do items from the meeting process information, generate standardized meeting minutes, automatically assign tasks and track progress, and support semantic retrieval of historical meeting records.

[0007] In an optional embodiment, the pre-meeting module includes: a conference invitation automatic generation unit, an automatic telephone notification unit, an IVR conference attendance confirmation unit, a conference attendance statistics unit, an automatic telephone reminder unit, and an attendance statistics unit; The automatic meeting invitation generation unit is used to integrate with the enterprise calendar management system through API, automatically extract the meeting metadata, generate standardized meeting invitation information, and send it in batches to the participants' email addresses or mobile terminals through network protocols; Automatic telephone notification unit, which is used to automatically dial according to the meeting schedule based on the built-in telephone call module, and use voice conversion technology to notify participants of meeting information in voice form; The IVR confirmation unit is used to enable participants to select their participation mode through keystrokes or voice dialogues through a multi-round interactive voice response process, and the feedback results are updated to the database in real time; A participant statistics unit is used to collect and count the participant's participation form reply information in real time and generate a dynamic statistics table; the dynamic statistics table includes the participant's name, contact information, participation form and confirmation status information; Automatic telephone reminder unit, used to automatically trigger telephone reminders for participants within a preset time before the meeting starts and broadcast the meeting countdown information; The attendance statistics unit is used to integrate with the preset personnel recognition system in the conference room to collect the actual attendance data in real time and automatically count the actual attendance data.

[0008] In an optional embodiment, the in-conference module includes: a multimodal data acquisition unit, a speech transcription and real-time subtitle unit, and a conference content structuring unit; A multimodal data acquisition unit, which uses a circular microphone array to collect conference audio, simultaneously connects to multiple cameras to record conference video, and uploads the conference audio and video to cloud storage in real time; The speech transcription and real-time subtitle unit is used to separate and transcribe the speech of multiple speakers based on an improved automatic speech recognition model, generate real-time subtitle information for the meeting, and push it to the mobile terminals of participants; The conference content structuring unit is used to perform real-time semantic analysis on the real-time subtitle information of the conference, mark key topics, decision points and controversial content, and generate a dynamic conference context diagram.

[0009] In an optional embodiment, the post-meeting module includes: a leadership instruction extraction unit, a task tracking and reminder unit, and a historical meeting retrieval unit; The leader instruction extraction unit is used to use the speech recognition model to identify key information in the leader's speech and generate complete instructions based on the context; Meeting minutes generation unit, used to automatically fill in key instructions in meeting information using preset templates, format and convert them into meeting minutes; supports ISO standard format export and custom style adjustment; Task tracking and reminder unit, used to break down to-do items in meeting information into subtasks through project and issue tracking tools, automatically assigning responsible persons and deadlines; The historical meeting retrieval unit is used to build a full-text retrieval engine, which uses the full-text retrieval engine to retrieve historical meeting records in cloud storage based on keywords, and displays the retrieval results in the form of a timeline or graph.

[0010] In a second aspect, the embodiments of the present application further provide a method for intelligent conference management based on a large model, including: Automatically generate and send meeting invitations through integration with the enterprise calendar system, use phone notifications and IVR interaction to confirm attendance, and conduct real-time statistics on attendance and attendance status to prepare for meetings. During the meeting, the system collects multi-source audio using a microphone array, combines speech recognition technology to transcribe the meeting content in real time and generate subtitles, simultaneously records video, records meeting process information, and performs structured analysis. Natural language processing technology is used to extract key instructions and to-do items from meeting process information, generate standardized meeting minutes, automatically assign tasks and track task progress.

[0011] In an optional embodiment, the automatic generation and sending of meeting invitations through the integrated enterprise calendar system, confirmation of attendance through telephone notifications and IVR interaction, and real-time statistics of attendance and presence status are performed to prepare for the meeting, including: Extract meeting metadata from the enterprise calendar system through the API interface, automatically generate standardized meeting invitation information, and send it to participants' terminals in batches via network protocols; Trigger the phone call system based on the preset time and broadcast the meeting information to the participants through voice conversion technology; Receive keystroke or voice feedback from participants through multiple rounds of IVR interaction, and update the participant confirmation information to the database in real time; Dynamically generate statistical forms containing participant identity information, contact information, and confirmation status; Automatically initiate a phone reminder at a preset threshold time before the meeting starts, broadcasting the meeting countdown information; The actual attendance data is collected through the conference room personnel identification system to generate real-time attendance statistics reports.

[0012] In an optional embodiment, the method of collecting multi-source audio based on a microphone array during a meeting, combining speech recognition technology to transcribe the meeting content in real time and generate subtitles, synchronously recording video, recording the meeting process information, and performing structured analysis includes: Use a circular microphone array to capture multi-channel conference audio, and use synchronous multi-channel video capture equipment to record conference images; Apply an improved ASR model to separate and recognize the speech of multiple speakers and generate a conference subtitle stream with timestamps in real time; The conference subtitle stream is semantically analyzed, key topics, decision nodes and controversial contents in the conference process are dynamically annotated, and a visual conference context map is generated.

[0013] In an optional embodiment, the method of extracting key instructions and to-do items from the meeting process information using natural language processing technology, generating standardized meeting minutes, automatically assigning tasks, and tracking task progress includes: The system uses speech feature recognition technology to extract the specific speaker's instruction information, combines the context to generate a complete instruction text, and extracts key instructions and to-do items from the meeting process information; Automatically fill in meeting elements using preset templates based on meeting process information to generate formatted meeting minutes documents that comply with ISO standards; Break down to-do items into structured tasks, automatically associate responsible persons, and set execution time nodes; Build a retrieval model based on semantic vectors to retrieve historical meeting content through natural language queries according to retrieval requirements; Dynamically monitor the task execution status and trigger progress reminders at preset time nodes.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the large-model-based intelligent conference management method as described in any one of the above items are implemented.

[0015] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the intelligent conference management method based on a large model as described in any one of the above items are implemented.

[0016] It can be seen from the above technical solutions that the present invention has the following advantages: The large-scale model-based intelligent conference management system provided in this application achieves efficient management of the entire conference process through highly automated and intelligent means, significantly improving the efficiency and quality of conference organization. The system integrates functions such as enterprise calendar management, automatic phone notifications, IVR interactive confirmation, real-time voice transcription and subtitle generation, structured analysis of meeting content, extraction of key instructions and to-do items, standardized meeting minutes generation, and automatic task allocation and tracking. This significantly reduces manual operations, ensures the accurate transmission of meeting information, facilitates the storage and query of meeting records, and provides strong support for enterprise decision-making and work progress.

[0017] This application integrates with enterprise calendar management systems to automatically generate and batch-send meeting invitations, significantly improving the efficiency of invitation delivery. Furthermore, by utilizing phone notifications and interactive IVR confirmation, the confirmation process is more convenient and accurate, effectively avoiding the tedious and error-prone manual confirmation process. This not only saves organizers significant time but also ensures accurate communication of meeting information.

[0018] This application uses a microphone array to capture multi-source audio, combined with advanced speech recognition technology, to enable real-time transcription and subtitle generation of meeting content. This not only ensures the completeness and accuracy of meeting records, but also allows participants to focus more on the discussion without worrying about missing anything. Furthermore, the simultaneously recorded video provides strong support for subsequent meeting review and analysis.

[0019] This application uses natural language processing technology to automatically extract key instructions and to-do items from meetings and generate standardized meeting minutes. This not only improves the efficiency of meeting minute generation but also ensures the accuracy and standardization of meeting content. The system also automatically assigns tasks and tracks their progress, enabling rapid implementation of meeting decisions and effectively driving progress.

[0020] This application builds a full-text search engine that supports semantic retrieval of historical meeting records. This allows users to quickly and accurately obtain key information from past meetings, providing a strong foundation for enterprise knowledge management and decision support. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a schematic diagram of the structure of the intelligent conference management system based on the large model provided in this application.

[0023] Figure 2 A flowchart of the large model-based intelligent conference management method provided in this application.

[0024] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0025] The various embodiments of the present disclosure will be described more fully below in the detailed description of the specific functional architecture of the large-scale model-based intelligent conference management system. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather that the present disclosure should be understood to encompass all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the present disclosure.

[0026] Hereinafter, the terms "include" or "may include" as used in various embodiments of the present disclosure indicate the presence of disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. In addition, as used in various embodiments of the present disclosure, the terms "include," "have," and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be understood as excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing.

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] See also Figure 1 The figure shows a system structure diagram of a large-scale model-based intelligent conference management system in a specific embodiment, wherein the system includes: a pre-meeting module, a mid-meeting module, and a post-meeting module; The pre-meeting module is used to automatically generate and send meeting invitations through integration with the enterprise calendar system, use telephone notifications and IVR interaction to confirm the form of participation, and conduct real-time statistics on participation and attendance status to prepare for the meeting. Specifically, the pre-meeting module can achieve the following functions: Automatic generation of meeting invitations: The system is integrated with the enterprise schedule management system to automatically generate meeting invitations and send them to participants based on the meeting theme, time, location and other information set by the meeting organizer.

[0029] Automatically notify meeting time and location by phone: Through the built-in telephone call module, participants are automatically dialed at the set time and the meeting time and location are notified by voice.

[0030] IVR (button / dialogue) confirmation of participation mode: After calling the participants, through the interactive voice response system, participants can select the participation mode (on-site participation, remote participation, etc.) by pressing buttons or voice dialogue.

[0031] Automatically output attendance statistics: The system collects and counts the responses of participants in real time, and automatically generates a attendance statistics table including information such as participant name and attendance form.

[0032] Automatic phone call reminder for upcoming meetings: At a certain time (configurable) before the meeting starts, the system will automatically call again to remind the participants.

[0033] Automatically count attendees: At the start of a meeting, the system integrates with the conference room check-in system or other recognition systems (such as facial recognition, QR code scanning, etc.) to automatically count the actual attendees.

[0034] In a specific embodiment, the pre-meeting module includes: an automatic meeting invitation generation unit, an automatic telephone notification unit, an IVR meeting participation confirmation unit, a meeting attendance statistics unit, an automatic telephone reminder unit, and an attendance statistics unit.

[0035] The automatic meeting invitation generation unit is used to integrate with the enterprise schedule management system through API, automatically extract the metadata of the meeting, generate standardized meeting invitation information, and send it in batches to the participants' email addresses or mobile terminals through the network protocol.

[0036] For example, the automatic meeting invitation generation unit integrates with enterprise calendar management systems such as Outlook and Google Calendar via APIs, automatically extracting metadata such as meeting subject, time, location, and attendee list, generating standardized meeting invitations, and sending them in batches to attendees' emails and mobile devices via the SMTP protocol. These standardized meeting invitations include iCalendar attachments.

[0037] The automatic telephone notification unit is used to automatically dial according to the time set in the meeting schedule based on the built-in telephone call module, and the voice conversion technology notifies participants of the meeting information in voice form.

[0038] For example, the automatic phone notification unit is based on Twilio or Amazon Connect API, automatically dials according to the scheduled time, and notifies participants of the meeting information in voice form through TTS technology. It supports multi-language broadcasting and retry mechanisms (such as automatic call back when no one answers).

[0039] The IVR confirmation unit is used to enable participants to select their participation mode through keystrokes or voice dialogues through a multi-round interactive voice response process, and the feedback results are updated to the database in real time.

[0040] For example, the IVR confirmation unit builds a multi-round interactive voice response process. Participants can select their participation mode by pressing buttons (e.g., "1 confirms in-person attendance, 2 selects remote access") or through voice dialogue (supporting dialect recognition). The feedback results are updated to the database in real time, triggering subsequent processes. These subsequent processes include, but are not limited to, sending the remote participation link.

[0041] The attendance statistics unit is used to collect and count the attendance form reply information of the participants in real time and generate a dynamic statistics table; the dynamic statistics table includes the participant's name, contact information, attendance form and confirmation status information.

[0042] For example, the attendance statistics unit aggregates email replies, IVR confirmation results, and manual feedback data in real time to generate dynamic statistical tables, support Excel / PDF export, and synchronize with the enterprise OA system.

[0043] The automatic telephone reminder unit is used to automatically trigger telephone reminders for participants within a preset time before the meeting starts and broadcast the meeting countdown information.

[0044] For example, the automatic telephone reminder unit can automatically trigger a telephone reminder 30 minutes before the meeting starts, broadcast the meeting countdown information, and support voice confirmation of changes in the participant status. Abnormal status triggers an alarm to the meeting organizer.

[0045] The attendance statistics unit is used to integrate with the preset personnel recognition system in the conference room to collect the actual attendance data in real time and automatically count the actual attendance data.

[0046] For example, the attendance statistics unit is integrated with the access control system, NFC check-in device, or QR code scanning terminal to collect data on actual attendees in real time, automatically compare it with the preset list, generate absence / lateness reports, and push them to the administrator terminal via Webhook.

[0047] The in-meeting module is used to collect multi-source audio based on a microphone array during a meeting, combine speech recognition technology to transcribe the meeting content in real time and generate subtitles, simultaneously record video, record meeting process information, and perform structured analysis.

[0048] The in-meeting module uses a highly sensitive microphone array to capture meeting audio and, combined with advanced speech recognition technology, converts the audio into text. The system also supports video recording of the meeting, providing a comprehensive record of the meeting process.

[0049] In a specific implementation, the in-meeting module includes: a multimodal data acquisition unit, a speech transcription and real-time subtitle unit, and a conference content structuring unit.

[0050] The multimodal data acquisition unit is used to collect conference audio using a circular microphone array, and simultaneously connect to multiple cameras to record conference video, and upload the conference audio and video to cloud storage in real time.

[0051] For example, the multimodal data acquisition unit uses an 8-channel circular microphone array (supporting beamforming noise reduction) to collect conference audio, simultaneously connects to multiple cameras (supporting H.264 / 265 encoding) to record video, and uploads the data to cloud storage in real time.

[0052] The speech transcription and real-time subtitle unit is used to perform speech separation and speech information transcription of multiple speakers based on an improved automatic speech recognition model, generate real-time subtitle information for the meeting, and push it to the mobile terminals of the participants.

[0053] For example, the speech transcription and real-time subtitle unit is based on the improved Wav2Vec 2.0 model (, which achieves multi-speaker speech separation and high-precision transcription, supports real-time subtitle generation in Chinese and English, and pushes it to the participant's terminal via WebSocket.

[0054] Among them, the Wav2Vec 2.0 model is a powerful automatic speech recognition model. Through self-supervised learning, it uses a large amount of unlabeled speech data to learn the deep representation of audio signals, providing powerful feature representation for downstream tasks such as speech recognition. The model has the characteristics of efficient fine-tuning and cross-language application.

[0055] The conference content structuring unit is used to perform real-time semantic analysis on the real-time subtitle information of the conference, mark key topics, decision points and controversial content, and generate a dynamic conference context diagram.

[0056] For example, the meeting content structuring unit performs real-time semantic analysis on the transcribed text, marking key topics (such as "budget discussion"), decision points (such as "approval of Plan A") and controversial content, generating a dynamic meeting context map to support the host in quickly identifying key points.

[0057] The post-meeting module uses natural language processing technology to extract key instructions and to-do items from the meeting process information, generate standardized meeting minutes, automatically assign tasks and track progress, and support semantic retrieval of historical meeting records.

[0058] Specifically, the post-meeting module can achieve the following functions: Generate leadership instructions and next steps based on recordings: Through intelligent analysis of meeting recordings, natural language processing technology is used to extract the instructions in the leadership speech and the next work plan determined by the meeting.

[0059] Output complete meeting minutes in standard format: Generate meeting minutes based on the extracted information in a preset standard format (such as meeting topic, time, location, participants, meeting content, leadership instructions, to-do items, etc.).

[0060] Track the progress of to-do items based on meeting minutes: The system automatically breaks down the to-do items in the meeting minutes, assigns them to the corresponding responsible persons, and sets tracking reminders. The responsible persons can update the progress of the to-do items in the system.

[0061] Historical meeting record query: All meeting records (including relevant information before, during and after the meeting) are stored in the database, providing a convenient query interface where users can search by keywords such as meeting topic, time, participants, etc.

[0062] In a specific implementation, the post-meeting module includes: a leadership instruction extraction unit, a task tracking and reminder unit, and a historical meeting retrieval unit.

[0063] The leadership instruction extraction unit is used to use the speech recognition model to identify key information in the leader's speech and generate complete instructions based on the context.

[0064] For example, the leadership instruction extraction unit performs semantic role labeling based on the BERT model, identifies the "action-subject-object" triples in the leader's speech, and associates the context to generate complete instructions.

[0065] The meeting minutes generation unit is used to automatically fill in key instructions in the meeting information using preset templates, and generate meeting minutes after formatting conversion; it supports ISO standard format export and custom style adjustment.

[0066] For example, the meeting minutes generator automatically populates fields such as meeting topic, time, location, attendee list, discussion summary, decision results, and to-do items based on a preset template. It supports exporting to ISO standard formats and customizing styles. Preset templates are available in Markdown, Word, HTML, and other formats.

[0067] The task tracking and reminder unit is used to break down to-do items in meeting information into subtasks through project and transaction tracking tools, and automatically assign responsible persons and deadlines.

[0068] For example, the task tracking and reminder unit breaks down to-do items into subtasks through Jira or Teambition API, automatically assigns responsible persons and deadlines, triggers enterprise WeChat or DingTalk reminders for overdue tasks, and synchronizes task progress to meeting minutes in real time.

[0069] The historical meeting retrieval unit is used to build a full-text retrieval engine, which uses the full-text retrieval engine to retrieve historical meeting records in cloud storage based on keywords, and displays the retrieval results in the form of a timeline or graph.

[0070] For example, the historical meeting retrieval unit can be used to build an Elasticsearch full-text search engine, and supports multi-dimensional queries such as keywords (such as "2024 Q1 budget"), participant names, time ranges, etc. The search results are displayed in the form of a timeline or graph, and support voice assistant interaction.

[0071] In this embodiment, the system integrates functions such as enterprise schedule management, automatic telephone notification, IVR interactive confirmation, real-time voice transcription and subtitle generation, structured analysis of meeting content, extraction of key instruction information and to-do items, generation of standardized meeting minutes, and automatic task allocation and tracking. It greatly reduces manual operations, ensures the accurate communication of meeting information, facilitates the storage and query of meeting records, and provides strong support for enterprise decision-making and work advancement.

[0072] like Figure 2 As shown, the following is an embodiment of the big model-based intelligent conference management method provided by the embodiment of the present disclosure. This method and the big model-based intelligent conference management system of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the big model-based intelligent conference management method, please refer to the above-mentioned big model-based intelligent conference management system embodiment.

[0073] An intelligent conference management method based on a large model includes the following steps: S1: Automatically generate and send meeting invitations through integration with the enterprise calendar system, use phone notifications and IVR interaction to confirm the form of participation, and conduct real-time statistics on participation and attendance status to prepare for the meeting.

[0074] In a specific implementation manner, the specific process of this step is as follows: S101: extracting the meeting metadata of the enterprise calendar system through the API interface, automatically generating standardized meeting invitation information, and sending it in batches to the participant terminals through the network protocol.

[0075] Specifically, the enterprise calendar system is called through the RESTful API, and meeting metadata (time, location, and attendee email addresses) is parsed using regular expressions or JSON Schema. Meeting invitation emails are sent in batches based on the SMTP protocol. The email body is dynamically rendered using the Jinja2 template engine, and an iCalendar standard format attachment is attached. An exponential backoff retry mechanism is enabled for emails that are not successfully sent (maximum retry count = 3), and failure records are written to the MySQL database exception log table.

[0076] S102: triggering a telephone call system based on a preset time, and broadcasting conference information to the participants through voice conversion technology.

[0077] Specifically, a Cron scheduled task or cloud function is used to trigger the Twilio / Amazon Connect API to initiate a voice call. The Google Cloud Text-to-Speech (TTS) engine is used to convert conference information into a voice stream, and the audio encoding adopts the Opus format (48kHz, 64kbps). For missed calls, the call tasks are managed through a round-robin queue, the number of automatic redials is ≤2, and the call status is written to the Redis cache in real time.

[0078] S103: Receive keystroke or voice feedback from the participants through multiple rounds of IVR interaction processes, and update the participation form confirmation information to the database in real time.

[0079] Specifically, a multi-round dialogue engine is built based on Amazon Lex, integrating a speech recognition (ASR) model. The feedback results are updated to the attendance table in the MySQL database through SQL UPDATE statements. Among them, the key response uses the Goertzel algorithm to detect the frequency and supports dual-tone multi-frequency signal analysis. By identifying abnormal input, the preset voice prompt branch is triggered, and the alarm is pushed to the administrator terminal through Webhook.

[0080] S104: Dynamically generate a statistical form containing the participant's identity information, contact information, and confirmation status.

[0081] Specifically, the Pandas framework is used to aggregate and analyze participant data to generate an Excel file containing a pivot table; the statistical results are synchronized with the enterprise OA system through the OAuth 2.0 protocol.

[0082] S105: Automatically initiate a phone reminder at a preset threshold time before the meeting starts, and broadcast the meeting countdown information.

[0083] Specifically, the reminder threshold is determined to be 30 minutes based on the meeting start time, and the reminder trigger condition is written to the Celery task queue.

[0084] Dynamically generate voice broadcast content through string templates and record call results to the Elasticsearch log system.

[0085] S106: Collect actual attendance data through the conference room personnel identification system and generate a real-time attendance statistics report.

[0086] Specifically, the YOLOv5 target detection model or NFC card reader is used to collect actual attendance data; the real-time attendance data is pushed to the front-end dashboard through WebSocket, the Levenshtein distance algorithm is used to match the names of attendees, and a real-time attendance statistics report is generated.

[0087] S2: During the meeting, the system collects multi-source audio based on a microphone array, combines speech recognition technology to transcribe the meeting content in real time and generate subtitles, simultaneously records the video, records the meeting process information, and performs structured analysis.

[0088] In a specific implementation manner, the specific process of this step is as follows: S201: Use a circular microphone array to capture multi-channel conference audio, and use a synchronous multi-channel video capture device to record conference images.

[0089] When collecting multi-channel conference audio, an 8-channel ring MEMs microphone array (based on the GCC-PHAT algorithm) is used to implement beamforming and dynamically suppress ambient noise. The audio sampling rate is set to 48kHz, the quantization bit is 24bit, and the audio is transmitted to the cloud server in real time via the WebRTC protocol.

[0090] When recording conference images, multiple 4K cameras are used, supporting H.264 encoding and a frame rate of 30fps; NTP calibration is used for audio and video synchronization, with a timestamp error of ≤10ms, and the original data is stored in the MinIO object storage system.

[0091] S202: Apply the improved ASR model to separate and recognize the multi-speaker voices, and generate a conference subtitle stream with a time stamp in real time.

[0092] Among them, the improved ASR model is based on the Wav2Vec 2.0 architecture, adds a speaker separation module, uses the PyAnnote toolkit to extract voiceprint embedding vectors, and adopts the Conv-TasNet model to achieve multi-source separation.

[0093] S203: Perform semantic analysis on the conference subtitle stream, dynamically annotate key topics, decision nodes, and controversial content during the conference, and generate a visual conference context map.

[0094] S3: Use natural language processing technology to extract key instructions and to-do items from meeting process information, generate standardized meeting minutes, automatically assign tasks and track task progress.

[0095] In a specific implementation manner, the specific process of this step is as follows: S301: extracting the instruction information of a specific speaker through voice feature recognition technology, generating a complete instruction text based on the context, and extracting key instruction information and to-do items in the meeting process information.

[0096] Specifically, the speaker identity is extracted through the voiceprint recognition model ResNet-34, and the BERT model is combined to perform context-dependent syntactic analysis, identify the "action-subject-object" triples, extract to-do items and match them with the preset keyword library.

[0097] S302: Automatically fill in meeting elements using a preset template according to the meeting process information to generate a formatted meeting minutes document that complies with ISO standards.

[0098] Specifically, based on the Jinja2 template engine, the meeting elements are automatically filled in according to the meeting process information in accordance with the ISO / IEC 26300 standard to generate a structured Word or PDF document.

[0099] S303: Decompose the to-do items into structured task items, automatically associate the responsible persons and set the execution time nodes.

[0100] Specifically, the TextRank algorithm is used to decompose the to-do items into subtasks, the responsible persons are matched through cosine similarity, and the execution time nodes are set according to the work calendar.

[0101] S304: Construct a retrieval model based on semantic vectors, and retrieve historical meeting content through natural language queries according to retrieval requirements.

[0102] Specifically, the Sentence-BERT model is used to encode meeting texts into semantic vectors, and an index is constructed based on the Faiss framework.

[0103] S305: Dynamically monitor the task execution status and trigger a progress reminder at a preset time node.

[0104] Specifically, the task status is polled every five minutes through the Jira API, the reminder time is set using the Cron expression, and the DingTalk robot is automatically triggered to provide progress reminders.

[0105] The big model-based intelligent conference management method provided in this embodiment integrates automated schedule management and intelligent conference technology to achieve full-process intelligence of the conference from preparation to execution, recording and subsequent task management, effectively improving meeting efficiency and ensuring accurate recording of information and timely follow-up of tasks.

[0106] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0107] The large-scale model-based intelligent conference management method provided in the embodiments of the present application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange components differently. In the embodiments of the present invention, electronic devices include but are not limited to laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0108] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a button, a camera, a display, and a SIM card interface, etc.

[0109] A processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0110] The processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0111] The processor may also include a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or is reusing. If the processor needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0112] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of an electronic device. The external memory card communicates with the processor through the external memory interface, enabling data storage. For example, files such as music and videos can be stored on the external memory card.

[0113] Internal memory can be used to store computer-executable program code, which includes instructions. The processor executes the instructions stored in the internal memory to perform various functional applications and data processing of the electronic device. The internal memory can include a program storage area and a data storage area. The internal memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0114] The wireless communication function of an electronic device can be implemented through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0115] Wireless communication modules can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0116] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0117] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0118] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0119] A GPU is a microprocessor for image processing that connects the display screen to the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0120] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0121] The electronic device implements the large-scale model-based intelligent conference management method of this application by integrating with the enterprise calendar system to automatically generate and send meeting invitations, combining telephone notifications and IVR interaction technology to confirm the form of participation, and utilizing advanced audio acquisition, speech recognition, video recording, and natural language processing technologies to fully automate the preparation, recording, and subsequent task management of meetings. The application of this series of intelligent means has achieved the beneficial effects of significantly improving the efficiency of meeting preparation, ensuring the complete recording of meeting information, optimizing the meeting management process, promoting the rapid execution of decisions, and enhancing task tracking capabilities.

[0122] The storage medium provided in this application stores a program product that can implement an intelligent conference management method based on a large model.

[0123] Intelligent conference management methods based on large models include: Automatically generate and send meeting invitations through integration with the enterprise calendar system, use phone notifications and IVR interaction to confirm attendance, and conduct real-time statistics on attendance and attendance status to prepare for meetings. During the meeting, the system collects multi-source audio using a microphone array, combines speech recognition technology to transcribe the meeting content in real time and generate subtitles, simultaneously records video, records meeting process information, and performs structured analysis. Natural language processing technology is used to extract key instructions and to-do items from meeting process information, generate standardized meeting minutes, automatically assign tasks and track task progress.

[0124] In some possible implementations, the big model-based intelligent conference management method disclosed herein can be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of this specification.

[0125] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0126] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent conference management system based on a large model, characterized in that: include: The pre-meeting module is used to automatically generate and send meeting invitations through integration with the enterprise calendar system, confirm the form of participation through telephone notifications and IVR interaction, and conduct real-time statistics on participation and attendance status to prepare for the meeting; The in-meeting module is used to collect multi-source audio using a microphone array during a meeting, transcribe the meeting content in real time and generate subtitles using speech recognition technology, simultaneously record the video, record the meeting process information, and perform structured analysis; The post-meeting module uses natural language processing technology to extract key instructions and to-do items from the meeting process information, generate standardized meeting minutes, automatically assign tasks and track progress, and support semantic retrieval of historical meeting records.

2. The intelligent conference management system based on a large model according to claim 1 is characterized in that: The pre-meeting module includes: an automatic meeting invitation generation unit, an automatic telephone notification unit, an IVR meeting participation confirmation unit, a meeting attendance statistics unit, an automatic telephone reminder unit, and an attendance statistics unit; The automatic meeting invitation generation unit is used to integrate with the enterprise calendar management system through API, automatically extract the meeting metadata, generate standardized meeting invitation information, and send it in batches to the participants' email addresses or mobile terminals through network protocols; Automatic telephone notification unit, which is used to automatically dial according to the meeting schedule based on the built-in telephone call module, and use voice conversion technology to notify participants of meeting information in voice form; The IVR confirmation unit is used to enable participants to select their participation mode through keystrokes or voice dialogues through a multi-round interactive voice response process, and the feedback results are updated to the database in real time; A participant statistics unit is used to collect and count the participant's participation form reply information in real time and generate a dynamic statistics table; the dynamic statistics table includes the participant's name, contact information, participation form and confirmation status information; Automatic telephone reminder unit, used to automatically trigger telephone reminders for participants within a preset time before the meeting starts and broadcast the meeting countdown information; The attendance statistics unit is used to integrate with the preset personnel recognition system in the conference room to collect the actual attendance data in real time and automatically count the actual attendance data.

3. The intelligent conference management system based on a large model according to claim 2 is characterized in that: The in-meeting module includes: a multimodal data acquisition unit, a speech transcription and real-time subtitle unit, and a meeting content structuring unit; A multimodal data acquisition unit, which uses a circular microphone array to collect conference audio, simultaneously connects to multiple cameras to record conference video, and uploads the conference audio and video to cloud storage in real time; The speech transcription and real-time subtitle unit is used to separate and transcribe the speech of multiple speakers based on an improved automatic speech recognition model, generate real-time subtitle information for the meeting, and push it to the mobile terminals of participants; The conference content structuring unit is used to perform real-time semantic analysis on the real-time subtitle information of the conference, mark key topics, decision points and controversial content, and generate a dynamic conference context diagram.

4. The intelligent conference management system based on a large model according to claim 3 is characterized in that: The post-meeting module includes: a leadership instruction extraction unit, a task tracking and reminder unit, and a historical meeting retrieval unit; The leader instruction extraction unit is used to use the speech recognition model to identify key information in the leader's speech and generate complete instructions based on the context; Meeting minutes generation unit, used to automatically fill in key instructions in meeting information using preset templates, format and convert them into meeting minutes; supports ISO standard format export and custom style adjustment; Task tracking and reminder unit, used to break down to-do items in meeting information into subtasks through project and issue tracking tools, automatically assigning responsible persons and deadlines; The historical meeting retrieval unit is used to build a full-text retrieval engine, which uses the full-text retrieval engine to retrieve historical meeting records in cloud storage based on keywords, and displays the retrieval results in the form of a timeline or graph.

5. An intelligent conference management method based on a large model, characterized in that: The method adopts the intelligent conference management system based on the large model as claimed in any one of claims 1 to 4; The method comprises: Automatically generate and send meeting invitations through integration with the enterprise calendar system, use phone notifications and IVR interaction to confirm attendance, and conduct real-time statistics on attendance and attendance status to prepare for meetings. During the meeting, the system collects multi-source audio using a microphone array, combines speech recognition technology to transcribe the meeting content in real time and generate subtitles, simultaneously records video, records meeting process information, and performs structured analysis. Natural language processing technology is used to extract key instructions and to-do items from meeting process information, generate standardized meeting minutes, automatically assign tasks and track task progress.

6. The intelligent conference management method based on a large model according to claim 5 is characterized in that: The system automatically generates and sends meeting invitations through the integration of the enterprise calendar system, uses telephone notifications and IVR interaction to confirm the form of participation, and conducts real-time statistics on participation and attendance status to prepare for the meeting, including: Extract meeting metadata from the enterprise calendar system through the API interface, automatically generate standardized meeting invitation information, and send it to participants' terminals in batches via network protocols; Trigger the phone call system based on the preset time and broadcast the meeting information to the participants through voice conversion technology; Receive keystroke or voice feedback from participants through multiple rounds of IVR interaction, and update the participant confirmation information to the database in real time; Dynamically generate statistical forms containing participant identity information, contact information, and confirmation status; Automatically initiate a phone reminder at a preset threshold time before the meeting starts, broadcasting the meeting countdown information; The actual attendance data is collected through the conference room personnel identification system to generate real-time attendance statistics reports.

7. The intelligent conference management method based on a large model according to claim 6 is characterized in that: During the meeting, the microphone array is used to collect multi-source audio, and speech recognition technology is used to transcribe the meeting content in real time and generate subtitles. The video is recorded synchronously, the process information of the meeting is recorded, and structured analysis is performed, including: Use a circular microphone array to capture multi-channel conference audio, and use synchronous multi-channel video capture equipment to record conference images; Apply an improved ASR model to separate and recognize the speech of multiple speakers and generate a conference subtitle stream with timestamps in real time; The conference subtitle stream is semantically analyzed, key topics, decision nodes and controversial contents in the conference process are dynamically annotated, and a visual conference context map is generated.

8. The intelligent conference management method based on a large model according to claim 7 is characterized in that: The method uses natural language processing technology to extract key instructions and to-do items from meeting process information, generate standardized meeting minutes, automatically assign tasks, and track task progress, including: The system uses speech feature recognition technology to extract the specific speaker's instruction information, combines the context to generate a complete instruction text, and extracts key instructions and to-do items from the meeting process information; Automatically fill in meeting elements using preset templates based on meeting process information to generate formatted meeting minutes documents that comply with ISO standards; Break down to-do items into structured tasks, automatically associate responsible persons, and set execution time nodes; Build a retrieval model based on semantic vectors to retrieve historical meeting content through natural language queries according to retrieval requirements; Dynamically monitor the task execution status and trigger progress reminders at preset time nodes.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the large model-based intelligent conference management method as described in any one of claims 5 to 8 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large model-based intelligent conference management method according to any one of claims 5 to 8 are implemented.

Images