Artificial intelligence automatic response conference assistant

Abstract

The invention discloses an artificial intelligence automatic response conference assistant, and particularly relates to the technical field of intelligent video conferences. The system comprises a user interface used for displaying conference information, function options and execution results, a large language model used for natural language processing and understanding, and a language recognition module used for converting voice input into characters to achieve real-time conference recording and voice instruction recognition. By means of friendly user interface design, conference information, function options and execution results can be visually displayed, a user can easily operate and master the whole conference process, the voice recognition module adopts a high-quality recording module, voice content in a conference is accurately transferred into characters in real time, and the user experience is improved. The completeness and the accuracy of conference records are ensured, the large language model further performs semantic understanding and processing on the transferred text, intelligent conference abstracts and responses are generated, a user is helped to rapidly obtain key information, and the postprocessing module automatically sends the abstracts to a user mailbox to achieve the purpose of meeting. Or the system is automatically integrated to an online office system of a user, so that a lot of time and energy are saved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent conference technology, and in particular to an artificial intelligence automatic answering conference assistant. Background Art

[0002] At present, remote video conferencing has become a widely accepted way of office collaboration around the world. With the popularization of smartphones and laptops, and most devices equipped with front cameras, combined with the rapid development of 5G networks, the convenience of remote video conferencing has been greatly improved. However, although video conferencing provides great convenience in connection and communication, it still faces some obvious pain points. Video conferencing usually takes a long time, especially when the content of the meeting is lengthy and not completely related to the responsibilities of each participant, participants often feel inefficient and dissatisfied. During the meeting, many participants may not need to pay attention to all the discussion content, but because the meeting time is too long, they still have to continue to participate, which not only reduces the efficiency of the meeting, but also increases the work pressure of the participants. In response to these problems, how to improve the efficiency of meetings, reduce unnecessary time waste of participants, and improve the interactivity and content accessibility during the meeting have become technical challenges that need to be solved urgently.

[0003] Traditional video conferencing systems often lack intelligent participation management and information extraction functions. Conference content mostly relies on manual recording or self-organization by participants, which easily leads to omission of information or inaccurate recording. Although some systems have voice transcription functions, the accuracy and real-time performance are often insufficient, and the transcribed text content lacks in-depth semantic analysis and summary, which cannot effectively help participants quickly grasp the core information of the meeting. Most current systems fail to fully consider the personalized needs of participants, such as automatically judging the attention of participants in long meetings, or automatically providing relevant information feedback in meetings. These defects make video conferencing still unable to meet the needs of efficient and intelligent meetings. Therefore, we provide an artificial intelligence automatic answering conference assistant. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an artificial intelligence automatic answering conference assistant.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An artificial intelligence automatic answering conference assistant includes a user interface for displaying conference information, function options and execution results, a large language model for natural language processing and understanding, a language recognition module for converting voice input into text to realize real-time conference recording and voice command recognition, a recording module for capturing and storing audio content in the meeting, a main application for integrating all functional modules to manage the overall workflow, an audio playback module for playing audio recorded in the meeting or audio prompts generated by the system, a portrait recognition module for identifying and marking portrait information of conference participants, and an audio and video driver for providing hardware interface support to ensure the normal operation of audio and video devices and data transmission.

[0007] The present invention is further configured as follows: the user interface receives the user's operation instructions through the main application and displays the processing results; the speech recognition module relies on the audio data provided by the recording module to perform accurate speech transcription.

[0008] The present invention is further configured as follows: the speech recognition module sends the transcribed text to the large language model for further semantic understanding and processing; the large language model passes the text or instructions recognized by the language recognition module to perform natural language understanding and generate responses and feed back to the user interface; the post-processing module automatically distributes the meeting summary to the user's information platform based on artificial intelligence reasoning and user selection.

[0009] The present invention is further configured as follows: the main application starts and stops the recording function of the recording module by sending control instructions, and receives the audio data captured by the recording module for storage, management and subsequent processing, while configuring the parameters of the recording module to ensure accurate recording of the conference audio; the recording module relies on the audio and video driver to manage the recording function of the audio device.

[0010] The present invention is further configured as follows: the audio data of the recording module is combined with the image data of the portrait recognition module to accurately record the speaker and his / her speech content and transmit it to the main application; the portrait recognition module obtains video data through an audio and video driver to perform real-time facial recognition and tracking.

[0011] The present invention is further configured as follows: the recording module relies on an audio and video driver to manage the recording function of the audio device to ensure high-quality capture of audio data; the user interface feeds back the user's operation content to the main application and performs corresponding operations.

[0012] The beneficial effects of the present invention are:

[0013] 1. The present invention has a friendly user interface design, which can intuitively display meeting information, function options and execution results, allowing users to easily operate and control the entire meeting process. The voice recognition module relies on a high-quality recording module to accurately transcribe the voice content in the meeting into text in real time, ensuring the integrity and accuracy of the meeting minutes. The large language model further performs semantic understanding and processing on the transcribed text, generates intelligent meeting summaries and responses, and helps users quickly obtain key information, saving a lot of time and energy. The close cooperation between the audio and video driver and the recording module ensures the efficient transmission and high-quality output of audio data, avoids the problem of poor sound quality or data loss, and improves the overall audio and video experience. The portrait recognition module accurately records each speaker and his or her speech content through real-time facial recognition and tracking, which not only enhances the interactivity of the meeting, but also provides rich data support for subsequent meeting analysis and review.

[0014] 2. The present invention determines whether the participant is present by periodically calling the camera data. If it is detected that the user is not present, the recording program is automatically started to record only the audio content, which not only ensures the integrity of the meeting minutes but also respects the privacy of the user. When someone calls the user's name in the meeting, the assistant can automatically reply with preset information to ensure the continuity and interactivity of the meeting, and at the same time record the questions and submit them to the user, which improves the response speed of the meeting and the user experience. After the meeting, the post-processing function of the assistant further enhances the practicality of the meeting minutes.

[0015] 3. The present invention converts the recorded content into text through a speech recognition engine, and then performs summary processing through a large language model to generate a meeting summary that is easy to read and consult. Users can choose a variety of acquisition methods, such as sending emails, integrating into various document platforms or instant messaging tools, which greatly facilitates the sharing and storage of information. The system also supports automatic playback of pre-recorded voice prompts, further improving the user experience and intelligent management of meetings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the operation flow in the present invention;

[0017] Figure 2 A schematic diagram of the distribution flow of the post-meeting summary post-processing module in the present invention;

[0018] Figure 3 It is a schematic diagram of the software module in the present invention. DETAILED DESCRIPTION

[0019] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments, and the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0020] It should be further explained that the drawings and implementation modes of the present invention mainly describe the concept of the present invention. On the basis of this concept, the specific forms and settings of some connection relationships, positional relationships, power mechanisms, power supply systems, hydraulic systems and control systems may not be fully described. However, on the premise that those skilled in the art understand the concept of the present invention, those skilled in the art can implement the above-mentioned specific forms and settings in a well-known manner.

[0021] When an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or indirectly connected to the other element.

[0022] The directional words "inside and outside" refer to the inside and outside relative to the outline of each component itself. The directions or positional relationships indicated by the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the directions or positional relationships shown in the drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0023] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" may include both "above" and "below". The device may also be positioned in other different ways, and the spatially relative descriptions used here are interpreted accordingly.

[0024] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0025] An artificial intelligence automatic answering conference assistant provided by the present invention is now described.

[0026] Example 1

[0027] like Figure 1-3 As shown, an artificial intelligence automatic answering conference assistant includes a user interface for displaying conference information, function options and execution results, a large language model for natural language processing and understanding, a language recognition module for converting voice input into text to realize real-time conference recording and voice command recognition, a recording module for capturing and storing audio content in the meeting, a main application for integrating all functional modules to manage the overall workflow, an audio playback module for playing audio recorded in the meeting or audio prompts generated by the system, a portrait recognition module for identifying and marking portrait information of conference participants, and an audio and video driver for providing hardware interface support to ensure the normal operation of audio and video devices and data transmission;

[0028] The user interface receives the user's operating instructions through the main application and displays the processing results; the speech recognition module relies on the audio data provided by the recording module to perform accurate speech transcription; the speech recognition module sends the transcribed text to the large language model for further semantic understanding and processing; the large language model passes the text or instructions recognized by the language recognition module to perform natural language understanding and generate responses and feedback to the user interface; the post-processing module automatically distributes the meeting summary to the user's information platform based on artificial intelligence reasoning and user selection; the main application starts and stops the recording function of the recording module by sending control instructions, and receives the audio data captured by the recording module for storage, management and subsequent processing, and configures the parameters of the recording module to ensure accurate recording of the meeting audio.

[0029] In the above embodiment, during actual use, the artificial intelligence video conferencing assistant will call the camera data of the application and make a judgment at the beginning of the meeting with a time interval preset to one second as the threshold. If the staff is in front of the camera, the meeting will proceed normally. If the staff is not in front of the camera, the recording program will be started to record the content of the meeting. At the same time, the time interval preset to 200 milliseconds is used as the threshold to monitor whether the user is called during the meeting. If not, the meeting process proceeds normally. If the name of the calling user is monitored during the meeting, the artificial intelligence video conferencing assistant will automatically reply to tell the participant that the person is not in front of the computer, and ask the questioner to continue asking questions. The assistant will record the question and submit it to the user himself.

[0030] Among them: users can choose the way to obtain the summary, including but not limited to: sending email, integrating into Tencent Docs, Kingsoft WPS, etc., integrating into Internet document platforms such as Notion and Microsoft 365, sending WeChat messages, sending DingTalk and Feishu messages, etc., automatic printing, etc.

[0031] Among them: the user can choose that when the assistant recognizes that the user is not there, the assistant can automatically play a pre-recorded sound: for example, "Hello, I am not at my seat now, please leave me a message and I will follow up later."

[0032] Example 2

[0033] like Figure 1-3 As shown, an artificial intelligence automatic answering conference assistant, the recording module relies on an audio and video driver to manage the recording function of the audio device; the audio data of the recording module is combined with the image data of the portrait recognition module to accurately record the speaker and his speech content and transmit it to the main application; the portrait recognition module obtains video data through the audio and video driver to perform real-time facial recognition and tracking; the recording module relies on the audio and video driver to manage the recording function of the audio device to ensure high-quality capture of audio data; the user interface feeds back the user's operation content to the main application and makes corresponding operations.

[0034] In the above embodiment, after the meeting is over, the artificial intelligence video conferencing assistant will review its own application process, mainly reviewing whether the meeting was recorded, and after determining that it was, it will call the language recognition engine to convert the content of the meeting process into text through voice, and then call the large language model to summarize the problems after the conversion by extracting important meeting records for user reading.

[0035] Among them, the user can choose that when the assistant recognizes that the user is not present and the user's name is called in the meeting, the assistant can automatically send a message in the meeting group (chat window of the meeting software), for example "Hello, I am not at my seat now, please leave me a message and I will follow up later."

[0036] Working principle: When the present invention is used, through the collaborative work of multiple modules, efficient and intelligent conference management and recording functions are realized. First, the user interacts with the main application through the user interface, and the interface is responsible for displaying the conference information, function options and execution results. The user's operation instructions are passed to each functional module through the main application to ensure that the user's needs can be responded to and executed in a timely manner. The speech recognition module relies on the high-quality audio data provided by the recording module to accurately transcribe the speech and convert the speech content in the meeting into text in real time. These transcribed texts are then sent to the large language model for semantic understanding and processing, thereby generating intelligent meeting records and responses. In addition, the audio and video driver is responsible for the underlying communication with the hardware device, managing the recording and playback functions of the audio and video devices, and ensuring the efficient transmission and high-quality output of audio data. The portrait recognition module performs real-time facial recognition and tracking through the video data obtained by the audio and video driver, and combines the audio data of the recording module to accurately record each speaker and his or her speech content. This close collaboration between multiple modules ensures that the conference assistant can comprehensively and accurately capture and process various types of information in the meeting.

[0037] In the actual process of the meeting, this artificial intelligence video conference assistant demonstrated its intelligent workflow. At the beginning of the meeting, the system will call the camera data at a preset time interval (once per second) to determine whether the meeting participants are in front of the camera. If the staff is detected in front of the camera, the system will record the meeting normally; if the staff is detected not in front of the camera, the recording program will be automatically started, and only the audio of the meeting content will be recorded. This not only ensures the complete record of the meeting content, but also protects the privacy of users to a certain extent. During the meeting, the system monitors whether there is a call to the user at a short time interval (200 milliseconds). When it is detected that someone calls the user name in the meeting, the assistant will automatically reply through the artificial intelligence video conference assistant, informing the participants that the user is not in front of the computer temporarily, please continue to ask questions, and record the questions and submit them to the user himself. This function greatly improves the interactivity and efficiency of the meeting, ensuring that users can obtain important information and problems in the meeting in a timely manner even when they are not in front of the computer. In addition, the main application starts and stops the recording function of the recording module by sending control instructions, and configures the relevant parameters of the recording module, such as sound quality, recording format and storage path, to ensure the stability and high quality of the recording process. This intelligent process management makes the entire meeting process smoother and more efficient, and can flexibly adjust the recording strategy according to actual conditions to meet the needs of different meeting scenarios.

[0038] After the meeting, this AI video conferencing assistant also has powerful post-processing functions, which further improves the practicality and convenience of meeting records. The system will review the entire meeting process, mainly checking whether the recording has been made, and if the recording is confirmed, call the speech recognition engine to convert the recording content into text. Subsequently, the large language model summarizes the converted text, extracts important meeting records and key information, and generates a meeting summary that is easy for users to read and consult. Users can choose a variety of ways to obtain meeting summaries, including sending emails, integrating into Internet document platforms such as Tencent Docs, Kingsoft WPS, Notion, Microsoft365, sending WeChat messages, DingTalk, Feishu messages, or automatically printing. These diverse options greatly facilitate users to choose the most appropriate acquisition method according to their own needs, and improve the utilization efficiency of meeting records. In addition, the system also supports automatic playback of pre-recorded voice prompts when the assistant recognizes that the user is not there, such as "Hello, I am not at my seat now, please leave me a message, I will follow up later", further enhancing the user experience and intelligent management of meetings. Through these comprehensive functional modules and intelligent workflows, this artificial intelligence video conferencing assistant can not only efficiently record and manage meeting content, but also improve the overall organization and management level of the meeting through intelligent interaction and post-processing functions, providing users with a comprehensive intelligent conference solution.

[0039] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps described in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

Claims

1. An artificial intelligence automatic answering conference assistant, characterized in that: It includes a user interface for displaying meeting information, function options and execution results, a large language model for natural language processing and understanding, a language recognition module for converting voice input into text to achieve real-time meeting records and voice command recognition, a recording module for capturing and storing audio content in meetings, a main application for integrating all functional modules to manage the overall workflow, a post-processing module for user meeting summary distribution and integration, an audio playback module for playing audio recorded in meetings or audio prompts generated by the system, a portrait recognition module for identifying and marking portrait information of meeting participants, and an audio and video driver for providing hardware interface support to ensure the normal operation of audio and video devices and data transmission.

2. The artificial intelligence automatic answering conference assistant according to claim 1, characterized in that: The user interface receives the user's operation instructions through the main application and displays the processing results; the speech recognition module relies on the audio data provided by the recording module to perform accurate speech transcription.

3. The artificial intelligence automatic answering conference assistant according to claim 1, characterized in that: The speech recognition module sends the transcribed text to the large language model for further semantic understanding and processing; the large language model passes the text or instructions recognized by the speech recognition module to perform natural language understanding and generate responses and feedback to the user interface; The post-processing module automatically distributes the conference summary to the user's information platform based on artificial intelligence reasoning and user selection.

4. The artificial intelligence automatic answering conference assistant according to claim 1, characterized in that: The main application starts and stops the recording function of the recording module by sending control instructions, and receives the audio data captured by the recording module for storage, management and subsequent processing, while configuring the parameters of the recording module to ensure accurate recording of the conference audio; the recording module relies on the audio and video driver to manage the recording function of the audio device.

5. The artificial intelligence automatic answering conference assistant according to claim 1, characterized in that: The audio data of the recording module is combined with the image data of the portrait recognition module to accurately record the speaker and his / her speech content and transmit it to the main application; the portrait recognition module obtains video data through an audio and video driver to perform real-time facial recognition and tracking.

6. The artificial intelligence automatic answering conference assistant according to claim 1, characterized in that: The recording module relies on the audio and video driver to manage the recording function of the audio device to ensure high-quality capture of audio data; the user interface feeds back the user's operation content to the main application and makes corresponding operations.

Images