Conference processing method, device, system, equipment and medium

Abstract

The embodiment of the invention provides a conference processing method, device and system, equipment and a medium. The method specifically comprises the following steps: acquiring conference content; the conference content comprises a conference audio and a conference video; constructing a conference theme structure according to the conference content; the conference theme structure comprises at least one theme; generating an interaction problem corresponding to any theme according to the conference content corresponding to any theme; and sending the interaction problem to the participating users corresponding to the any theme. According to the embodiment of the invention, the integrity and accuracy of the conference abstract can be improved, and the quality of the conference abstract can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular, to a conference processing method, apparatus, system, device and medium. Background Art

[0002] With the rapid development of communication technology, various telephone conferences and video conferences are becoming more and more popular. Meeting summaries are short texts that extract key information from meeting minutes. They are not only an effective way for participants to review the key points and for non-participants to learn about the meeting overview, but also a key basis for post-meeting decision-making and work advancement.

[0003] At present, the method of generating conference summaries usually first uses speech recognition technology to convert the speech in the meeting into conference text; then uses natural language processing technology to extract key information from the conference text and generate a conference summary based on the key information.

[0004] However, since current conference summaries are mainly further integrated based on key information extracted from conference texts, if the key information extracted is inappropriate, it is easy for conference summaries to have problems such as missing information. For example, when generating a conference summary of approximately 200 words for a lengthy conference text of 50,000 words, some important information may be omitted due to the complexity of semantic understanding and the limitations of natural language processing technology in information screening. Summary of the invention

[0005] The embodiment of the present application provides a conference processing method, which can improve the completeness and accuracy of the conference summary, and thus can improve the quality of the conference summary.

[0006] Correspondingly, the embodiment of the present application also provides a conference processing device, an electronic device and a machine-readable medium to ensure the implementation and application of the above method.

[0007] In order to solve the above problems, the present application discloses a conference processing method, which includes:

[0008] Obtaining conference content; the conference content includes: conference audio and conference video;

[0009] Constructing a conference theme structure according to the conference content; the conference theme structure includes: at least one theme;

[0010] Generate interactive questions corresponding to any topic based on the conference content corresponding to any topic;

[0011] Send the interactive question to the participating users corresponding to any of the topics.

[0012] In order to solve the above problems, an embodiment of the present application discloses a conference processing method, which is applied to a conference terminal or a user terminal, and includes:

[0013] Receive interaction questions sent by the server;

[0014] Play the interactive questions;

[0015] Receive response information from the participating users regarding the interactive questions;

[0016] The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

[0017] In order to solve the above problems, the embodiment of the present application discloses a conference processing device, which is applied to a conference terminal or a user terminal, and the device includes:

[0018] The receiving module is used to receive the interaction questions sent by the server;

[0019] A playback module, used to play interactive questions;

[0020] A reply module, used to receive reply information from the participating users to the interactive questions;

[0021] The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

[0022] In order to solve the above problems, the present application discloses a conference processing device, which includes:

[0023] A conference content acquisition module is used to acquire conference content; the conference content includes: conference audio and conference video;

[0024] A theme structure building module, used to build a conference theme structure according to the conference content; the conference theme structure includes: at least one theme;

[0025] An interactive question generation module is used to generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0026] The interactive question sending module is used to send the interactive question to the participating users corresponding to any of the topics.

[0027] Optionally, the interactive question generating module includes:

[0028] A keyword acquisition module is used to acquire keywords from the conference content corresponding to any topic;

[0029] The template question generation module is used to generate interactive questions corresponding to the keywords using question templates.

[0030] Optionally, the process of determining the question template includes:

[0031] Collecting a first data sample; the first data sample includes: a meeting content sample corresponding to the theme of the meeting, and a meeting summary sample of the meeting;

[0032] Based on the sample meeting content and sample meeting abstracts, identify the types of questions and key messages;

[0033] Determine the question template based on the question type and key information.

[0034] Optionally, the keyword acquisition module includes:

[0035] A first keyword acquisition module is used to extract a first keyword that meets a first preset condition from the conference content corresponding to any topic; the first preset condition includes: a keyword that is related to any topic and appears in the conference content of any topic but does not appear in the conference summary; or

[0036] A second keyword acquisition module is used to extract a second keyword that meets the fuzzy condition from the conference content corresponding to any topic; or

[0037] The third keyword acquisition module is used to extract third keywords that meet the second preset condition from the conference content corresponding to any topic; the second preset condition includes: there is a semantic difference between the conference content and the conference summary of any topic.

[0038] Optionally, the question generating module includes:

[0039] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0040] The emotional question generation module is used to generate interactive questions corresponding to any topic according to the emotional state.

[0041] Optionally, the device further comprises:

[0042] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0043] The display module is used to display the first prompt information corresponding to the change of the emotional state of the participating users on the timeline of the conference content.

[0044] Optionally, the sentiment analysis module includes:

[0045] A first emotion determination module, used to determine a first emotional state of a conference user based on facial features of the conference user in a conference video corresponding to any topic;

[0046] A second emotion determination module, used to determine a second emotional state of a conference user according to the body features of the conference user in a conference video corresponding to any topic;

[0047] A third emotion determination module, used to determine a third emotion state of a conference user according to a voice feature of the conference user in the conference audio corresponding to any topic;

[0048] The fusion module is used to fuse the first emotional state, the second emotional state and the third emotional state to obtain the emotional state of the participating user.

[0049] Optionally, the device further comprises:

[0050] A reply receiving module, used to receive reply information of the participating users to the interactive questions;

[0051] The improvement and correction module is used to improve and correct the meeting summary according to the reply information.

[0052] Optionally, the device further comprises:

[0053] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0054] Added module for adding said emotional state in meeting summary.

[0055] Optionally, the device further comprises:

[0056] Summary sending module, used to send conference summaries to conference participants;

[0057] A feedback receiving module, used to receive feedback information from conference participants on the conference summary;

[0058] The model optimization module is used to optimize the processing model according to the feedback information; the processing model includes: a first processing model for generating a meeting summary, or a second processing model for determining an emotional state.

[0059] Optionally, the device further comprises:

[0060] The emotion determination module is used to determine the emotional state of the participating users according to the meeting content corresponding to any topic during the meeting;

[0061] a communication barrier detection module, used for performing communication barrier detection according to the emotional state;

[0062] The prompt sending module is used to send corresponding second prompt information to the participating users when a communication barrier is detected.

[0063] The embodiment of the present application also discloses a conference processing system, including: a conference terminal and a server;

[0064] The server obtains the conference content; the conference content includes: conference audio and conference video; a conference theme structure is constructed according to the conference content; the conference theme structure includes: at least one theme; an interactive question corresponding to any theme is generated according to the conference content corresponding to any theme; and the interactive question is sent to the conference terminal corresponding to the participating user corresponding to any theme;

[0065] The conference terminal receives the interactive questions sent by the server; plays the interactive questions; and receives reply information of the participating users to the interactive questions.

[0066] The embodiment of the present application further discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the method described in the embodiment of the present application.

[0067] The embodiment of the present application further discloses a machine-readable medium on which executable code is stored. When the executable code is executed, a processor executes the method described in the embodiment of the present application.

[0068] The embodiments of the present application include the following advantages:

[0069] In the technical solution of the embodiment of the present application, conference audio and conference video are obtained as conference content. Among them, the audio records the speech content of the participating users. The video supplements non-verbal information, such as the expressions, body movements and displayed materials of the participating users. This multi-dimensional information acquisition method can provide a basis for the construction of the conference theme structure and the generation of interactive questions. On this basis, the interactive questions generated based on multi-dimensional information can more effectively guide the participating users to supplement the key information for the conference summary, thereby improving the richness of the conference summary.

[0070] Secondly, because the conference summary is usually a concise presentation of the conference content, some information will inevitably be omitted in the process of generating the conference summary, and the conference content covered by a specific topic is often more detailed and comprehensive than the corresponding topic part in the conference summary. Based on this, the embodiment of the present application constructs a conference theme structure based on the acquired conference content, generates interactive questions based on the conference content corresponding to any topic, and sends these interactive questions to the participating users who participate in the discussion of the topic. The present application generates interactive questions based on the conference content corresponding to a specific topic, which can deeply explore the information under a specific topic that is not covered by the conference summary. Sending interactive questions to participating users can guide participating users to provide missing information, which can effectively fill the gaps formed by the missing information in the conference summary, and thus can improve the completeness of the conference summary.

[0071] In addition, in addition to being used to supplement information included in the meeting content but not included in the meeting summary, the interactive questions in the embodiment of the present application can also be used to supplement information not included in the meeting content. In this way, it is possible to guide the participating users to supplement information that was not fully discussed or not noticed during the meeting. This can make the meeting summary more complete and accurate, and provide a more reliable basis for subsequent work and decision-making. For example, the embodiment of the present application can generate a corresponding interactive question B "About project progress: What do you think about the current progress of the project?" for the theme of "Project Progress Report". Interactive question B can guide the participating users to feedback their opinions on the project progress, improve the meeting summary, and provide a basis for decision-making and work.

[0072] Furthermore, in the case where there are semantic differences between the conference summary and the conference content, the interactive questions in the embodiments of the present application can also clarify the doubts in the conference summary. For example, the conference content includes: the statement A corresponding to M and N, but the conference summary contains the statement B corresponding to M and P. In this case, the interactive question can be: "Which of the statements A and B is more accurate?" Such interactive questions can guide the participating users to compare the conference summary with the conference content and think about the accuracy of the statements, so as to correct the doubts in the conference summary and thus improve the accuracy of the conference summary.

[0073] In summary, the embodiments of the present application guide users to complete and confirm key information of the meeting based on interactive problems, which can overcome the problems of information omission and inaccurate summary in the preliminary meeting summary to a certain extent, and thus can significantly improve the quality of the meeting summary. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a structural diagram of a conference processing system according to an embodiment of the present application;

[0075] Figure 2 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0076] Figure 3 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0077] Figure 4 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0078] Figure 5 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0079] Figure 6 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0080] Figure 7 This is a schematic diagram of the steps of a conference processing method according to an embodiment of the present application;

[0081] Figure 8 is a structural diagram of a conference processing device according to an embodiment of the present application;

[0082] Fig. 9 is a structural diagram of a conference processing device according to an embodiment of the present application;

[0083] Fig.10 It is a schematic diagram of the structure of a device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0084] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0085] The conference processing method of the embodiment of the present application can be applied to various video conferences. The video conference scenarios specifically include: internal enterprise meetings, remote work meetings, academic seminars, etc. It can be understood that the embodiment of the present application is not limited to specific video conferences.

[0086] A meeting summary is a brief summary of the meeting content, which can provide users with a quick way to understand the meeting content. In related technologies, the method of generating a meeting summary usually includes: first, using speech recognition technology to convert the speech in the meeting into meeting text; then using natural language processing technology to extract key information from the meeting text, and generating a meeting summary based on the key information.

[0087] However, since the current conference summary is mainly based on the key information extracted from the conference text, if the key information is extracted improperly, it is easy to cause the conference summary to have problems such as information omission.

[0088] In response to technical problems such as information omission in meeting summaries in related technologies, an embodiment of the present application provides a meeting processing method, which may specifically include the following steps: obtaining meeting content; the above-mentioned meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the above-mentioned meeting content; the above-mentioned meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme; and sending the above-mentioned interactive questions to participating users corresponding to any of the above-mentioned themes.

[0089] First, the embodiment of the present application obtains the conference audio and conference video as the conference content. Among them, the audio records the speech content of the participating users. The video supplements the non-verbal information, such as the expressions, body movements and displayed materials of the participating users. This multi-dimensional information acquisition method can provide a basis for the construction of the conference theme structure and the generation of interactive questions. On this basis, the interactive questions generated based on multi-dimensional information can more effectively guide the participating users to supplement the key information for the conference summary, thereby improving the richness of the conference summary.

[0090] Secondly, because the conference summary is usually a concise presentation of the conference content, some information will inevitably be omitted in the process of generating the conference summary, and the conference content covered by a specific topic is often more detailed and comprehensive than the corresponding topic part in the conference summary. Based on this, the embodiment of the present application constructs a conference theme structure based on the acquired conference content, generates interactive questions based on the conference content corresponding to any topic, and sends these interactive questions to the participating users who participate in the discussion of the topic. The present application generates interactive questions based on the conference content corresponding to a specific topic, which can deeply explore the information under a specific topic that is not covered by the conference summary. Sending interactive questions to participating users can guide participating users to provide missing information, which can effectively fill the gaps formed by the missing information in the conference summary, and thus can improve the completeness of the conference summary.

[0091] In one example 1 of the present application, at a project progress meeting, topics such as "project progress report", "budget adjustment" and "decision-making" were discussed. Among them, the topic of "project progress report" includes "completion ratio", "quality of completed parts" and "next week's plan" and other aspects.

[0092] The following is an example of a meeting summary that is initially generated in an embodiment of the present application, wherein the content of the meeting summary is shown in {}. The initially generated meeting summary specifically includes fields such as title, date, participants, main topics of discussion, and decision. In the initially generated meeting summary, for the topic of "project progress report", only "completion ratio" and "next week's plan" are mentioned, but "quality of completed parts" is omitted.

[0093] {

[0094] Title: Meeting Summary Report

[0095] Date: YYYY-MM-DD

[0096] Participants: Zhang San, Li Si, Wang Wu

[0097] Main topics of discussion:

[0098] Topic 1: Project Progress Report

[0099] Key Info: As of this week, the project is 50% complete.

[0100] Action Item: The remaining 50% of the work needs to be completed next week.

[0101] Topic 2: Budget Adjustment

[0102] Key message: The budget exceeded expectations by 10% and needs to be re-evaluated.

[0103] Action Item: Submit new budget plan next week.

[0104] decision making:

[0105] Decision 1: Approve a one-month extension of the project.

[0106] Decision 2: Increase the project budget by 10%.

[0107] }

[0108] The embodiment of the present application constructs a conference theme structure based on the acquired conference content, generates interactive questions based on the conference content corresponding to any topic, and sends these interactive questions to the participating users who participate in the discussion of the topic. Assume that the embodiment of the present application generates interactive question A related to "quality of the completed part" based on the conference content corresponding to the topic of "project progress report": "Does the 50% completed project content meet the quality standards?" and sends interactive question A to participating user A.

[0109] Assume that the reply information A of participating user A to interactive problem A is "the completed part basically meets the quality standards, but there are some minor problems in module A. Personnel have been arranged to repair it, and it is expected that it will not affect the overall progress." Then the embodiment of the present application can improve the preliminary meeting summary based on the reply information A. Specifically, the target position of the reply information A in the meeting summary can be determined, and the target content corresponding to the reply information A can be added to the target position. Among them, the target content can be the same as the reply information A, or the reply information A can be further processed to obtain the target content.

[0110] Since the reply information A is about the content of "Partial Quality Completed" under the theme of "Project Progress Report", the target position can be found in the field of "Project Progress Report" in the meeting summary. Since the "Project Progress Report" field includes: "Key Information" subfield and "Action Item" subfield, and the correlation between "Partial Quality Completed" and "Key Information" subfield is greater than the correlation between "Partial Quality Completed" and "Action Item" subfield, the target position can be located in the "Action Item" subfield. For example, the target position can be the position after: "As of this week, the project has been completed 50%."

[0111] Given that the reply information A is about the content related to "quality of the completed part" in the theme of "project progress report", the target position can be determined in the theme of "project progress report" in the meeting summary. The theme of "project progress report" includes the "key information" subfield and the "action item" subfield. After analysis, it can be seen that the information of "quality of the completed part" is more closely related to the "key information" subfield, so the target position can be determined to be after the key information statement "As of this week, the project has been completed 50%." In this way, the coherence and logic of the information can be improved, so that users can know the quality status of the completed part immediately after understanding the completion of the project progress, providing more sufficient background information for the subsequent understanding of the content of the action item.

[0112] It should be noted that, in addition to being used to supplement information included in the meeting content but not included in the meeting summary, the interactive questions in the embodiments of the present application can also be used to supplement information not included in the meeting content. In this way, it is possible to guide the participating users to supplement information that was not fully discussed or not noticed during the meeting. This can make the meeting summary more complete and accurate, and provide a more reliable basis for subsequent work and decision-making. For example, the embodiment of the present application can generate a corresponding interactive question B "About project progress: What do you think about the current progress of the project?" for the theme of "Project Progress Report". Interactive question B can guide the participating users to feedback their opinions on the project progress, improve the meeting summary, and provide a basis for decision-making and work.

[0113] Furthermore, in the case where there are semantic differences between the conference summary and the conference content, the interactive questions in the embodiments of the present application can also clarify the doubts in the conference summary. For example, the conference content includes: the statement A corresponding to M and N, but the conference summary contains the statement B corresponding to M and P. In this case, the interactive question can be: "Which of the statements A and B is more accurate?" Such interactive questions can guide the participating users to compare the conference summary with the conference content and think about the accuracy of the statements, so as to correct the doubts in the conference summary and thus improve the accuracy of the conference summary.

[0114] In summary, the embodiments of the present application guide users to complete and confirm key information of the meeting based on interactive problems, which can overcome the problems of information omission and inaccurate summary in the preliminary meeting summary to a certain extent, and thus can significantly improve the quality of the meeting summary.

[0115] As the core control device of the conference site, the conference terminal 101 integrates multiple components such as high-definition camera, microphone array, touch screen, etc. It can not only collect conference video and conference audio, but also display conference summary.

[0116] The server 102 can be deployed in the cloud and has powerful functions. It undertakes the important task of generating a meeting summary and can also execute the meeting processing method in the embodiment of the present application to generate interactive questions. The interactive questions can guide users to improve and confirm the key information of the meeting, thereby effectively improving the quality of the meeting summary.

[0117] The user terminal 103 specifically includes mobile devices such as smart phones and tablet computers. The conference users receive the conference summary with the help of the user terminal 103 and can send feedback information on the conference summary to the server 102, thus achieving good interaction between the conference users and the conference processing system.

[0118] In terms of communication, HTTP (HyperText Transfer Protocol) / HTTPS (HyperText Transfer Protocol Secure) can be used for communication between the conference terminal 101 and the server 102. Among them, HTTPS uses encryption protocol to transmit conference audio, video, interactive questions, conference summaries and other conference-related data, which can effectively prevent data leakage or tampering, and improve the security of conference-related data during transmission.

[0119] The WebSocket protocol is used for real-time communication between the user terminal 103 and the server 102. The real-time communication technology of WebSocket enables the instant transmission of conference-related data such as conference summaries or feedback information, greatly reducing transmission delays. It should be noted that HTTP / HTTPS and WebSocket here are only examples of communication protocols. In fact, the embodiments of the present application do not limit the specific communication protocols.

[0120] Any of the conference terminal 101, the server 102 and the user terminal 103 can achieve network connection through Wi-Fi (Wireless Fidelity) or a cellular network, which facilitates the wide application of the conference processing system and enables it to adapt to different network environments.

[0121] In terms of specific implementation details, the conference terminal 101 supports touch interaction and multi-person collaboration, and has excellent functions such as high-definition video recording and multi-microphone array. The client program installed on the conference terminal 101 is responsible for user interface display, data collection, etc., providing participants with a convenient operating experience and efficient data collection functions.

[0122] In addition, the conference terminal 101 also supports voice interaction. Specifically, it can receive and play the interactive questions sent by the server 102. If the user replies to the interactive questions in voice form, the conference terminal 101 can also collect the reply information in voice form, further enriching the interactive mode.

[0123] The server 102 integrates a high-performance server cluster and is equipped with a GPU (Graphics Processing Unit) acceleration unit, which can support large-scale concurrent processing and has powerful computing and processing capabilities. Processing models are running on the server 102, including a first processing model for generating meeting summaries and a second processing model for determining emotional states. These processing models are diverse, covering but not limited to speech recognition models, large language models, emotion classification models, etc., which together provide rich functional support for meeting processing.

[0124] At the same time, a database is also running on the server 102. The database stores conference-related data such as conference video, conference audio, conference summary, and interactive questions for conference identification, providing a complete solution for the storage and management of conference-related data.

[0125] Method Example 1

[0126] refer to Figure 2 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0127] Step 201, obtaining the meeting content; the above-mentioned meeting content specifically includes: meeting audio and meeting video;

[0128] Step 202: construct a conference theme structure according to the conference content; the conference theme structure specifically includes: at least one theme;

[0129] Step 203: Generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0130] Step 204: Send the above interactive questions to the participating users corresponding to any of the above topics.

[0131] Figure 2 The method embodiment 1 shown can be executed by a conference terminal or a server. It can be understood that the embodiment of the present application is for Figure 2 The specific execution subject of the method embodiment 1 is not limited.

[0132] In step 201, the conference terminal may collect conference content and send the conference content to the server, and the server may receive the conference content sent by the conference terminal.

[0133] In a specific implementation, the conference terminal can use a microphone array to collect conference audio in real time. The collected conference audio can be transmitted to the server via Wi-Fi or cellular network.

[0134] The conference terminal can use a high-definition camera to collect conference videos in real time, and the video screen of the conference video can include the facial expressions and body movements of the participants. The conference video can be transmitted to the server through the network channel.

[0135] In step 202, the server may use NLP (Natural Language Processing) technology to construct a conference theme structure corresponding to the conference content.

[0136] The embodiment of the present application utilizes NLP technology, and the specific process of constructing a conference theme structure corresponding to the conference content is not limited.

[0137] In one example, the process of constructing a conference topic structure corresponding to the conference content using NLP technology specifically includes the following steps:

[0138] Step A1: Determine the conference text corresponding to the conference content; the conference text may include: audio text and video text.

[0139] Among them, ASR (Automatic Speech Recognition) technology can be used to convert the conference audio into audio text in real time. ASR technology is a technical means that can convert human voice signals into readable text form. The audio text obtained by ASR can be saved in a database for subsequent processing.

[0140] Video text may include: subtitles in conference videos, text in presentation content, etc.

[0141] The video text may also include: an emotional state text obtained by performing emotion recognition on the conference video.

[0142] Step A2: Determine at least one theme of the meeting based on the meeting questions;

[0143] In practical applications, text classification algorithms (such as classifiers based on machine learning), topic word extraction technology (statisticing high-frequency words and related field words in the text to determine the topic category) and other technologies can be used to determine at least one topic of the meeting.

[0144] Step A3: Divide the topics according to the conference contents corresponding to each topic to obtain subtopics contained in the topic.

[0145] For example, in Example 1, the topic "Project Progress Report" can be divided into sub-topics such as "Key Information" and "Action Items".

[0146] In practical applications, text clustering algorithms (further clustering topic-related texts into sub-topic-related texts) and semantic analysis algorithms (determining more detailed content divisions by understanding text semantics) can be used to divide topics.

[0147] Step A4: Construct a conference theme structure corresponding to the conference content according to the hierarchical relationship between the theme and the sub-theme.

[0148] For example, a hierarchical modeling tool (which may be a simple tree structure modeling software, or a tree structure may be used to represent the data structure) may be used to construct a conference theme structure corresponding to the conference content.

[0149] In another example, a hierarchical clustering model or a large language model can be used to construct a conference topic structure corresponding to the conference content. Taking the large language model as an example, the conference text sample and the conference topic structure sample can be used as the first training sample, and the large language model can be trained using the first training sample so that the large language model has the ability to generate the conference topic structure. Since the large language model learns the language pattern and semantic association during the training process of the first training sample, it can identify the topic clues in the conference text and sort out the structure of the topic and sub-topic. Therefore, the large language model has the ability to generate the conference topic structure.

[0150] The large language model of the embodiment of the present application can be an artificial intelligence model designed to understand and generate human language. The large language model can perform a wide range of tasks, including the graphic description language file generation task of the embodiment of the present invention, etc. by training on a large amount of text data. The large language model is usually based on a deep learning architecture, such as a transformer (Transformer) structure. The embodiment of the present application does not limit the specific type of the large language model. For example, the type of the large language model may include: GPT (Generative Pre-Trained Transformer) model and GLM (Generative Language Model, General Language Model), etc.

[0151] For example, the GPT model is a large-scale natural language generation model based on the Transformer structure. It uses massive amounts of text data for pre-training and can generate high-quality natural language text, including articles, conversations, summaries, etc. The core of the GPT model is the transformer structure, which uses an attention mechanism to enable the GPT model to consider all positions in the input sequence at the same time, thereby better capturing long-distance dependencies. After pre-training the GPT model, fine-tuning it can adapt to various downstream text generation tasks.

[0152] The embodiment of the present application can perform fine-tuning training on the large language model so that the large language model can generate a conference topic structure based on the conference text.

[0153] In step 203, interactive questions corresponding to any topic are generated according to the conference content corresponding to any topic, which can guide the user to complete and confirm the key information of the meeting.

[0154] The embodiment of the present application can provide the following technical solution for generating interactive questions corresponding to any topic:

[0155] Technical Solution 1

[0156] In technical solution 1, the process of generating interactive questions corresponding to any topic specifically includes: obtaining keywords from the conference content corresponding to any topic; and generating interactive questions corresponding to the keywords using question templates.

[0157] The function of the question template is to provide a standardized sentence structure, using keywords as variables to generate interactive questions with clear logic and clear directions. Sentences such as "What do you think of [keyword]?", "Which is more accurate, [keyword A] or [keyword B]?", "Do they both meet the quality standards?", "What are the tasks to be done for [keyword]?", "Or "What specific actions need to be included in the to-do list around [keyword]?", etc., help guide users to improve and confirm key meeting information around keywords.

[0158] In the embodiments of the present application, the keyword can take various forms, and can be a single word, a phrase consisting of multiple characters, or even a combination of multiple words.

[0159] In the embodiment of the present application, the method for obtaining keywords from the conference content corresponding to any topic may specifically include:

[0160] Acquisition method 1: extracting a first keyword that meets a first preset condition from the conference content corresponding to any topic; the first preset condition includes: a keyword that is related to any topic and appears in the conference content of any topic but does not appear in the conference summary; or

[0161] Acquisition method 2: extracting a second keyword that meets the fuzzy condition from the conference content corresponding to any topic; or

[0162] Acquisition method 3: extracting a third keyword that meets a second preset condition from the conference content corresponding to any topic; the second preset condition includes: there is a semantic difference between the conference content and the conference summary of any topic.

[0163] Among them, the first keyword of acquisition method 1 is used to obtain some key information in the meeting content that is closely related to the topic and appears in the meeting content but is not presented in the meeting summary. For example, in a meeting discussing new product development, a specific new technology application may be mentioned, but the new technology application is not mentioned in the meeting summary.

[0164] In this case, interactive questions can be generated based on the first keyword, such as "What do you think about [first keyword]?" This can guide participants to further explore the missing key information, thereby improving the meeting summary.

[0165] The second keyword of acquisition method 2 is specifically the ambiguous words in the meeting content. The types of ambiguous words include: unclear time (such as "soon", "recently"), uncertain quantity (such as "several"), unclear concept (such as "new strategy" but not specifying what is new), etc.

[0166] When generating interactive questions for the second keyword, you can generate “When was ‘[second keyword]’ mentioned in the meeting?” for a second keyword with an unclear time range; or “How many times was [second keyword] mentioned in the meeting?” for a second keyword with an uncertain quantity. In this way, you can guide meeting participants to clarify ambiguous information and avoid problems in subsequent actions due to misunderstandings.

[0167] Acquisition method 3: Applicable to situations where there are semantic differences between the meeting content and the meeting summary. For example, the meeting content includes: A corresponding to M and N, but the meeting summary contains B corresponding to M and P. In this case, the interactive question can be: "Which is more accurate, [third keyword A] or [third keyword B]?" Such interactive questions can guide the participants to compare the meeting summary with the meeting content, find out the reasons for the differences and clarify the correct statement.

[0168] It can be understood that the above acquisition methods 1 to 3 are only optional embodiments of the present application. In fact, those skilled in the art can use other acquisition methods to obtain keywords from the conference content corresponding to any topic according to actual application requirements. For example, when certain words appear frequently in the conference content, they can be extracted as keywords.

[0169] In the embodiment of the present application, the process of determining the question template specifically includes:

[0170] Step B1, collecting a first data sample; the first data sample includes: a meeting content sample corresponding to the theme of the meeting, and a meeting summary sample of the meeting;

[0171] Step B2: determining the type of question and key information based on the meeting content sample and the meeting summary sample;

[0172] Step B3: Determine the question template based on the question type and key information.

[0173] For step B1, you can collect data including meeting content samples and meeting summary samples corresponding to the meeting theme by reviewing past meeting records, investigating meeting materials in related fields, etc. The rich first data samples can provide a reference for the subsequent determination of the question template.

[0174] In step B2, the collected meeting content samples and meeting summary samples can be analyzed to find out the types of questions that appear repeatedly and the key information that plays a key role in meeting decision-making. By determining the types of questions and key information, the focus of common concerns and areas that are prone to doubts in different meetings can be identified, laying the foundation for the formulation of question templates.

[0175] In step B3, a universal question sentence pattern and structure may be designed based on the determined question type and key information to form a question template.

[0176] Referring to Table 1, examples of question types and question templates in an embodiment of the present application are shown.

[0177] Table 1

[0178]

[0179] After designing the first question template, we used it together with the topic and summary of the conference content as the second data sample. These data samples provide rich information and specific question scenarios for the training of the large language model.

[0180] The first question template provides examples of question sentences and structures for the large language model, allowing the model to learn how to construct effective questions. The theme of the conference content helps the model understand the specific areas and core concerns involved in different conferences, making the generated question templates more targeted. The conference summary provides the model with a summary of the main content of the conference, allowing the model to grasp the key information and problem points of the conference in a shorter space.

[0181] By training the large language model with the second data sample, the model gradually acquires the ability to generate question templates. When new meeting content and meeting summaries are input, the large language model can output corresponding question templates for each topic. These question templates can guide participants to discuss key issues in the meeting in depth, promote the formulation and implementation of meeting decisions, and also help to improve the meeting summary, making the meeting record more accurate and comprehensive.

[0182] For example, for a meeting about project progress, the topics of the meeting may include task allocation, time nodes, resource requirements, etc. After training, the large language model can output question templates corresponding to various topics such as "Who is the specific person in charge of Task A?" "Is it possible to adjust the key time nodes of the project?" "Can the current resource requirements be met?" based on the content and summary of the meeting.

[0183] The process of generating the interactive question corresponding to the keyword by using the question template may include: filling the keyword into the corresponding position in the question template, and the filled question template may be used as the interactive question.

[0184] Technical Solution 2

[0185] In Technical Solution 2, the process of generating interactive questions corresponding to any topic based on the meeting content corresponding to any topic specifically includes: determining the emotional state of the participating users based on the meeting content corresponding to any topic; and generating interactive questions corresponding to any topic based on the emotional state.

[0186] If the system detects that a participant is "confused" when discussing a topic or a keyword of a topic, it can generate an interactive question: "Do you have any other questions about this topic or the keyword of the topic?" For another example, if it is detected that a participant is "dissatisfied" when discussing a topic, it can generate a question: "What opinions or suggestions do you have on this topic?"

[0187] "Confused emotion" and "dissatisfied emotion" may be examples of emotional states. In fact, the embodiments of the present application do not limit specific emotional states.

[0188] For example, the emotional state may also include: "excited emotion", and the corresponding interactive question may be "What direction of development do you most expect for this topic?". For another example, the emotional state may also include: "anxious emotion", and the corresponding interactive question may be "What do you think is the biggest challenge for this topic?". For another example, the emotional state may also include: "positive emotion", and the corresponding interactive question may be "What positive ideas or suggestions do you have for this topic?".

[0189] The embodiment of the present application can use SA (Sentiment Analysis) technology to determine the emotional state of the participating users. SA is a natural language processing technology used to identify and extract emotional colors in text.

[0190] The embodiments of the present application do not limit the specific process of determining the emotional state of the participating users. In an optional implementation, the process of determining the emotional state of the participating users according to the conference content corresponding to any topic may specifically include: determining the first emotional state of the participating users according to the facial features of the participating users in the conference video corresponding to any topic; determining the second emotional state of the participating users according to the body features of the participating users in the conference video corresponding to any topic; determining the third emotional state of the participating users according to the voice features of the participating users in the conference audio corresponding to any topic; and fusing the first emotional state, the second emotional state, and the third emotional state to obtain the emotional state of the participating users.

[0191] Single facial features, body features or voice features may be misjudged or inaccurate. For example, facial expressions may be misinterpreted due to factors such as lighting and angle; body movements may be vague or ambiguous; voice features may also be disturbed by noise, etc. However, the emotional states corresponding to facial features, body features and voice features are integrated to form a comprehensive emotional state; they can verify and supplement each other, reduce the errors caused by a single factor, and thus greatly improve the accuracy of the emotional state.

[0192] The process of determining the first emotional state of the conference user according to the facial features of the conference user in the conference video corresponding to any topic specifically includes:

[0193] Step C1, preprocessing: preprocess the conference video to remove noise and interference and improve the image quality of the video frame.

[0194] Step C2, face detection: Use a face detection algorithm to detect the face area in the video frame.

[0195] Step C3, key point positioning: Use the facial key point detection algorithm to locate the key parts of the face, such as eyes, mouth, eyebrows, etc.

[0196] Step C4, feature extraction: extract facial features such as the degree of eye opening and closing, curvature of the mouth corners, etc. according to the changes in the positions of key points.

[0197] Step C5, emotion classification: the extracted features are input into a trained CNN (Convolutional Neural Network) model to perform emotion classification to obtain a first emotion state.

[0198] The process of determining the second emotional state of the conference user according to the body features of the conference user in the conference video corresponding to any topic specifically includes:

[0199] Step D1, preprocessing: preprocess the conference video to remove noise and interference and improve the image quality of the video frame.

[0200] Step D2, human posture estimation: Use a human posture estimation algorithm to detect key points of the human body in the video, such as joint positions.

[0201] Step D3: extract limb features from the change trajectory of key points of the human body, such as the swing amplitude of the arm, the tilt angle of the head, etc.

[0202] Step D4, action recognition: The extracted features are input into the first LSTM (Long-Short Term Memory) model that has completed training, and the first LSTM model outputs the classification results of body movements, such as nodding, shaking head, gestures, etc.

[0203] The process of determining the third emotional state of the conference user according to the voice features of the conference user in the conference audio corresponding to any topic may specifically include the following steps:

[0204] Step E1, preprocessing: preprocess the conference audio to remove background noise and improve audio quality.

[0205] Step E2, feature extraction: extracting speech features from the conference audio, such as fundamental frequency (pitch), energy (volume), zero-crossing rate (speech speed), etc.

[0206] Step E3, emotion classification: the extracted features are input into the trained second LSTM model, and the second LSTM model outputs the third emotional state. Examples of the third emotional state may include: calm, excited, anxious, etc.

[0207] The process of fusing the first emotional state, the second emotional state and the third emotional state specifically includes the following steps:

[0208] Step F1, weight allocation: according to experimental results and expert experience, different weights are allocated to the first emotional state, the second emotional state and the third emotional state.

[0209] Step F2, comprehensive evaluation: weighted processing is performed on the first emotional state, the second emotional state and the third emotional state according to their respective corresponding weights, and the obtained weighted processing result can be used as the final emotional state.

[0210] The above technical solutions 1 and 2 have specifically introduced the process of generating interactive questions corresponding to any topic. It can be understood that technical personnel in this field can adopt any one or a combination of technical solutions 1 to 2 according to actual application requirements, or can also adopt other technical solutions for generating interactive questions corresponding to any topic.

[0211] For example, in one implementation, interactive questions can be generated based on the roles of the participating users. Determining the roles of the participating users helps to better understand their concerns and needs in the meeting, so as to raise more targeted questions and promote in-depth discussion and decision-making in the meeting.

[0212] The role types of the participating users include: leaders, technical experts, ordinary employees, etc. The user's role can be judged by the personnel introduction before the meeting, position information or the characteristics of the speech during the meeting.

[0213] For leaders, you can generate an interactive question: "From a management decision-making perspective, what important impact does this topic have on the overall strategy?" Leaders are usually concerned with macro-level decision-making and strategic direction, and this interactive question can guide them to think about the relationship between the topic and the overall goals of the organization.

[0214] For technical experts, an interactive question can be generated: "From a professional and technical perspective, what are the technical difficulties and solutions for [topic keywords]?" Technical experts have in-depth professional knowledge and are able to conduct in-depth analysis of the technical aspects of the topic and provide solutions.

[0215] For ordinary employees, you can generate interactive questions: "In actual work, what specific changes has [topic keywords] brought to your work process?" Ordinary employees are more concerned about the actual impact of the topic on their daily work. This question can prompt them to share specific work experiences and changes, and provide practical operational feedback for the meeting.

[0216] For example, in another implementation, a gradually in-depth questioning method can be adopted, based on multiple rounds of interactive questions, to guide the participating users to think deeply about the topic and provide more detailed and valuable answers, so as to better improve the meeting summary.

[0217] Start by asking a broad first-round interactive question to a single user, such as: "What do you think are the key points about this topic?" The first round of interactive questions allows users to think about the topic as a whole from the beginning and put forward the aspects they think are most important.

[0218] Generate a second round of interactive questions based on the user’s answer. For example, if the user answers “The implementation progress of [theme keyword] is critical,” you can follow up with the question: “What is the current specific situation of the implementation progress of [theme keyword] you mentioned? What factors may affect the progress?” The second round of interactive questions can guide users to explore the key points they raised in depth and provide more details and analysis.

[0219] In step 204, the above-mentioned interactive questions are sent to the participating users corresponding to any of the above-mentioned topics. The participating users corresponding to any of the above-mentioned topics may specifically be: target participating users who speak on any of the topics. Since there may be one or more participating users who speak on any of the topics, the target participating users may also be one or more. Of course, if the interactive questions correspond to specific keywords in the conference video, the target participating users who speak on the specific keywords may be determined based on the facial features of the participating users who are speaking in the conference video clip containing the specific keywords.

[0220] Since the conference video contains the facial features of the participating users, the conference video received by the server also contains: the identification of the conference terminal; in this way, the embodiment of the present application can associate the identification of the participating user with the identification of the conference terminal. On this basis, the interactive questions can be sent to the target conference terminal corresponding to the target participating user, thereby sending the above interactive questions to the participating users corresponding to any of the above topics. For example, user A uses conference terminal A, and user B uses conference terminal B. In the case where it is necessary to send interactive questions to user A, the interactive questions can be sent to conference terminal A.

[0221] Of course, the participating user may also correspond to information such as a user account, so the embodiment of the present application may send the interaction question corresponding to the target participating user to the user terminal or conference terminal logged in with the user account. For example, an interaction question X is generated for a specific keyword X in a conference video. Based on the analysis of facial features in the conference video, it is found that keyword X is spoken by user C, so the user terminal or conference terminal logged in with the user account of user C may be sent the interaction question X.

[0222] In an optional implementation of the present application, the method may further include: receiving reply information from the participating user to the interactive question; and improving and revising the meeting summary according to the reply information. Among them, improving the meeting summary can supplement key information to make the meeting focus clearer; revising the meeting summary can correct errors or inaccurate content contained in the meeting summary.

[0223] The conference summary for improving and revising the conference summary may be a preliminary conference summary. The preliminary conference summary may be a conference summary generated by a large language model according to the conference content.

[0224] The embodiment of the present application can fine-tune the large language model to enable it to generate conference summaries. Specifically, a large number of paired conference content samples and their corresponding conference summary samples can be used as second training samples. By learning the second training samples, the large language model can understand the various features and patterns of conference content, as well as how to extract key information from rich conference content and form conference summaries in a concise and clear manner.

[0225] For conference content samples, the large language model can learn the topic distribution, discussion methods, language styles, etc. of different types of conferences. This helps the large language model quickly identify the type and characteristics of new conference content, thereby better understanding the topic structure of the conference.

[0226] The meeting summary samples provide specific target output examples for the large language model. By analyzing these meeting summary samples, the large language model learns how to accurately extract key information from the meeting, such as topics, important decisions, etc., and presents it in a language style and logical organization that meets the requirements of the meeting summary.

[0227] In one example 1 of the present application, at a project progress meeting, topics such as "project progress report", "budget adjustment" and "decision-making" were discussed. Among them, the topic of "project progress report" includes content such as "completion ratio", "quality of completed part" and "next week's plan". Assume that the embodiment of the present application generates an interactive question A related to "quality of completed part" based on the meeting content corresponding to the topic of "project progress report": "Does the 50% of project content that has been completed meet the quality standards?" and sends interactive question A to participating user A. Assume that the reply information A of participating user A to interactive question A is "The completed part basically meets the quality standards, but there are some minor problems in module A. Personnel have been arranged to repair it, and it is expected that it will not affect the overall progress." Then the embodiment of the present application can improve the preliminary meeting summary based on the reply information A. Specifically, the target position of the reply information A in the meeting summary can be determined, and the target content corresponding to the reply information A can be added to the target position. Among them, the target content can be the same as the reply information A, or the reply information A can be further processed to obtain the target content.

[0228] In an example 2 of the present application, it is assumed that the meeting content includes: statement A corresponding to M and N, but the meeting summary includes statement B corresponding to M and P. In this case, the interactive question may be: "Which of statement A and statement B is more accurate?"

[0229] If the participant's reply information believes that statement A is accurate, the target position of the reply information in the meeting summary can be determined, that is, the part involving statement B, and statement B at the target position can be replaced with statement A or the processed content of statement A.

[0230] If the participant's reply information indicates that statement B is accurate, a second round of interactive questions can be further generated: "Why is statement B accurate, and what are the differences between statement B and statement A?" In this case, the rationality of statement B can be supplemented in the meeting summary based on the reply information of the second round of interactive questions, making the meeting summary clearer and more convincing.

[0231] The embodiment of the present application can provide conference summaries in various formats for users to choose from. The various formats of conference summaries specifically include: text format, chart format, video format, etc.

[0232] Different users may prefer different ways of presenting information. Providing a variety of formats can meet the needs of more users. Presenting information in the form of charts and videos can increase the intuitiveness and understandability of information, which helps to improve the effectiveness of information dissemination. Users can choose the most suitable format according to their preferences, which improves the personalization of the conference processing system.

[0233] In one example, the text-formatted meeting summary specifically includes fields such as title, date, participants, main discussion topics, and decisions.

[0234] Then, the embodiment of the present application can generate a meeting summary in a chart format based on the meeting summary in text format. For example, data of a field or a subfield under a field can be automatically extracted from the meeting summary in text format, and a bar chart, pie chart, or other chart can be generated based on the extraction result. For example, the embodiment of the present application can generate a bar chart of the project progress to show the completion status of each stage.

[0235] The embodiment of the present application can generate a video-formatted conference summary based on a text-formatted conference summary. For example, the embodiment of the present application can combine the text-formatted conference summary with the conference video to generate an annotated video summary. For example, the system can highlight the topic part of the conference summary in the conference video and mark the first prompt information corresponding to the change of emotional state on the timeline.

[0236] In summary, the conference processing method of the embodiment of the present application obtains conference audio and conference video as conference content. Among them, the audio records the speech content of the participating users. The video supplements non-verbal information, such as the expressions, body movements, and displayed materials of the participating users. This multi-dimensional information acquisition method can provide a basis for the construction of the conference theme structure and the generation of interactive questions. On this basis, the interactive questions generated based on multi-dimensional information can more effectively guide the participating users to supplement the key information used for the conference summary, thereby improving the richness of the conference summary.

[0237] Secondly, because the conference summary is usually a concise presentation of the conference content, some information will inevitably be omitted in the process of generating the conference summary, and the conference content covered by a specific topic is often more detailed and comprehensive than the corresponding topic part in the conference summary. Based on this, the embodiment of the present application constructs a conference theme structure based on the acquired conference content, generates interactive questions based on the conference content corresponding to any topic, and sends these interactive questions to the participating users who participate in the discussion of the topic. The present application generates interactive questions based on the conference content corresponding to a specific topic, which can deeply explore the information under a specific topic that is not covered by the conference summary. Sending interactive questions to participating users can guide participating users to provide missing information, which can effectively fill the gaps formed by the missing information in the conference summary, and thus can improve the completeness of the conference summary.

[0238] In addition, in addition to being used to supplement information included in the meeting content but not included in the meeting summary, the interactive questions in the embodiment of the present application can also be used to supplement information not included in the meeting content. In this way, it is possible to guide the participating users to supplement information that was not fully discussed or not noticed during the meeting. This can make the meeting summary more complete and accurate, and provide a more reliable basis for subsequent work and decision-making. For example, the embodiment of the present application can generate a corresponding interactive question B "About project progress: What do you think about the current progress of the project?" for the theme of "Project Progress Report". Interactive question B can guide the participating users to feedback their opinions on the project progress, improve the meeting summary, and provide a basis for decision-making and work.

[0239] Furthermore, in the case where there are semantic differences between the conference summary and the conference content, the interactive questions in the embodiments of the present application can also clarify the doubts in the conference summary. For example, the conference content includes: the statement A corresponding to M and N, but the conference summary contains the statement B corresponding to M and P. In this case, the interactive question can be: "Which of the statements A and B is more accurate?" Such interactive questions can guide the participating users to compare the conference summary with the conference content and think about the accuracy of the statements, so as to correct the doubts in the conference summary and thus improve the accuracy of the conference summary.

[0240] In summary, the embodiments of the present application guide users to complete and confirm key information of the meeting based on interactive problems, which can overcome the problems of information omission and inaccurate summary in the preliminary meeting summary to a certain extent, and thus can significantly improve the quality of the meeting summary.

[0241] Method Example 2

[0242] refer to Figure 3 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0243] Step 301: Obtain the meeting content; the above-mentioned meeting content specifically includes: meeting audio and meeting video;

[0244] Step 302: construct a conference theme structure according to the conference content; the conference theme structure specifically includes: at least one theme;

[0245] Step 303: Generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0246] Step 304: Send the above interactive questions to the conference participants corresponding to any of the above topics;

[0247] Step 305: Determine the emotional state of the participating users according to the conference content corresponding to any topic;

[0248] Step 306: Display the first prompt information corresponding to the change in the emotional state of the participating user on the timeline of the conference content.

[0249] Method Example 2 adds step 305 and step 306 on the basis of method example 1. In terms of the analysis of the meeting content, traditional meeting records often only focus on the meeting audio and video content, as well as the meeting summary. Steps 305 and 306 display the first prompt information corresponding to the change in the emotional state of the participating users on the timeline, which can add a new dimension to the meeting records. This allows the participants to intuitively feel the emotional reactions of the participating users at different time points when reviewing the meeting later, providing more basis for evaluating the effectiveness of the meeting.

[0250] It should be noted that the timeline may specifically be: the playback progress timeline of the conference video and conference audio. In the embodiment of the present application, the conference video and conference audio may be time synchronized first, and then the timeline corresponding to the conference video and conference audio after time synchronization may be determined.

[0251] Time synchronization refers to matching the timestamps of the conference video and audio with the text data obtained through speech recognition in the time dimension. The purpose of time synchronization is to accurately match the non-verbal information such as the emotional state and body movements of the participants with the conference video and audio, so as to better analyze and understand the conference situation.

[0252] In one example, assuming that a participant has a nervous expression when speaking, the embodiment of the present application can mark this speech on the timeline as "potentially controversial" or "needs further discussion."

[0253] Emotional state change specifically refers to the situation where the participant's emotions change from one emotional state to another during the meeting, such as from calm to nervous, from excited to frustrated, etc. In the above example, the nervous expression of the participant when speaking is a change in emotional state.

[0254] There can be many specific examples of the first prompt information. For example, you can directly use text to mark "may be controversial" or "need further discussion" on the timeline, or use specific icons (such as exclamation marks, question marks, etc.) to indicate that the emotional state corresponding to this speech deserves attention. Different colors can also be used to distinguish different changes in emotional states. For example, red indicates a tense and controversial emotional state, and yellow indicates an emotional state that requires attention and may have potential problems. In this way, when viewing the timeline, you can see at a glance which parts of the meeting content may have problems or need further discussion.

[0255] When the changed emotional state is a negative emotion, such as tension, anxiety, or anger, the first prompt message often indicates that the content of the meeting may be controversial or require further attention. For example, when the emotional state of the participant changes from calm to angry, the first prompt message may be "strong opposition" or "emotionally excited, there may be contradictions." This is because anger usually means that the participant has a strong resistance to or disagreement with the content of the meeting.

[0256] If the emotional state changes to positive emotions, such as excitement or satisfaction, the first prompt message may be "positive progress in reaching consensus" or "satisfied with the plan." For example, when discussing a new project plan, if the participant's expression changes from confusion to excitement, the first prompt message may be "strong interest in the plan, and new ideas may be generated."

[0257] Method Example 3

[0258] refer to Figure 4 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0259] Step 401: Obtain the meeting content; the above-mentioned meeting content specifically includes: meeting audio and meeting video;

[0260] Step 402: construct a conference theme structure according to the conference content; the conference theme structure specifically includes: at least one theme;

[0261] Step 403: Generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0262] Step 404: Send the above interactive questions to the conference participants corresponding to any of the above topics;

[0263] Step 405: Determine the emotional state of the participating users according to the conference content corresponding to any topic;

[0264] Step 406: Add the above emotional state to the meeting summary.

[0265] Method embodiment 3 adds step 405 and step 406 on the basis of method embodiment 1.

[0266] Among them, step 405 can deeply understand the psychological reaction of the participating users when discussing a specific topic by determining the emotional state of the participating users according to the conference content corresponding to any topic. For example, when discussing the topic of "budget adjustment", by analyzing the tone and speed of the conference audio and the expressions of the participating users in the conference video, it can be determined whether the participating users are anxious, dissatisfied, or understand and accept. This helps to determine the attitudes of the participating users towards various topics.

[0267] Step 406 adds emotional states to the meeting summary, which can provide richer information for meeting summary and subsequent analysis. For example, if the meeting summary only records the discussion content and decision results on the topic of "budget adjustment" without emotional information, the potential resistance of the participating users may be ignored. After adding emotional states, users who read the meeting summary can better understand the problems that may be encountered during the decision-making process, and users who use the meeting summary later can further evaluate the effectiveness of the meeting and the feasibility of the decision from the emotional dimension.

[0268] The following is an example of a meeting summary after adding emotional states. In addition to the fields of title, date, participants, main topics discussed, and decision, the meeting summary also includes the sentiment analysis field.

[0269] {

[0270] Title: Meeting Summary Report

[0271] Date: YYYY-MM-DD

[0272] Participants: Zhang San, Li Si, Wang Wu

[0273] Main topics of discussion:

[0274] Topic 1: Project Progress Report

[0275] Key Info: As of this week, the project is 50% complete.

[0276] Action Item: The remaining 50% of the work needs to be completed next week.

[0277] Topic 2: Budget Adjustment

[0278] Key message: The budget exceeded expectations by 10% and needs to be re-evaluated.

[0279] Action Item: Submit new budget plan next week.

[0280] decision making:

[0281] Decision 1: Approve a one-month extension of the project.

[0282] Decision 2: Increase the project budget by 10%.

[0283] Sentiment Analysis:

[0284] Zhang San: He appears nervous when discussing project progress and may need attention.

[0285] Li Si: He expressed dissatisfaction when discussing the budget adjustment and further communication is needed.

[0286] }

[0287] Method Example 4

[0288] refer to Figure 5 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0289] Step 501: Obtain the meeting content; the above-mentioned meeting content specifically includes: meeting audio and meeting video;

[0290] Step 502: construct a conference theme structure according to the conference content; the conference theme structure specifically includes: at least one theme;

[0291] Step 503: Generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0292] Step 504: Send the above interactive questions to the conference participants corresponding to any of the above topics;

[0293] Step 505: Send the conference summary to the participating users;

[0294] Step 506: receiving feedback information from conference participants regarding the conference summary;

[0295] Step 507: Optimize the processing model according to the feedback information; the processing model specifically includes: a first processing model for generating a meeting summary, or a second processing model for determining an emotional state.

[0296] Method embodiment 4 adds steps 505 to 507 based on method embodiment 1.

[0297] Among them, step 505 sends the meeting summary to the participating users, so that the participating users can review the key points of the meeting in time, helping them to better understand the overall context and key decisions of the meeting. During or after the meeting, the participating users may not remember some discussion points clearly for various reasons. Sending the meeting summary can play a role in reminding and strengthening memory. At the same time, this also provides a channel for the participating users to check and confirm the accuracy of the meeting summary.

[0298] Step 506 receives feedback information from conference participants on the conference summary. Different conference participants can evaluate the conference summary from different perspectives, and their feedback information can supplement omissions in the conference summary, correct errors in the conference summary, or provide more in-depth insights.

[0299] The feedback information may be in voice or text form. In the embodiment of the present application, a first feedback button with a voice feedback function may be set on the viewing interface of the conference summary. After the user clicks the first feedback button, the conference processing system may call the built-in microphone of the user terminal to collect voice to obtain feedback information in voice form. In the embodiment of the present application, a second feedback button may be set on the viewing interface of the conference summary. After receiving the user's click operation on the second feedback button, a text box may pop up to allow the user to enter feedback information in text form in the text box.

[0300] In step 507, the processing model is optimized according to the feedback information to improve the performance of the processing model.

[0301] The first processing model may be a large language model for generating conference summaries. According to the feedback information, the first processing model may be optimized to improve the generation quality of the conference summaries, so that the conference summaries may be more accurately reflected.

[0302] For example, user feedback information A indicates that a key decision is omitted from the meeting summary. The embodiment of the present application can adjust the algorithm or parameters of the first processing model such as the large language model according to the feedback information A to enhance the first processing model's ability to recognize keywords, so that similar information is no longer omitted in subsequent meeting summaries.

[0303] Specifically, first, based on the feedback information A, the missing key decision content can be analyzed to extract the core keywords in the missing key decision content; then, the first processing model can be retrained. During the retraining of the first processing model, the weights of these core keywords in the loss function calculation are increased. Assuming that the loss function is a cross-entropy loss function, for training samples containing core keywords, a weight coefficient greater than 1 can be multiplied when calculating the loss. The size of this weight coefficient can be dynamically adjusted according to the importance of the core keywords and the severity of the omission. If a core keyword is very important and has been frequently omitted in previous meeting summaries, a larger weight coefficient can be set; if the core keyword is relatively unimportant, the weight coefficient can be appropriately reduced.

[0304] The second processing model may be a machine learning model for determining the emotional state. Optimizing the second processing model according to the feedback information can improve the performance of the second processing model in accurately determining the emotional state of the participating user.

[0305] For example, user feedback information B indicates that the emotion analysis result in the meeting summary is inaccurate. Then, the embodiment of the present application can optimize the second processing model according to feedback information B, increase the recognition accuracy of the second processing model for different emotional states, and thus provide more accurate emotion analysis results in subsequent meeting summaries.

[0306] The embodiment of the present application may provide the following example for optimizing the second processing model:

[0307] First, a deviation analysis is conducted to determine the difference between the sentiment analysis results and the sentiment reflected in the meeting content to obtain the deviation analysis results.

[0308] Specifically, the emotional features in the meeting content are compared with the existing emotional analysis results, and the feedback information of multiple users is manually reviewed and summarized and analyzed (if multiple people point out the same inaccuracy, special attention will be paid to it) to obtain the deviation analysis results.

[0309] Then, the model is optimized based on the deviation analysis results.

[0310] Specifically, collect more meeting data with accurately labeled emotional states, which can cover various scenarios and emotion types.

[0311] Adjust the model structure, such as adding a neural network layer (such as an attention layer) that is more sensitive to emotional feature extraction, so that it can focus on key emotional information such as specific tone vocabulary and text features corresponding to expression changes.

[0312] During the training process, weights are set for different emotional features based on the results of deviation analysis, and emotion-related features that frequently appear inaccurately analyzed in feedback are given higher weights in the loss function calculation.

[0313] For example, the second processing model often makes incorrect judgments about the "angry" emotional state. After comparing the content of the meeting with the existing sentiment analysis results, it was found that the second processing model did not process and identify specific features related to the "angry" emotional state very well. For example, from the transcribed text of the meeting audio, it can be seen that the "angry" emotional state is usually accompanied by a high-pitched speaking style and a rapid speaking speed. These intonation features are important clues for judging anger. At the same time, in the transcribed text, words with strong negative emotional tendencies such as "dissatisfaction" and "protest" are also target word features of the "angry" emotional state.

[0314] In order to solve this problem, in the process of optimizing the training of the second processing model, the weight of the target intonation and word features that are closely related to the "angry" emotional state can be deliberately increased in the calculation of the loss function of the second processing model. In this way, the second processing model will pay more attention to the target intonation and word features during the training process, and by constantly adjusting its own parameters, the second processing model can more accurately identify the "angry" emotional state when processing the meeting content in the subsequent process, thereby improving the accuracy of emotional analysis. This feature adjustment method for specific emotions can be analogized to other emotional states that are easily misanalyzed, thereby comprehensively improving the model's recognition accuracy for different emotional states.

[0315] Method Example 5

[0316] refer to Figure 6 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0317] Step 601: Obtain the meeting content; the meeting content specifically includes: meeting audio and meeting video;

[0318] Step 602: construct a conference theme structure according to the conference content; the conference theme structure specifically includes: at least one theme;

[0319] Step 603: Generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0320] Step 604: Send the above interactive questions to the conference participants corresponding to any of the above topics;

[0321] Step 605: during the meeting, determine the emotional state of the participating users according to the meeting content corresponding to any topic;

[0322] Step 606: Perform communication barrier detection based on the above emotional state;

[0323] Step 607: When a communication barrier is detected, a corresponding second prompt message is sent to the conference participant.

[0324] Method Embodiment 5 adds steps 605 to 607 based on method embodiment 1. The above steps 605 to 607 play a role in real-time monitoring of the meeting atmosphere, timely discovery and handling of communication barriers to ensure efficient conduct of the meeting.

[0325] Among them, step 605 determines the emotional state of the participating users, which can keenly perceive the potential problem tendency in the meeting. The emotional state is often an intuitive reflection of the communication effect. Positive emotions help promote discussion, while negative emotions may indicate poor communication or contradictions. For example, dissatisfaction may come from conflicting opinions, and confusion may be due to unclear information communication.

[0326] Step 606 performs communication barrier detection, which can accurately identify the actual problems in the meeting based on the emotional state, so that the meeting organizer and participating users can be aware of the problems as soon as they occur, avoiding further expansion of the problems and affecting the efficiency and quality of the meeting.

[0327] Step 607 sends the corresponding second prompt information, which can provide specific guidance for solving communication barriers.

[0328] In an embodiment of the present application, when a possible communication barrier is detected, the voice function of the conference terminal can be used to send a second prompt message to some or all of the participating users. For example, if it is monitored that Li Si is dissatisfied in the "budget adjustment" discussion session, the corresponding second prompt message may be: "Please note that Li Si is dissatisfied when discussing the budget adjustment. It is recommended that everyone work together to ease the atmosphere and continue the orderly discussion." When other participating users receive this prompt message, they may take the initiative to take action to ease the atmosphere. For example, someone may say: "Li Si, don't worry, let's discuss this plan in depth together." The conference organizer may also adjust the direction of the meeting accordingly, for example: "Then let's temporarily suspend the discussion on the budget adjustment issue and see if there are other topics that are easier to reach consensus on."

[0329] In addition, the embodiments of the present application can continuously conduct comprehensive monitoring and analysis of the content of the meeting (including conference audio and video). By accurately interpreting the changes in tone, speaking speed, volume in the audio, as well as facial expression details, body language movements and other information in the video, the emotional state of the participating users can be accurately judged. Once emotions that may cause communication barriers (such as confusion, dissatisfaction, etc.) are found, targeted response strategies will be generated based on the specific situation. If it is noticed that the participating users are confused, guiding questions can be asked with the help of the conference terminal to promote further discussion and eliminate confusion. For example, the second prompt message may be: "Do you have any other questions about this topic? We can discuss it in more detail."

[0330] In summary, in the conference processing method of the embodiment of the present application, a voice assistant can be run on the conference terminal, and the voice assistant can interact with the participating users based on voice. Specifically, the voice assistant can send interactive questions, first prompt information and second prompt information to the user based on voice.

[0331] First, the embodiment of the present application uses natural language processing technology and machine learning algorithms to generate interactive questions, and uses a voice assistant to play interactive questions for participating users. In this way, users can be guided to complete and confirm the key information of the meeting, and can overcome the problems of information omissions and inaccurate summaries in the preliminary meeting summary to a certain extent, thereby significantly improving the quality of the meeting summary. In addition, participating users can directly participate in the generation process of the meeting summary, which improves their understanding and memory of the meeting content. In addition, a more natural and smooth communication method makes users feel more comfortable and satisfied.

[0332] Secondly, the emotion analysis of the participants in the present embodiment can better understand the emotional factors behind the conference theme, thereby improving the accuracy and comprehensiveness of the conference summary. In addition, emotion analysis can help discover possible communication barriers, adjust the direction of the meeting in time, or take measures to ease the atmosphere.

[0333] In addition, the embodiments of the present application can collect user feedback information on conference summaries based on user terminals, and use the feedback information for continuous optimization of the system. The regularly collected feedback information can help technicians in this field understand the actual performance of the processing model, and improve the performance of the processing model based on the adjustment of the processing model. Among them, the user can send feedback information based on voice, and the feedback information is heard and valued by the conference processing system, which increases the user's trust and reliance on the conference processing system. With the development of user usage habits and technology, the conference processing system can continuously adapt to new environments and new needs.

[0334] Furthermore, the embodiment of the present application can use HTTP / HTTPS protocol and WebSocket technology for data transmission. Among them, the HTTPS protocol uses an encrypted protocol to transmit data, which can prevent data leakage or tampering. The real-time communication technology corresponding to WebSocket realizes the instant transmission of data and improves the response speed of the conference processing system.

[0335] Method Example 6

[0336] refer to Figure 7 , shows a schematic flow chart of the steps of a conference processing method according to an embodiment of the present application. The method can be applied to a conference terminal or a user terminal, and specifically may include the following steps:

[0337] Step 701: receiving an interaction question sent by a server;

[0338] Step 702: Play the interactive question;

[0339] Step 703: receiving the reply information of the conference participant to the interactive question;

[0340] The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

[0341] In the embodiment of the present application, the conference terminal or user terminal can perform voice interaction with the participating users. Specifically, the conference terminal can play the interactive questions and collect the reply information in the form of voice.

[0342] The interactive questions in the embodiment of the present application can be one round of interactive questions or multiple rounds of interactive questions. Among them, multiple rounds of interactive questions can guide the participating users to think deeply about the topic and provide more detailed and valuable answers to better improve the meeting summary.

[0343] The above voice interaction does not require manual input from the user, which greatly saves time. Participants can respond to interactive questions instantly without typing on the keyboard, avoiding delays caused by slow typing or searching for a specific input interface. In addition, using voice interaction, participants can express their opinions and answer questions naturally, just like daily conversations. This naturalness reduces the user's learning cost and usage threshold, and both young participants who are familiar with technology and older participants who are not very proficient in new equipment can easily participate.

[0344] It can be understood that the conference terminal or the user terminal can also send reply information to the server, so that the server can correct and improve the conference summary according to the reply information.

[0345] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0346] Based on the above embodiment, this embodiment also provides a conference processing device. Figure 8 The conference processing device may specifically include: a conference content acquisition module 801, a topic structure construction module 802, an interactive question generation module 803 and an interactive question sending module 804.

[0347] The conference content acquisition module 801 is used to acquire the conference content; the conference content includes: conference audio and conference video;

[0348] The theme structure building module 802 is used to build a conference theme structure according to the conference content; the conference theme structure includes: at least one theme;

[0349] An interactive question generating module 803 is used to generate interactive questions corresponding to any topic according to the conference content corresponding to any topic;

[0350] The interactive question sending module 804 is used to send the interactive question to the participating users corresponding to any of the topics.

[0351] Optionally, the interactive question generating module includes:

[0352] A keyword acquisition module is used to acquire keywords from the conference content corresponding to any topic;

[0353] The template question generation module is used to generate interactive questions corresponding to the keywords using question templates.

[0354] Optionally, the process of determining the question template includes:

[0355] Collecting a first data sample; the first data sample includes: a meeting content sample corresponding to the theme of the meeting, and a meeting summary sample of the meeting;

[0356] Based on the sample meeting content and sample meeting abstracts, identify the types of questions and key messages;

[0357] Determine the question template based on the question type and key information.

[0358] Optionally, the keyword acquisition module includes:

[0359] A first keyword acquisition module is used to extract a first keyword that meets a first preset condition from the conference content corresponding to any topic; the first preset condition includes: a keyword that is related to any topic and appears in the conference content of any topic but does not appear in the conference summary; or

[0360] A second keyword acquisition module is used to extract a second keyword that meets the fuzzy condition from the conference content corresponding to any topic; or

[0361] The third keyword acquisition module is used to extract third keywords that meet the second preset condition from the conference content corresponding to any topic; the second preset condition includes: there is a semantic difference between the conference content and the conference summary of any topic.

[0362] Optionally, the question generating module includes:

[0363] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0364] The emotional question generation module is used to generate interactive questions corresponding to any topic according to the emotional state.

[0365] Optionally, the device further comprises:

[0366] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0367] The display module is used to display the first prompt information corresponding to the change of the emotional state of the participating users on the timeline of the conference content.

[0368] Optionally, the sentiment analysis module includes:

[0369] A first emotion determination module, used to determine a first emotional state of a conference user based on facial features of the conference user in a conference video corresponding to any topic;

[0370] A second emotion determination module, used to determine a second emotional state of a conference user according to the body features of the conference user in a conference video corresponding to any topic;

[0371] A third emotion determination module, used to determine a third emotion state of a conference user according to a voice feature of the conference user in the conference audio corresponding to any topic;

[0372] The fusion module is used to fuse the first emotional state, the second emotional state and the third emotional state to obtain the emotional state of the participating user.

[0373] Optionally, the device further comprises:

[0374] A reply receiving module, used to receive reply information of the participating users to the interactive questions;

[0375] The improvement and correction module is used to improve and correct the meeting summary according to the reply information.

[0376] Optionally, the device further comprises:

[0377] The sentiment analysis module is used to determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic;

[0378] Added module for adding said emotional state in meeting summary.

[0379] Optionally, the device further comprises:

[0380] Summary sending module, used to send conference summaries to conference participants;

[0381] A feedback receiving module, used to receive feedback information from conference participants on the conference summary;

[0382] The model optimization module is used to optimize the processing model according to the feedback information; the processing model includes: a first processing model for generating a meeting summary, or a second processing model for determining an emotional state.

[0383] Optionally, the device further comprises:

[0384] The emotion determination module is used to determine the emotional state of the participating users according to the meeting content corresponding to any topic during the meeting;

[0385] a communication barrier detection module, used for performing communication barrier detection according to the emotional state;

[0386] The prompt sending module is used to send corresponding second prompt information to the participating users when a communication barrier is detected.

[0387] This embodiment also provides a conference processing device, which can be applied to a conference terminal or a user terminal. Fig. 9 , the conference processing device may specifically include:

[0388] Receiving module 901, used for receiving interaction questions sent by the server;

[0389] A playing module 902 is used to play the interactive questions;

[0390] Reply module 903, used to receive reply information of the participating user to the interactive question;

[0391] The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

[0392] The embodiment of the present application also discloses a conference processing system, including: a conference terminal and a server;

[0393] The server obtains the conference content; the conference content includes: conference audio and conference video; a conference theme structure is constructed according to the conference content; the conference theme structure includes: at least one theme; an interactive question corresponding to any theme is generated according to the conference content corresponding to any theme; and the interactive question is sent to the conference terminal corresponding to the participating user corresponding to any theme;

[0394] The conference terminal receives the interactive questions sent by the server; plays the interactive questions; and receives reply information of the participating users to the interactive questions.

[0395] The conference terminal may also send reply information to the server, so that the server can correct and improve the conference summary according to the reply information.

[0396] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0397] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and server terminals (clusters).

[0398] The embodiments of the present disclosure may be implemented as a device that uses any appropriate hardware, firmware, software, or any combination thereof to perform desired configurations. The device may include: electronic devices such as terminal devices and server (cluster). Fig.10 An exemplary apparatus 2100 that can be used to implement various embodiments described in this application is schematically shown.

[0399] For one embodiment, Fig.10 An exemplary apparatus 2100 is shown having one or more processors 2102, a control module (chip set) 2104 coupled to at least one of the (one or more) processors 2102, a memory 2106 coupled to the control module 2104, a non-volatile memory / storage device 2108 coupled to the control module 2104, one or more input / output devices 2110 coupled to the control module 2104, and a network interface 2112 coupled to the control module 2104.

[0400] The processor 2102 may include one or more single-core or multi-core processors, and the processor 2102 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 2100 can be used as a terminal device, a server (cluster), and other devices described in the embodiments of the present application.

[0401] In some embodiments, the apparatus 2100 may include one or more computer-readable media (e.g., memory 2106 or non-volatile memory / storage 2108) having instructions 2114 and one or more processors 2102 combined with the one or more computer-readable media and configured to execute the instructions 2114 to implement a module to perform the actions described in the present disclosure.

[0402] For one embodiment, the control module 2104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 2102 and / or any suitable device or component in communication with the control module 2104 .

[0403] The control module 2104 may include a memory controller module to provide an interface to the memory 2106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0404] The memory 2106 may be used, for example, to load and store data and / or instructions 2114 for the device 2100. For one embodiment, the memory 2106 may include any suitable volatile memory, such as a suitable DRAM (Dynamic Random Access Memory). In some embodiments, the memory 2106 may include a double data rate type four synchronous dynamic random access memory.

[0405] For one embodiment, the control module 2104 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 2108 and the input / output device(s) 2110 .

[0406] For example, non-volatile memory / storage 2108 may be used to store data and / or instructions 2114. Non-volatile memory / storage 2108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital versatile optical disk drives).

[0407] The non-volatile memory / storage device 2108 may include storage resources that are physically part of the device on which the apparatus 2100 is installed, or it may be accessible to the device without being part of the device. For example, the non-volatile memory / storage device 2108 may be accessed via (one or more) input / output devices 2110 over a network.

[0408] The input / output device(s) 2110 may provide an interface for the apparatus 2100 to communicate with any other appropriate device, and the input / output device 2110 may include a communication component, an audio component, a sensor component, etc. The network interface 2112 may provide an interface for the apparatus 2100 to communicate through one or more networks, and the apparatus 2100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi (Wireless Fidelity), 2G (2nd Generation Wireless Telephone Technology), 3G (3rd Generation Wireless Telephone Technology), 4G (4th Generation Wireless Telephone Technology), 5G (5th Generation Wireless Telephone Technology), etc., or a combination thereof for wireless communication.

[0409] For one embodiment, at least one of the processor(s) 2102 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 2104. For one embodiment, at least one of the processor(s) 2102 may be packaged together with the logic of one or more controllers of the control module 2104 to form a system-in-package. For one embodiment, at least one of the processor(s) 2102 may be integrated on the same die with the logic of one or more controllers of the control module 2104. For one embodiment, at least one of the processor(s) 2102 may be integrated on the same die with the logic of one or more controllers of the control module 2104 to form a system-on-chip.

[0410] In various embodiments, the device 2100 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 2100 may have more or fewer components and / or different architectures. For example, in some embodiments, the device 2100 includes one or more cameras, a keyboard, a liquid crystal display screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, a dedicated integrated circuit, and a speaker.

[0411] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or non-volatile memory / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0412] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0413] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0414] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0415] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0416] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0417] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0418] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0419] The above is a detailed introduction to a conference processing method and device, an electronic device and a machine-readable medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A conference processing method, characterized in that: The method comprises: Obtaining conference content; the conference content includes: conference audio and conference video; Constructing a conference theme structure according to the conference content; the conference theme structure includes: at least one theme; Generate interactive questions corresponding to any topic based on the conference content corresponding to any topic; Send the interactive question to the participating users corresponding to any of the topics.

2. The method according to claim 1, characterized in that The generating of interactive questions corresponding to any topic according to the conference content corresponding to any topic includes: Get keywords from the conference content corresponding to any topic; Generate interactive questions corresponding to the keywords using question templates.

3. The method according to claim 2, characterized in that The process of determining the question template includes: Collecting a first data sample; the first data sample includes: a meeting content sample corresponding to the theme of the meeting, and a meeting summary sample of the meeting; Based on the sample meeting content and sample meeting abstracts, identify the types of questions and key messages; Determine the question template based on the question type and key information.

4. The method according to claim 2, characterized in that: The obtaining of keywords from the conference content corresponding to any topic includes: Extracting a first keyword that meets a first preset condition from the conference content corresponding to any topic; the first preset condition includes: a keyword that is related to any topic and appears in the conference content of any topic but does not appear in the conference summary; or Extracting a second keyword that meets the fuzzy semantic condition from the conference content corresponding to any topic; or A third keyword that meets a second preset condition is extracted from the conference content corresponding to any topic; the second preset condition includes: there is a semantic difference between the conference content and the conference summary of any topic.

5. The method according to claim 1, characterized in that The generating of interactive questions corresponding to any topic according to the conference content corresponding to any topic includes: Determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic; According to the emotional state, an interactive question corresponding to any topic is generated.

6. The method according to claim 1, characterized in that The method further comprises: Determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic; On the timeline of the conference content, the first prompt information corresponding to the change in the emotional state of the participating users is displayed.

7. The method according to claim 5 or 6, characterized in that: Determining the emotional state of the participating users according to the conference content corresponding to any topic includes: Determine a first emotional state of a conference user according to facial features of the conference user in a conference video corresponding to any topic; Determine a second emotional state of the participant user according to the body features of the participant user in the conference video corresponding to any topic; Determine a third emotional state of the conference user according to the voice features of the conference user in the conference audio corresponding to any topic; The first emotional state, the second emotional state and the third emotional state are integrated to obtain the emotional state of the participating user.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Receive response information from the participating users regarding the interactive questions; Improve and revise the meeting summary based on the response information.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Determine the emotional state of the users participating in the meeting based on the meeting content corresponding to any topic; Add said emotional state to the meeting summary.

10. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Send meeting summaries to participating users; Receive feedback information from conference participants regarding the conference summary; According to the feedback information, a processing model is optimized; the processing model includes: a first processing model for generating a meeting summary, or a second processing model for determining an emotional state.

11. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: During the meeting, the emotional state of the participating users is determined based on the meeting content corresponding to any topic; Based on the emotional state, performing communication disorder detection; When a communication barrier is detected, a corresponding second prompt message is sent to the participating users.

12. A conference processing method, characterized in that: Applied to a conference terminal or a user terminal, the method includes: Receive interaction questions sent by the server; Play the interactive questions; Receive response information from the participating users regarding the interactive questions; The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

13. A conference processing device, characterized in that: The device comprises: A conference content acquisition module is used to acquire conference content; the conference content includes: conference audio and conference video; A theme structure building module, used to build a conference theme structure according to the conference content; the conference theme structure includes: at least one theme; An interactive question generation module is used to generate interactive questions corresponding to any topic according to the conference content corresponding to any topic; The interactive question sending module is used to send the interactive question to the participating users corresponding to any of the topics.

14. A conference processing device, characterized in that: Applied to a conference terminal or a user terminal, the device comprises: The receiving module is used to receive the interaction questions sent by the server; A playback module, used to play interactive questions; A reply module, used to receive reply information from the participating users to the interactive questions; The process of generating the interactive questions includes: obtaining the meeting content; the meeting content includes: meeting audio and meeting video; constructing a meeting theme structure based on the meeting content; the meeting theme structure includes: at least one theme; generating interactive questions corresponding to any theme based on the meeting content corresponding to any theme.

15. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, causes the processor to execute the method according to any one of claims 1 to 12.

16. A machine-readable medium having executable codes stored thereon, which, when executed, causes a processor to perform the method according to any one of claims 1 to 12.

17. A conference processing system, characterized in that: include: Conference terminal and server; The server obtains the conference content; the conference content includes: conference audio and conference video; a conference theme structure is constructed according to the conference content; the conference theme structure includes: at least one theme; an interactive question corresponding to any theme is generated according to the conference content corresponding to any theme; and the interactive question is sent to the conference terminal corresponding to the participating user corresponding to any theme; The conference terminal receives the interactive questions sent by the server; plays the interactive questions; and receives reply information of the participating users to the interactive questions.

Images