Evaluation report generation method, electronic equipment and storage medium

Abstract

The invention provides an evaluation report generation method, electronic equipment and a computer readable storage medium, and relates to the field of computers. According to the method and the device, the online interaction application can generate the evaluation report after the online interaction is finished. The report may evaluate the performance of the user's online interaction. The online interaction application can be an online conference application, an online teaching application and an online live broadcast application. For example, in an online conference scene, an online conference application can generate an evaluation report after a conference is ended, and the evaluation report is used for evaluating conference performance of a user. Therefore, the user can conveniently know own conference performance, and the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an evaluation report generating method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of mobile Internet, users can interact online through online interactive applications (e.g., online conference applications, online teaching applications, online live broadcast applications, etc.) to facilitate communication between users. However, after the online interaction ends, these online interactive applications cannot meet the needs of users who want to understand their own interactive performance.

[0003] For example, Tencent Meeting TM , Feishu TM , DingTalk Conference TM Online conference applications can record meeting information, organize meeting information after the meeting, and generate minutes of the current meeting to help users organize meeting information and reduce their workload. However, this is limited to recording and organizing the information of the meeting itself, and cannot provide users with some value-added services, such as evaluating the user's meeting performance. Summary of the invention

[0004] Some embodiments of the present application provide an evaluation report generation method, an electronic device, a computer-readable storage medium, and a computer program product. In the technical solution of the present application, in an online interactive scenario, the online interactive application can extract interactive information from the interactive information of the online interaction after the interaction ends, and determine the key information in the interactive information, and then generate an evaluation report based on the field to which the interactive information belongs and the key information. The evaluation report can evaluate the performance of the user, so that the user can understand his or her performance in the interactive process in a timely manner.

[0005] In a first aspect, the present application provides an evaluation report generation method for an electronic device, the method comprising: obtaining interaction information during the operation of an online interactive application; based on the interaction information, determining key information during the operation of the online interactive application; and obtaining an evaluation report based on the field to which the interaction information belongs and the key information.

[0006] Among them, online interactive applications may include online conference applications, online teaching applications, online live broadcast applications, etc. Interactive information includes conference information, live broadcast information, teaching information, etc. Key information may include keywords, key images, keyword extension information, user question and answer information, and other information.

[0007] For example, for an online conference scenario, an online interactive application may be an online conference application. The interactive information in the operation of an online interactive application may be conference information, including information displayed by the online conference application and conference voice information. The field to which the interactive information belongs may be the field discussed at the conference, such as security and privacy, communications, computers and other technical fields. The evaluation report may include comprehensive diagnostic information, improvement suggestion information, and learning and plan recommendation information. Comprehensive diagnostic information is used to evaluate the user's conference performance, such as which aspects the user is more familiar with, which aspects the user is not familiar with, and which aspects the user is more interested in. Improvement suggestion information is used to provide improvement suggestions to the user, such as what measures can be taken to improve aspects that the user is not familiar with. Learning and plan recommendation information is used to recommend learning materials to the user, such as books, cases, papers, etc.

[0008] For another example, for an online teaching scenario, the online interactive application may be an online teaching application. The interactive information during the operation of the online interactive application may be teaching information, including information displayed by the online teaching application and teaching voice information. The field to which the interactive information belongs may be a subject field, such as Chinese, mathematics, chemistry, English, etc. The evaluation report may include teaching evaluation information, teaching suggestion information, and teaching learning plan information. The teaching evaluation information is used to evaluate the user's teaching performance, such as which knowledge points the user has explained more fully and which knowledge points have been ignored. The teaching suggestion information is used to provide teaching suggestions to the user, such as knowledge points that need to be emphasized. The teaching learning plan information is used to provide users with relevant teaching cases, such as the names and links of file information such as courseware information and videos.

[0009] For another example, for an online live broadcast scenario, the online interactive application may be an online live broadcast application. The interactive information in the operation of the online interactive application may be live broadcast information, including information displayed by the online live broadcast application and live broadcast voice information. The field to which the interactive information belongs may be a product type, such as daily necessities, home furnishings, electrical appliances, clothing, luggage, etc. The evaluation report may include information such as live broadcast evaluation information, live broadcast suggestion information, and live broadcast learning plan information. Among them, the live broadcast evaluation information is used to evaluate the user's behavioral performance during the live broadcast, such as which product information the user focused on, which product information was ignored, and which questions the audience answered. The live broadcast suggestion information is used to provide live broadcast suggestions to users, such as which product information needs to be introduced in detail. The live broadcast learning plan information is used to provide users with live broadcast learning cases, such as live broadcast learning materials, live broadcast case video links, etc.

[0010] In some embodiments, based on the interaction information, key information in the operation of the online interactive application is determined, including: extracting first key information from the interaction information; and expanding the first key information based on the field to which the interaction information belongs to obtain second key information.

[0011] The first key information may be a keyword and / or a key picture, and the second key information may be extended information of the keyword and / or the key picture.

[0012] For example, in a conference scenario, an online conference application can extract keywords from conference information, and then expand the keywords based on the field to which the conference information belongs to obtain the expanded information of the keywords. The expanded information of keywords is not a direct interpretation or retrieval of the keywords, but an expansion of the keywords in combination with the field to which the conference information belongs. This information is more in line with the content discussed in the conference, and can provide users with more effective information to help users better understand the content discussed in the conference.

[0013] In some embodiments, based on the field of the interactive information, the first key information is expanded to obtain the second key information, including: based on the field to which the interactive information belongs, the first key information is expanded to obtain multiple key information to be selected; multiple key information to be selected are displayed to the user, and the key information selected by the user from the multiple key information to be selected is used as the second key information.

[0014] The multiple key information to be selected may be multiple extended information of keywords and / or key images, and the second key information may be extended information that the user is interested in.

[0015] For example, in a conference scenario, keywords in conference information can be expanded to obtain multiple expanded information as candidate key information, and the multiple candidate key information can be displayed to the user. When a user is interested in a candidate key information, the key information can be selected as the second key information, for example, by clicking an interest button corresponding to the key information, which is the expanded information that the user is interested in. Then, an evaluation report can be obtained based on the expanded information that the user is interested in, and the evaluation report can evaluate the user's interest, so that the user can understand his or her interest more intuitively.

[0016] In some embodiments, extracting the first key information from the interactive information includes: extracting the first key information from the interactive information based on the evaluation model, wherein the first key information includes text (ie, keywords) and pictures (ie, key pictures).

[0017] Among them, the evaluation model mentioned in this application may be the intelligent model mentioned in this application.

[0018] The interactive information can be input into the intelligent model, and the intelligent model can extract the keywords and / or key images in the interactive information. Based on the intelligent model, the keywords and / or key images can be accurately extracted, so that the interactive information can be effectively expanded.

[0019] For example, in a meeting scenario, the meeting information can be filled into a preset prompt template to obtain a corresponding prompt, and then the prompt is input into the intelligent model, which performs semantic analysis on the prompt and outputs keywords and / or key images.

[0020] Taking keywords as an example, the content of the meeting information includes "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose a certain important authority into multiple permissions, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ...". The preset prompt template is "..., what are the keywords of the above information?". The prompt obtained by filling the content of the meeting information into the prompt template is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose a certain important authority into multiple permissions, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ..." and other information. After adding the above meeting information to the prompt text template, the prompt text obtained is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose a certain important authority into multiple permissions, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ......, what are the keywords of the above information?". Then, after the prompt is input into the intelligent model, the output of the intelligent model can be "Keywords: authority separation, object protection".

[0021] In some embodiments, extracting the first key information from the interaction information includes: acquiring display information selected by the user in the application interface of the online interactive application; and using the display information as the first key information.

[0022] For example, in a conference scenario, an online conference application can display the file information shared by users on the screen of an electronic device, such as slides, documents, videos, pictures, tables, etc. The user can circle some information on the screen of the electronic device, such as pictures or text, as the first key information, and then the online conference application can expand the information circled by the user to obtain the corresponding expanded information. In other words, the online conference application can expand based on the information selected by the user on the screen, making it easier for the user to understand the expanded information of the information they are concerned about, thereby improving the user experience.

[0023] In some embodiments, based on the interaction information, key information in the operation of the online interactive application is determined, including: determining the user's question and answer information in the interaction information, and using the question and answer information as key information, wherein the question and answer information includes the question and answer information in the collected user voice information and the question and answer information displayed on the interface of the online interactive application.

[0024] For example, in a conference scenario, the speech information of the local user can be identified from the conference voice information, for example, the speech collected by the local microphone is converted into text information as the speech information of the local user. Then, based on the speech information of the local user, a prompt about the user question and answer information is generated, for example, "..., the above is the user speech information, please identify which information is the user's question and which information is the user's answer". Then the prompt is input into the intelligent model, and the intelligent model analyzes the user speech information according to the prompt and outputs the user's question and answer information. The user question and answer information includes user questions and user answers. For example, the output of the intelligent model can be "User questions: 1,...; 2,...; User answers: 1,...; 2,...". The question and answer information of the local user can also be identified from the chat information displayed on the application interface of the online meeting.

[0025] In some embodiments, an evaluation report is obtained based on the field to which the interactive information belongs and the key information, including: inputting the field to which the key information and the interactive information belong into an evaluation model to obtain evaluation information output by the evaluation model; and obtaining an evaluation report based on the evaluation information.

[0026] Among them, the evaluation report mentioned in this application may be the intelligent analysis report mentioned in this application.

[0027] For example, in a meeting scenario, after determining the field to which the current meeting belongs, the extended information that the user is interested in, and the user's question and answer information, the online meeting application generates a prompt about comprehensive diagnostic information (as evaluation information) based on the field to which the meeting belongs, the extended information that the user is interested in, and the user's question and answer information, for example, "In the field of security, I have...problems, I am interested in..., I understand..., please evaluate my strengths and weaknesses based on the above information." The prompt is then input into the intelligent model, which analyzes the user's meeting behavior based on the prompt and outputs comprehensive diagnostic information for the user, for example, "In the current meeting, through analysis of your speeches, discussions, searches, etc., it is found that you have a very good understanding of the concept and significance of security architecture equipment, and are more interested in the equipment and principle cases of security architecture, but are relatively lacking in professionalism in the specific implementation methods of security design." Comprehensive diagnostic information can evaluate the user's behavioral performance in the meeting, allowing users to more easily understand their own strengths and weaknesses.

[0028] After the online conference application generates comprehensive diagnostic information about the user's conference behavior, it can also generate prompts about improvement suggestions (i.e., recommendation information) based on the comprehensive diagnostic information, for example, "What are the suggestions for improvement in view of my shortcomings?" The prompt is then input into the intelligent model, which analyzes the user's conference behavior based on the prompt and outputs improvement suggestions for the user, for example, "You can combine security threat analysis with security architecture design, establish and improve a security design pattern library and a security design solution library, and continuously absorb excellent design cases in the industry." This information can effectively provide users with improvement suggestions based on their behavioral performance in the meeting, thereby helping users improve their meeting performance.

[0029] After the online conference application generates the improvement suggestion information, it can also generate a prompt about the recommended information of the learning and expansion plan (i.e., the learning plan information) based on the improvement suggestion information, for example, "How should I implement the improvement suggestion? Please make a learning plan for me, such as books, cases, papers, etc." Then the prompt is input into the intelligent model, and the intelligent model analyzes the improvement suggestion information according to the prompt, and outputs the recommended information of the learning and expansion plan for the user, such as "Books: ...; Cases: ...; Papers: ...". This information can provide users with information about learning materials, saving users time and energy in searching for learning materials.

[0030] After the meeting, the online conference application can organize the comprehensive diagnostic information, improvement suggestion information and learning and expansion plan recommendation information to obtain an intelligent analysis report (i.e., an evaluation report). When the user views the intelligent analysis report, the comprehensive diagnostic information, improvement suggestion information and learning and expansion plan recommendation information are displayed.

[0031] In some embodiments, the evaluation model includes at least one of the following: a generative pre-trained transformer GPT model, a pre-trained bidirectional encoder BERT model, a long short-term memory network LSTM model, and a transformer Transformer model.

[0032] The evaluation model is a pre-trained model. Taking the training process of the GPT model as an example, a large amount of text can be collected, such as meeting minutes, web pages, books, news, forum posts, papers, journals, conversations, etc., and then these texts are preprocessed, such as data cleaning, word segmentation, tokenization, etc., to obtain a large number of training samples. Then, based on these training samples, the GPT model to be trained is trained to obtain a trained GPT model. It can be understood that the GPT model is trained using a large number of training samples. These training samples involve various fields. The GPT model can be comprehensively trained so that the GPT model can recognize the semantics of the input information and output the corresponding results based on the semantics of the input information.

[0033] For example, for the question-answering function, sentence pairs (such as questions and answers) can be collected as training samples for the GPT model. Then the sentence pairs are input into the GPT model, and the answers in the sentence pairs are masked or hidden. For example, the question in the sentence pair is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose an important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus safer. What field does the above information fall into?", and the answer in the sentence pair is "Field: Security and Privacy Protection". When the sentence pair is input into the GPT model, the answer: "Field: Security and Privacy Protection" can be masked or hidden. Then the GPT model outputs the corresponding prediction results based on the questions in the sentence pair, calculates the loss function based on the prediction results and the masked answers, and then adjusts the model parameters of the GPT model based on the loss function. In this way, the GPT model can be trained for question-answering tasks, so that the trained GPT model has question-answering functions.

[0034] For online conference scenarios, the model parameters of the GPT model can be further fine-tuned based on the conference scenario to improve the effect of the GPT model in the conference scenario. In some embodiments, the training samples of the GPT model can be obtained based on key information in the conference information, such as keywords, keyword extension information, etc., and then the model parameters of the trained GPT model can be further adjusted based on the training samples.

[0035] Exemplarily, based on the keyword: "open principle", a corresponding prompt can be generated: "Please explain the open principle from the perspective of security and privacy". Then the prompt is input into the GPT model as a training sample of the GPT model, and the GPT model outputs the corresponding prediction result. The loss function is calculated based on the prediction result and the label of the training sample, and then the model parameters of the GPT model are adjusted based on the loss function to train the GPT model. Among them, the label of the training sample is the true answer corresponding to the prompt, for example, "The open principle means that the design should not be a secret, security should not rely on the confidentiality of the mechanism design and implementation and the ignorance of the attacker, and the design of the product should be subject to sufficient testing." After the training end conditions are met, for example, the loss function converges, the number of training times reaches the set number, the model achieves the expected effect, etc., the training is terminated to obtain the trained GPT model.

[0036] In some embodiments, the online interactive application includes at least one of the following: an online conference application, an online teaching application, an online live broadcast application, and the interactive information includes at least one of the following: information displayed by the online interactive application, interactive voice information.

[0037] In some embodiments, corresponding to the online interactive application being an online conference application, the information displayed by the online interactive application includes file information and chat information displayed by the interactive application, and the interactive voice information includes audio data recorded during the operation of the online conference application; corresponding to the online interactive application being an online teaching application, the information displayed by the online interactive application includes courseware information and chat information displayed by the interactive application, and the interactive voice information includes audio data recorded during the operation of the online teaching application; corresponding to the online interactive application being an online live broadcast application, the information displayed by the online interactive application includes product information and chat information displayed by the online live broadcast application, and the interactive voice information includes audio data recorded during the operation of the online live broadcast application.

[0038] In some embodiments, the evaluation report includes at least one of the following: evaluation information, recommendation information, and learning plan information.

[0039] For online conference scenarios, the evaluation information may be comprehensive diagnosis information, the suggestion information may be improvement suggestion information, and the learning plan information may be learning and expansion plan recommendation information. For online teaching scenarios, the evaluation information may be teaching evaluation information, the suggestion information may be teaching suggestion information, and the learning plan information may be teaching learning plan information. For online live broadcast scenarios, the evaluation information may be live broadcast evaluation information, the suggestion information may be live broadcast suggestion information, and the learning plan information may be live broadcast learning plan information.

[0040] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, when the processor executes the instructions in the memory, the electronic device can execute the method described in any embodiment of the first aspect of the present application. The beneficial effects that can be achieved in the second aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0041] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on a computer, the computer can execute the method described in any embodiment of the first aspect. The beneficial effects that can be achieved in the third aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0042] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed, causes a computer to execute the method described in any embodiment of the first aspect. The beneficial effects that can be achieved in the fourth aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 An exemplary application scenario of the present application is shown;

[0044] Figure 2 An example diagram of a process for generating meeting minutes provided in some embodiments;

[0045] Figure 3 An example diagram of information of meeting minutes provided for some embodiments;

[0046] Figure 4 An example flow chart of the evaluation report generation method provided in the embodiment of the present application;

[0047] Figure 5A Example of the conference intelligent service interface provided in the embodiment of the present application Figure 1 ;

[0048] Figure 5B An example diagram of a chat interface provided in an embodiment of the present application;

[0049] Figure 6 Example of the conference intelligent service interface provided in the embodiment of the present application Figure 2 ;

[0050] Figure 7 Example of the conference intelligent service interface provided in the embodiment of the present application Figure 3 ;

[0051] Figure 8An example diagram of a process for an online conference application according to an embodiment of the present application to push extended information based on keywords and / or key images;

[0052] Fig. 9 An example diagram of a process for an online conference application according to an embodiment of the present application to push extended information based on user interaction behavior;

[0053] Fig.10 An example diagram of a process for adjusting a push strategy for extended information based on interest feedback in an online conference application provided in an embodiment of the present application;

[0054] Fig.11 Example of the conference intelligent service interface provided in the embodiment of the present application Figure 4 ;

[0055] Fig.12 An example flow chart of the process of generating an intelligent analysis report provided in an embodiment of the present application;

[0056] Fig.13 Schematic diagram of the module interaction of the online conference application provided in the embodiment of the present application Figure 1 ;

[0057] Fig.14 Schematic diagram of the module interaction of the online conference application provided in the embodiment of the present application Figure 2 ;

[0058] Fig.15 Schematic diagram of the module interaction of the online conference application provided in the embodiment of the present application Figure 3 ;

[0059] Fig.16 Schematic diagram of the module interaction of the online conference application provided in the embodiment of the present application Figure 4 ;

[0060] Fig.17 A schematic diagram of the architecture of an online conference application provided in an embodiment of the present application;

[0061] Fig.18 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0062] The embodiment of the present application is used to provide a method for generating an evaluation report. The method provided by the present application can automatically generate an evaluation report after the online interaction ends, and the evaluation report is used to evaluate the user's interactive performance, so that the user can more easily understand his or her own interactive performance.

[0063] It can be understood that the technical solution of the present application is applicable to various online interactive scenarios, such as online meetings, online teaching, online live broadcasts, etc. For the convenience of description, the following mainly takes the online meeting scenario as an example for detailed description.

[0064] Figure 1 An exemplary application scenario of the present application is shown. Figure 1 , user A can use the online conference application (e.g., Tencent Conference) on the electronic device 100 TM , Feishu TM , DingTalk Conference TM Online meetings can be held with other participants through the use of applications that support online meeting functions, so that users can communicate remotely with other participants.

[0065] The present application does not limit the specific form of the electronic device 100, and the electronic device 100 may be a mobile phone, a laptop computer, a tablet, a large-screen device, a desktop computer, an augmented reality (AR) / virtual reality (VR) device, a personal digital assistant (PDA), etc. A tablet is used as an example of the electronic device 100 below.

[0066] In some embodiments, during the meeting, the online meeting application can record the meeting information of the current meeting, and automatically organize the recorded meeting information after the meeting to generate corresponding meeting minutes. In this way, the user's workload can be reduced, and the user's hands and brain can be effectively freed, so that the user has more time and energy to focus on the meeting itself.

[0067] refer to Figure 2 ,The process of generating meeting minutes by online conference application includes the following steps:

[0068] (1) Meeting Recording

[0069] The online conference application has the function of automatically generating meeting minutes. When the user turns on this function, the online conference application can record the current meeting and end the recording after the meeting to obtain the meeting video of the current meeting.

[0070] (2) Audio Extraction

[0071] After the meeting is over, the electronic device 100 can extract the audio in the meeting video. For example, the audio in the meeting video can be extracted by tools such as advanced Linux sound architecture (ALSA) and pulse audio system (PulseAudio).

[0072] (3) Automatic transcription

[0073] After extracting the audio from the conference video, automatic speech recognition (ASR) technology can be used to convert the extracted audio into corresponding text.

[0074] (4) Punctuation prediction

[0075] Punctuation prediction refers to segmenting the text accordingly based on semantics and adding corresponding punctuation marks at the segmentation positions, such as commas, periods, semicolons, etc.

[0076] After the extracted audio is converted into the corresponding text, the text converted from the audio can be segmented using natural language processing (NLP) technology, and corresponding punctuation marks can be added at the segmentation positions. In some embodiments, punctuation prediction processing can be performed on the text converted from the audio using models such as bidirectional encoder representations from transformers (BERT) and enhanced context-aware pre-training (ALBERT).

[0077] (5) Generate minutes

[0078] After adding punctuation marks to the text after the conference audio is converted, the text information can be further sorted, for example, paragraph adjustment, format adjustment, position adjustment, etc., so as to obtain the corresponding meeting minutes after the sorting is completed.

[0079] For example, Figure 3 As shown, the meeting minutes may include the meeting topic, meeting time, participants, meeting content, etc. For example, the meeting topic is "Introduction to the Separation of Privileges Principle", the meeting time is "xxxx / xx / xx", the participants include user A, user B, user C, user D..., and the meeting content includes "xxxxxxx".

[0080] In the above embodiment, after the meeting, the online conference application can automatically generate meeting minutes to help users record and organize meeting information, but it is limited to recording and organizing the content of the meeting itself and cannot provide users with some value-added services, such as being unable to evaluate users' meeting behavior.

[0081] In order to solve the above technical problems, the embodiment of the present application provides a method for generating an evaluation report. In the method of this embodiment, based on the interactive information in the operation of online interactive applications, such as conference information in the operation of online conference applications (such as conference shared file information, conference voice information, etc.), teaching information in the operation of online teaching applications (such as courseware information, teaching voice information, etc.), and live broadcast information in the operation of online live broadcast applications (such as displayed product information, interactive information with buyers, etc.), determine the key information in the online interactive information, such as keywords, keyword extension information, user question and answer information, etc., and then obtain an evaluation report based on the field and key information to which the interactive information belongs. For example, the corresponding prompt information (prompt) can be obtained based on the field and key information to which the interactive information belongs, and then the prompt is input into an intelligent model (or "AI model", "big data model") that can generate the information required for the evaluation report, and an evaluation report is generated based on the output result of the intelligent model. Prompt is used to guide the model to generate an expected answer or response, which can be a short question, a description, a set of instructions, or a complete conversation. The evaluation report is used to evaluate the user's behavior performance, making it easier for the user to understand their own behavior performance, and to provide other additional information to the user to improve the user experience.

[0082] In addition, the evaluation report is determined based on the field to which the interactive information belongs, so that the evaluation content of the evaluation report can be more closely related to the interactive content and improve the accuracy of the evaluation. For example, if the current meeting discusses the field of security and privacy, the user's meeting performance can be evaluated in the field of security and privacy, such as what the user knows about the field of security and privacy, and what the user does not know about the field of security and privacy.

[0083] It can be understood that the interactive information in the operation of the online interactive application may include the information displayed by the online interactive application and the interactive voice information. Corresponding to the online interactive application being an online conference application, the information displayed by the online interactive application may include the file information (i.e., conference shared file information) and chat information displayed by the online conference application, and the interactive voice information includes the audio data recorded by the electronic device during the operation of the online conference application. Corresponding to the online interactive application being an online teaching application, the information displayed by the online interactive application includes the courseware information and chat information displayed by the online teaching application, and the interactive voice information includes the audio data recorded by the electronic device during the operation of the online teaching application. Corresponding to the online interactive application being an online live broadcast application, the information displayed by the online interactive application may include the product information and chat information displayed by the online live broadcast application, and the interactive voice information includes the audio data recorded by the electronic device during the operation of the online live broadcast application.

[0084] It can be understood that for online conference scenarios, the online interactive application can be an online conference application. The interactive information in the operation of the online interactive application can be conference information, including information displayed by the online conference application and conference voice information. The field to which the interactive information belongs can be the field discussed in the conference, for example, technical fields such as security and privacy, communications, and computers. The evaluation report can include comprehensive diagnostic information, improvement suggestion information, and learning and plan recommendation information. Comprehensive diagnostic information is used to evaluate the user's meeting performance, such as which aspects the user is more familiar with, which aspects the user is not familiar with, and which aspects the user is more interested in. Improvement suggestion information is used to provide improvement suggestions to users, such as what measures can be taken to improve aspects that users are not familiar with. Learning and plan recommendation information is used to recommend learning materials to users, such as books, cases, papers, etc.

[0085] For example, in an online conference scenario, an online conference application can obtain conference information and determine key information in the conference information, such as user question and answer information and information that the user is interested in. Then the field and key information can be filled into the preset prompt template to obtain the corresponding prompt, and the prompt can be input into the intelligent model to instruct the intelligent model to output the corresponding result based on the field and key information, and generate a corresponding evaluation report based on the output result of the intelligent model. The evaluation report can evaluate the user's conference performance (such as the user's question and answer behavior and interests, etc.) in combination with the field to which the conference belongs.

[0086] It can be understood that for online teaching scenarios, the online interactive application can be an online teaching application. The interactive information in the operation of the online interactive application can be teaching information, including information displayed by the online teaching application and teaching voice information. The field to which the interactive information belongs can be a subject field, such as Chinese, mathematics, chemistry, English, etc. The evaluation report can include teaching evaluation information, teaching suggestion information, and teaching learning plan information. The teaching evaluation information is used to evaluate the user's teaching performance, such as which knowledge points the user has explained more fully and which knowledge points have been ignored. The teaching suggestion information is used to provide users with teaching suggestions, such as knowledge points that need to be emphasized. The teaching learning plan information is used to provide users with relevant teaching cases, such as the names and links of file information such as courseware information and videos.

[0087] For example, in an online teaching scenario, the online teaching application can obtain teaching information and determine key information in the teaching information, such as the user's question and answer information. Then, the field and key information can be filled into the preset prompt template to obtain the corresponding prompt, and the prompt can be input into the intelligent model to instruct the intelligent model to output the corresponding result based on the field and key information, and generate a corresponding evaluation report based on the output result of the intelligent model. The evaluation report can evaluate the user's teaching performance in combination with the current teaching subject.

[0088] It can be understood that for online live broadcast scenarios, the online interactive application can be an online live broadcast application. The interactive information in the operation of the online interactive application can be live broadcast information, including information displayed by the online live broadcast application and live broadcast voice information. The field to which the interactive information belongs can be a product type, such as daily necessities, home furnishings, electrical appliances, clothing, luggage, etc. The evaluation report can include information such as live broadcast evaluation information, live broadcast suggestion information, and live broadcast learning plan information. Among them, the live broadcast evaluation information is used to evaluate the user's behavioral performance during the live broadcast, such as which product information the user focused on, which product information was ignored, and which questions the audience answered. The live broadcast suggestion information is used to provide live broadcast suggestions to users, such as which product information needs to be introduced in detail. The live broadcast learning plan information is used to provide users with live broadcast learning cases, such as live broadcast learning materials, live broadcast case video links, etc.

[0089] For example, in an online live broadcast scenario, the online live broadcast application can obtain live broadcast information and determine key information in the live broadcast information, such as the user's question and answer information. Then, based on the field and key information to which the teaching information belongs, the corresponding prompt is obtained and the prompt is input into the intelligent model. Then, according to the output results of the intelligent model, a corresponding evaluation report can be obtained. The evaluation report can evaluate the user's live broadcast performance in combination with the current live broadcast product type.

[0090] It can be understood that the model involved in the evaluation report generation method of the present application can be a pre-trained intelligent model, such as a generative pre-trained transformer (GPT), a pre-trained bidirectional encoder representations from transformers (BERT), a long short term memory network (LSTM), a transformer model (Transformer), etc. The intelligent model of the present application can recognize the semantics of input information, such as questions, instructions, etc., and then output the corresponding results according to the semantics of the input information, such as the answer to the question, the response result of the instruction, etc.

[0091] The following is a detailed description of the functions and training methods of the models involved in the evaluation report generation method of this application. For example, the GPT model is taken as an example for detailed description.

[0092] The GPT model has functions such as question answering, common sense reasoning, semantic similarity recognition, and classification. For example, for the question answering function, when the input of the GPT model is: "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose an important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus safer. What is the scope of the above information?", the GPT model can output the corresponding answer: "Field: Security and privacy protection".

[0093] In some embodiments, a large amount of text can be collected, such as meeting minutes, web pages, books, news, forum posts, papers, journals, conversations, etc., and then these texts are preprocessed, such as data cleaning, word segmentation, tokenization, etc., to obtain a large number of training samples. Then, based on these training samples, the GPT model to be trained is trained to obtain a trained GPT model. It can be understood that the GPT model is trained using a large number of training samples, and these training samples involve various fields. The GPT model can be comprehensively trained so that the GPT model can recognize the semantics of the input information and output corresponding results based on the semantics of the input information.

[0094] For example, for the question-answering function, sentence pairs (such as questions and answers) can be collected as training samples for the GPT model. Then the sentence pairs are input into the GPT model, and the answers in the sentence pairs are masked or hidden. For example, the question in the sentence pair is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose an important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus safer. What field does the above information fall into?", and the answer in the sentence pair is "Field: Security and Privacy Protection". When the sentence pair is input into the GPT model, the answer: "Field: Security and Privacy Protection" can be masked or hidden. Then the GPT model outputs the corresponding prediction results based on the questions in the sentence pair, calculates the loss function based on the prediction results and the masked answers, and then adjusts the model parameters of the GPT model based on the loss function. In this way, the GPT model can be trained for question-answering tasks, so that the trained GPT model has question-answering functions.

[0095] For online conference scenarios, the model parameters of the GPT model can be further fine-tuned based on the conference scenario to improve the effect of the GPT model in the conference scenario. In some embodiments, the training samples of the GPT model can be obtained based on key information in the conference information, such as keywords, keyword extension information, etc., and then the model parameters of the trained GPT model can be further adjusted based on the training samples.

[0096] Exemplarily, based on the keyword: "open principle", a corresponding prompt can be generated: "Please explain the open principle from the perspective of security and privacy". Then the prompt is input into the GPT model as a training sample of the GPT model, and the GPT model outputs the corresponding prediction result. The loss function is calculated based on the prediction result and the label of the training sample, and then the model parameters of the GPT model are adjusted based on the loss function to train the GPT model. Among them, the label of the training sample is the true answer corresponding to the prompt, for example, "The open principle means that the design should not be a secret, security should not rely on the confidentiality of the mechanism design and implementation and the ignorance of the attacker, and the design of the product should be subject to sufficient testing." After the training end conditions are met, for example, the loss function converges, the number of training times reaches the set number, the model achieves the expected effect, etc., the training is terminated to obtain the trained GPT model.

[0097] The following describes in detail the evaluation report generation method provided in the embodiment of the present application by taking an online conference scenario as an example.

[0098] Figure 4 This is a flowchart of an evaluation report generation method provided in an embodiment of the present application. Figure 4 As shown, the evaluation report generation method provided in the embodiment of the present application includes the following steps:

[0099] S101: Acquire the meeting information of the current meeting.

[0100] The conference information includes the information displayed by the online conference application and the conference voice information. The information displayed by the online conference application includes the conference shared file information (for example, slides, documents, videos, pictures, tables, etc.) and chat information displayed by the online conference application.

[0101] In some embodiments, during the meeting, the online conference application can use ASR technology to convert the conference voice recorded by the electronic device into corresponding text in real time to obtain the conference voice information as the conference information. The online conference application can also use optical character recognition (OCR) and image recognition technology to identify the file information and / or chat information displayed by the online conference application to extract the information displayed by the online conference application as the conference information.

[0102] For example, refer to Figure 5A In a scenario where a user uses an online conference application on an electronic device 100 to hold an online conference with other participants, when the user or other participants share the screen, the screen of the electronic device 100 can be divided into window 11 and window 12 for display. Among them, window 11 is used to display the shared screen interface, and the shared screen interface includes file information shared by the participants in the conference. During the conference, the online conference application can extract the information displayed in window 11 as conference information.

[0103] For example, refer to Figure 5B , the window 11 of the electronic device 100 displays the chat information. In this case, the online conference application can extract the chat information displayed in the window 11 as the conference information.

[0104] S102: Determine the field to which the conference belongs based on the conference information.

[0105] After the online conference application obtains the conference information of the current conference, it can analyze and process the conference information to determine the field to which the conference belongs.

[0106] In some embodiments, the online conference application can generate a prompt about the field to which the conference belongs based on the conference information of the current conference. The online conference application then inputs the prompt into the intelligent model, which analyzes and processes the conference information according to the prompt and outputs the field to which the conference belongs.

[0107] Specifically, the online conference application can fill the conference information into the prompt template about the field to which the conference belongs, obtain the prompt about the field to which the conference belongs, and then input the prompt into the intelligent model. The intelligent model analyzes the conference information according to the prompt and outputs the field to which the conference belongs.

[0108] For example, the prompt template about the field to which the conference belongs can be: "..., what field does the above information belong to?". The conference information includes information such as "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose a certain important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ..." After filling the above conference information into the above prompt template about the field to which the conference belongs, the resulting prompt is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Decompose a certain important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ......, what field does the above information belong to?". Then, after the prompt is input into the intelligent model, the intelligent model can output: "Field: Security and Privacy Protection".

[0109] In this application, the intelligent model can be deployed in the cloud or locally. The online conference application can call the interface of the intelligent model to use the intelligent model.

[0110] S103: Generate extended information of the conference information based on the field to which the current conference belongs.

[0111] During the meeting, when users encounter professional technical points, information they do not understand, or want to know more in-depth information, they often need to search online or search after the meeting, which is less timely and convenient. In this embodiment, during the meeting, the online conference application can push extended information of the conference information to the user in real time. Moreover, this extended information is not a summary or overview of the conference information, nor is it a literal interpretation of the conference information (such as keywords, etc.) or a direct search result. The extended information of this application can be information that is enhanced, extended, and expanded in combination with the conference context, social cases, etc., or it can be information that is related to the theme of the meeting, or it can be a strongly related explanation of the conference information and an explanation of information that is highly discussed on the spot by the participants.

[0112] In some embodiments, the online conference application can identify keywords in the conference information, and then generate extended information of the keyword based on the field to which the conference belongs.

[0113] For example, a prompt about keywords can be generated based on the conference information, such as "..., what are the keywords of the above information?". Then the prompt is input into the intelligent model, and the intelligent model identifies and outputs the keywords in the conference information according to the prompt. For example, the output of the intelligent model is "Keywords: permission separation, object protection". After the intelligent model outputs the keywords of the conference information, the online conference application can generate a prompt for obtaining extended information of the keywords based on the keywords and the field to which the conference belongs, such as "In the field of security and privacy, for permission separation and object protection, please show me other relevant information in the form of entries in combination with important concepts, key technologies, hot events, etc." Then the prompt is input into the intelligent model, and the intelligent model outputs extended information of the keywords according to the prompt, such as "1. Three-sided isolation refers to the security isolation of the management plane, the control plane, and the user from each other. 2. Open principle (Shannon's motto - the enemy knows you better). 3. The open principle means that the design should not be a secret, and security should not rely on the confidentiality of the mechanism design and implementation and the ignorance of the attacker. The design of the product should be fully tested."

[0114] For example, Figure 5A As shown, the extended information of the keywords output by the intelligent model: information S1, information S2 and information S3, can be displayed in the intelligent prompt box M1, intelligent prompt box M2 and intelligent display box M3 of the window 12 respectively.

[0115] In other embodiments, the online conference application can also identify key images in the conference information, and then generate extended information of the key images based on the field to which the conference belongs. For example, images that appear repeatedly in the conference information are identified as key images, or semantic recognition is performed on the conference information to determine the key images in the conference information based on the contextual semantics of the conference information.

[0116] In other embodiments, the online conference application can identify the information selected by the user on the screen, such as pictures, text, etc., and then generate extended information of the user selected information based on the field to which the conference belongs.

[0117] For example, Figure 6As shown, when the user circles the picture P1 on the screen of the electronic device 100, the online conference application generates a prompt about the picture P1 based on the picture P1 selected by the user: "Please explain the picture P1 in the field of security and privacy", and then inputs the prompt into the intelligent model. After the intelligent model analyzes the picture P1 according to the prompt, it outputs the extended information of the picture P1. For example, the extended information output by the intelligent model is the information S4 displayed in the intelligent prompt box M4, and its specific information is "Aristotle proposed the famous three-element theory, xxxxx." It should be understood that information S4 is not a direct introduction and explanation of the picture P1, but an extended introduction and explanation of the picture P1 in the field of security and privacy, and its information has a certain relationship with the information of the current meeting. For example, the three-element theory proposed by Aristotle in information S4 has a certain analogy with the principle of separation of powers introduced in the shared file information of the meeting at this time, that is, there is a certain relationship.

[0118] When the user circles the word "key" on the screen of the electronic device 100, the online conference application generates a prompt about the word "key" based on the word "key" selected by the user: "Please explain the key from the perspective of security and privacy", and then inputs the prompt into the intelligent model, which analyzes the word "key" according to the prompt and outputs the extended information of the word "key". For example, the extended information output by the intelligent model is the information S5 displayed in the intelligent prompt box M5, and its specific information is "Key, a key is a password or code used to encrypt and decrypt data, which can be a string, a number, or a complex combination of numbers and letters". It should be understood that information S5 is not a literal interpretation of the word "key" itself, but information related to security keys, rather than a lock-picking tool in the usual sense.

[0119] Depend on Figure 5A , Figure 5B and Figure 6 It can be seen that during the meeting, the online meeting application can push extended information related to the meeting information to users in real time to help users understand the specific information discussed in the meeting, and help users learn more extended information about the meeting, reduce users' retrieval of meeting information, and save users' time and energy.

[0120] S104: Determine the extended information that the user is interested in.

[0121] The online conference application can determine the extended information that the user is interested in based on the user's interactive behavior. The interactive behavior includes but is not limited to click, slide, long press, tap and other operation behaviors.

[0122] In some embodiments, as described above Figure 5A , Figure 5B and Figure 6 As shown, each smart prompt box includes not only the extended information of the conference information, but also the interested button and the uninterested button corresponding to the extended information, and the user can click these buttons according to his or her interests. The online conference application can determine the extended information that the user is interested in based on the user's click situation.

[0123] For example, Figure 6 As shown, if the user clicks the Interest button L1 in the smart prompt box M3, it can be determined that the information S3 is the extended information that the user is interested in. If the user does not click the Interest button L1, it can be determined that the information S3 is not the extended information that the user is interested in. If the user clicks the Not Interested button L2 in the smart prompt box M3, it can be determined that the information S3 is the extended information that the user is not interested in.

[0124] S105: Generate comprehensive diagnostic information based on the field to which the current conference belongs, the extended information that the user is interested in, and the user's question and answer information in the conference information.

[0125] In some embodiments, the speech information of the local user can be identified from the conference voice information, for example, the speech collected by the local microphone is converted into text information as the speech information of the local user. Then, based on the speech information of the local user, a prompt about the user question and answer information is generated, for example, "..., the above is the user speech information, please identify which information is the user's questions and which information is the user's answers". The prompt is then input into the intelligent model, and the intelligent model analyzes the user speech information according to the prompt and outputs the user's question and answer information. The user question and answer information includes user questions and user answers. For example, the output of the intelligent model can be "User questions: 1,...; 2,...; User answers: 1,...; 2,..."

[0126] In another embodiment, the question and answer information of the local user can also be identified from the chat information displayed on the application interface of the online conference.

[0127] After determining the field to which the current meeting belongs, the extended information that the user is interested in, and the user's question and answer information, the online conference application generates a prompt about comprehensive diagnostic information based on the field to which the meeting belongs, the extended information that the user is interested in, and the user's question and answer information, for example, "In the field of security, I have... questions, I am interested in..., I understand..., please evaluate my strengths and weaknesses based on the above information." The prompt is then input into the intelligent model, which analyzes the user's meeting behavior based on the prompt and outputs comprehensive diagnostic information for the user, for example, "In the current meeting, through analysis of your speeches, discussions, searches, etc., it is found that you have a very good understanding of the concept and significance of security architecture equipment, and are more interested in the equipment and principle cases of security architecture, but are relatively lacking in professionalism in the specific implementation methods of security design." Comprehensive diagnostic information can evaluate the user's behavior in the meeting, allowing users to more easily understand their own strengths and weaknesses.

[0128] S106: Generate improvement suggestion information based on the comprehensive diagnosis information.

[0129] After the online conference application generates comprehensive diagnostic information about the user's conference behavior, it can also generate prompts about improvement suggestions based on the comprehensive diagnostic information, such as "What are the suggestions for improvement based on my shortcomings?" The prompt is then input into the intelligent model, which analyzes the user's conference behavior based on the prompt and outputs improvement suggestions for the user, such as "You can combine security threat analysis with security architecture design, establish a complete security design pattern library and security design solution library, and continuously absorb excellent design cases in the industry." This information can effectively provide users with improvement suggestions based on their behavioral performance in the meeting, thereby helping users improve their meeting performance.

[0130] S107: Generate learning and expansion plan recommendation information based on the improvement suggestion information.

[0131] After the online conference application generates the improvement suggestion information, it can also generate a prompt about the recommended information of the learning and expansion plan based on the improvement suggestion information, for example, "How should I implement the improvement suggestion? Please make a learning plan for me, such as books, cases, papers, etc." Then the prompt is input into the intelligent model, and the intelligent model analyzes the improvement suggestion information according to the prompt and outputs the recommended information of the learning and expansion plan for the user, such as "Books: ...; Cases: ...; Papers: ...". This information can provide users with information about learning materials, saving users time and energy in searching for learning materials.

[0132] S108: Generate an intelligent analysis report based on the comprehensive diagnosis information, the improvement suggestion information, and the learning and expansion plan recommendation information, and display the intelligent analysis report.

[0133] After the meeting, the online conference application can organize the comprehensive diagnostic information, improvement suggestion information and learning and expansion plan recommendation information into an intelligent analysis report (as an evaluation report). When the user views the intelligent analysis report, the comprehensive diagnostic information, improvement suggestion information and learning and expansion plan recommendation information are displayed.

[0134] For example, Figure 7 As shown, after the meeting, when the user clicks the button H5 for viewing the intelligent analysis report in window 11, window 12 displays a comprehensive diagnosis box M6, an improvement suggestion box M7, and a learning and expansion plan recommendation box M8, wherein the comprehensive diagnosis box M6, the improvement suggestion box M7, and the learning and expansion plan recommendation box M8 include comprehensive diagnosis information S6, improvement suggestion information S7, and learning and expansion plan recommendation information S8, respectively. The learning and expansion plan recommendation information S8 includes "The Protectino of information Computer Systems" TM 、Security Design Guide_Security DesignGuide_V1.2_Chinese and English Version TM 、Guide to Threat Modeling Methodology TM ”.

[0135] In summary, in an embodiment of the present application, during a meeting, an online conference application can push extended information of conference information to users in real time to help users understand conference information and help understand extended information of conference information. In addition, the online conference application can adjust the push strategy according to the user's interests to increase the push of extended information that the user is interested in and reduce the push of extended information that the user is not interested in, thereby enhancing the user's interest in use. After the meeting, the online conference application can generate an intelligent analysis report, which includes comprehensive diagnostic information, improvement suggestion information, and learning and expansion plan recommendation information for the user. This information can help users understand their performance during the meeting and provide users with relevant improvement suggestions and learning plans. In this way, the online conference application can provide users with some value-added services after the meeting to enhance the user's experience.

[0136] The online conference application provided in the embodiment of the present application includes at least the following four functions:

[0137] Function 1: During a meeting, push extended information based on keywords and / or key images.

[0138] Function 2: During the meeting, push extended information based on user interaction behavior.

[0139] Function 3: During the meeting, adjust the push strategy of expanded information based on interest feedback.

[0140] Function 4: Generate an intelligent analysis report after the meeting.

[0141] The following mainly introduces the function 1 of the online conference application using keywords as an example.

[0142] Figure 8 This is an example diagram of a process for an online conference application to push extended information based on keywords and / or key images provided in an embodiment of the present application. Figure 8 , the process may include the following steps:

[0143] S201: Screen content recognition.

[0144] When a user uses an online conference application on an electronic device to hold an online conference with other participants, the online conference application can identify the content displayed on the screen of the electronic device to determine whether a conference shared file is displayed on the screen of the electronic device.

[0145] S202: Text extraction.

[0146] In some embodiments, after the online conference application recognizes that a conference shared file is displayed on the screen of the electronic device, the text in the conference shared file can be extracted to obtain the conference shared file information.

[0147] For example, in Figure 5A In the illustrated scenario, the online conference application can extract text from the conference shared file displayed in the window 11 as conference information based on recognizing that a conference shared file is displayed in the window 11 on the screen.

[0148] In other embodiments, pictures, links, etc. in the conference shared files may also be extracted as conference information.

[0149] S203: Conference speech recognition.

[0150] During a meeting, electronic devices can record the audio in the meeting scene in real time, and then filter out the noise in the audio to obtain the conference voice.

[0151] S204: voice to text conversion.

[0152] After acquiring the conference audio, the online conference application can use ASR technology to convert the conference audio into corresponding text as conference information.

[0153] S205: Generate prompt-1.

[0154] In some embodiments, after the online conference application obtains the conference information, the conference information may be added to the template of prompt-1 to obtain prompt-1.

[0155] For example, the meeting information includes "There are two main purposes of authority separation: 1. Preventing conflicts of interest and abuse of individual rights. 2. Breaking down an important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ..."

[0156] The template for prompt-1 is "……, what are the keywords of the above information? What field does it belong to?".

[0157] prompt-1 can be "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Break down a certain important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ..." and other information. After adding the above meeting information to the prompt text template, the resulting prompt text is "There are two main purposes of authority separation: 1. Prevent conflicts of interest and abuse of individual rights. 2. Break down a certain important authority into multiple authorities, making it more difficult for the object to be protected to be illegally obtained, and thus more secure. ..., what are the keywords of the above information? What field does it belong to?".

[0158] S206: Input to the intelligent model.

[0159] After the online conference application generates prompt-1, prompt-1 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-1.

[0160] S207: Analyze the answer.

[0161] After prompt-1 is input into the intelligent model, the intelligent model can parse the question in prompt-1, and then generate and output the corresponding answer after the parsing is completed.

[0162] For example, the answer output by the intelligent model based on prompt-1 is "Keywords: permission separation, object protection; Field: security and privacy protection."

[0163] S208: Get the “field”.

[0164] After the intelligent model outputs the answer corresponding to prompt-1, the domain information in the answer (for example, "domain: security and privacy protection") can be extracted and stored in the database of the electronic device for subsequent use.

[0165] S209: Obtain “keywords”.

[0166] After the intelligent model outputs the answer corresponding to prompt-1, the key words in the answer (for example, "privilege separation" and "object protection") can be extracted and stored in the data of the electronic device for subsequent use.

[0167] S210: Prompt-2 is generated.

[0168] The fields and keywords stored in the database of the electronic device are obtained, and the fields and keywords are filled into the template of prompt-2 to obtain prompt-2.

[0169] For example, the field is “security and privacy protection” and the keywords are “privilege separation, object protection”.

[0170] The template for prompt-2 is "In the field of xx, for xxx, please show me other relevant information in the form of entries based on important concepts, key technologies, hot events, etc."

[0171] Prompt-2 could be "In the field of security and privacy protection, regarding permission separation and object protection, please show me other relevant information in the form of entries based on important concepts, key technologies, hot events, etc."

[0172] S211: Input to the intelligent model.

[0173] After the online conference application generates prompt-2, prompt-2 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-2.

[0174] S212: Analyze the answer.

[0175] After prompt-2 is input into the intelligent model, the intelligent model can parse the question in prompt-2, and then generate and output the corresponding answer after the parsing is completed.

[0176] For example, the answer output by the intelligent model based on prompt-2 is "1. Three-plane isolation means that the management plane, control plane, and user plane are securely isolated from each other. 2. Open principle (Shannon's motto: the enemy knows you better). 3. The open principle means that devices should not be secrets, security should not rely on the confidentiality of mechanism design and implementation and the ignorance of attackers, and the design of the product should be fully tested."

[0177] S213: Obtain “extended information”.

[0178] After the intelligent model outputs the answer corresponding to prompt-2, the extended information in the answer can be extracted and stored in the database of the electronic device for subsequent use.

[0179] S214: The source is set to automatic recommendation.

[0180] The online conference application has a conference intelligent service function. Users can set up the conference intelligent service of the online conference application accordingly, so that the online conference application can automatically recommend extended information related to the conference information to the user during the conference. For example, when the user sets the source of extended information to automatic recommendation in the conference intelligent service function, the online conference application can automatically recommend corresponding extended information to the user based on the conference information.

[0181] S215: Displaying extended information.

[0182] After the online conference application obtains the extended information of the keyword, it displays the extended information.

[0183] Exemplarily, as shown in FIG5 , after the online conference application obtains the extended information of the keywords “permission separation, object protection”: information S1, information S2, and information S3, it can fill information S1, information S2, and information S3 into smart prompt boxes M1, smart prompt boxes M2, and smart prompt boxes M3 for display respectively.

[0184] S216: Extracting the speech of the local user.

[0185] In some embodiments, the online conference application can extract the voice from the local microphone as the local user's speech voice, and convert the local user's speech voice into corresponding text.

[0186] S217: Generate prompt-3.

[0187] After the online conference application converts the local user's speech into corresponding text, the text can be added to the template of prompt-3 to obtain prompt-3. For example, prompt-3 can be "..., the above is the user's speech information, please identify which information is the user's question and which information is the user's answer."

[0188] S218: Input to the intelligent model.

[0189] After the online conference application generates prompt-3, prompt-3 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-3.

[0190] S219: Analyze the answer.

[0191] After prompt-3 is input into the intelligent model, the intelligent model can parse the question in prompt-3, and then generate and output the corresponding answer after the parsing is completed.

[0192] For example, the answer output by the intelligent model based on prompt-3 may be “User question: 1, …; 2, …; User answer: 1, …; 2, …”.

[0193] S220: Obtain user questions.

[0194] After the intelligent model outputs the answer corresponding to prompt-3, the user question in the answer is extracted and stored in the database of the electronic device for subsequent use.

[0195] S221: Obtain user answers.

[0196] After the intelligent model outputs the answer corresponding to prompt-3, the user's answer is extracted from the answer and stored in the database of the electronic device for subsequent use.

[0197] After the meeting, the online conference application can obtain user questions and user answers from the database of the electronic device, and then generate an intelligent analysis report based on the user questions and user answers. In order to ensure the coherence of the narrative, the process of the intelligent analysis report is explained below.

[0198] In this embodiment, during the meeting, the online meeting application can identify keywords in the meeting information in real time, and push extended information related to the keywords based on the field to which the meeting belongs. In this way, more effective information can be provided to users, helping them understand the meeting information and helping them gain a deeper understanding of the extended information related to the meeting information. In addition, the user's search for meeting information can be reduced, saving the user's time and energy.

[0199] During the meeting, the online conference application can not only automatically push extended information based on the keywords and / or key images of the conference information, but also push extended information based on the user's interactive behavior. For example, when the user wants to know certain information in the conference information, he can select the information, and then the online conference application can generate and display the extended information of the information based on the information selected by the user. The following is an introduction to the second function of the online conference application.

[0200] Fig. 9 This is an example diagram of a process for an online conference application to push extended information based on user interaction behavior provided in an embodiment of the present application. Fig. 9 , the process includes the following steps:

[0201] S301: Identify the circled content on the screen.

[0202] When a user uses an online conference application on an electronic device to hold an online conference with other participants, if the user is interested in certain information displayed on the screen of the electronic device, the user can circle the information. The online conference application then identifies the information, generates extended information of the information and displays it. In this way, the online conference application can push corresponding extended information to the user based on the user's interaction.

[0203] For example, Figure 6 As shown, when the user circles the picture P1 on the screen of the electronic device 100, the online conference application can recognize the picture P1 selected by the user, and then display information S4 related to the picture P1 in the smart prompt box M4 of the window 12. When the user circles the text "key" on the screen of the electronic device 100, the online conference application can recognize the text "key" selected by the user, and then display information S5 related to the text "key" in the smart prompt box M5 of the window 12.

[0204] S302: Determine whether it is text.

[0205] If not, execute step S303. If yes, execute step S304.

[0206] In some embodiments, OCR technology may be used to identify whether the information circled by the user is text. If not, the process proceeds to step S303 ; if so, the process proceeds to step S304 .

[0207] S303: Save as a picture.

[0208] When the online conference application recognizes that the information circled by the user on the screen of the electronic device is not text, the information can be saved as a picture.

[0209] S304: Obtain the “field” from the database.

[0210] The database of the electronic device stores information about the field to which the current meeting belongs. The online conference application can directly obtain information about the field to which the current meeting belongs from the data of the electronic device, and then determine the field to which the meeting belongs based on the information, for example, the field of security and privacy.

[0211] S305: Prompt-4 is generated.

[0212] The online conference application can add the field to which the current conference belongs and the information circled by the user on the screen to the template of prompt-4 to obtain prompt-4.

[0213] For example, the template of prompt-4 is "Please explain ..." from the perspective of the field of ..." The field of the current meeting is "security and privacy field". When the information circled by the user is picture P1, prompt-4 can be "Please explain picture P1 from the perspective of the field of security and privacy". When the information circled by the user is the word "key", prompt-4 can be "Please explain the key from the perspective of the field of security and privacy".

[0214] S306: Input to the intelligent model.

[0215] After the online conference application generates prompt-4, prompt-4 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-4.

[0216] S307: Analyze the answer.

[0217] After prompt-4 is input into the intelligent model, the intelligent model can parse the question in prompt-4, and then generate and output the corresponding answer after the parsing is completed.

[0218] For example, when the information circled by the user is picture P1, the answer output by the intelligent model may be "Aristotle proposed the famous three-element theory..."

[0219] For another example, when the information circled by the user is the word "key", the answer output by the intelligent model may be "key, a key is a password or code used to encrypt and decrypt data. It can be a string, a number, or a complex combination of numbers and letters."

[0220] S308: Obtain “extended information”.

[0221] The online conference application obtains the answers output by the intelligent model as extended information of the user's circled information.

[0222] S309: The source is set to automatic recommendation.

[0223] The principle of step S309 is substantially the same as that of the above step S214, and will not be repeated here.

[0224] S310: Displaying extended information.

[0225] After the online conference application obtains the extended information of the information circled by the user, it displays the extended information.

[0226] For example, Figure 6As shown, after the online conference application obtains the extended information S4 of the picture P1 and the extended information S5 of the word "key", it fills the information S4 and the information S5 into the smart prompt box M4 and the smart prompt box M5 for display respectively.

[0227] In this embodiment, during the meeting, the user can circle the information he wants to know on the screen of the electronic device. After the user circles, the online conference application can push extended information related to the user's circled information to the user, so as to help the user to understand the information he is concerned about in a timely manner, which is convenient and fast.

[0228] During the meeting, the online conference application can also adjust the push strategy of extended information according to the user's interests. The following is an introduction to function three of the online conference application.

[0229] Fig.10 This is an example flow chart of an online conference application providing an embodiment of the present application, which adjusts a push strategy for extended information based on interest feedback.

[0230] refer to Fig.10 , the process includes the following steps:

[0231] S401: Interest feedback.

[0232] In the above Figure 5A , Figure 5B and Figure 6 In the scenario shown, when the user is interested in a certain extended information, the user can click the interested button in the smart prompt box where the extended information is located to feedback the information that the user is interested in the extended information to the online conference application. When the user is not interested in a certain extended information, the user can click the uninterested button in the smart prompt box where the extended information is located to feedback the information that the user is not interested in the extended information to the online conference application.

[0233] The user's interest feedback information can be recorded in a database of the electronic device. During the meeting, the online conference application can query the interest feedback information of each extended information recorded in the database at any time to adjust the interest value of the corresponding information according to the interest feedback information.

[0234] S402: Query database records.

[0235] In some embodiments, the interest value of each information is recorded in the database of the electronic device, and its default value is 0. When the user clicks the interest button of a certain extended information, the interest value of the extended information can be updated to 1, indicating that the user is interested in the extended information. When the user clicks the not interested button of a certain extended information, the interest value of the extended information is updated to 2, indicating that the user is not interested in the extended information. In this embodiment, the setting method of the interest value is only used as an exemplary description and is not a limitation of this application. In other embodiments, other methods can also be used to set the interest value. For example, for the extended information that the user is interested in, its interest value can be set to a logical value of true; for the extended information that the user is not interested in, its interest value can be set to a logical false.

[0236] After the online conference application receives the user's interest feedback information, that is, after the user clicks a button corresponding to a certain extended information, the online conference application queries the interest value of the extended information in the database, and updates the interest value of the extended information when the query result does not correspond to the user's feedback.

[0237] For example, Fig.11 As shown, when the user clicks the interest button L3 in the smart prompt box M2, if the interest value of information S2 in the database is 0 or 2, the interest value of information S2 in the database can be updated to 1. If the interest value of information S2 in the database is 1, the interest value of information S2 in the database does not need to be updated.

[0238] S403: Feedback whether you are interested.

[0239] If yes, then go to step S404. If no, then go to step S409.

[0240] The online conference application determines whether the user is interested according to the button clicked by the user. If the user clicks the interested button, it is determined that the user is interested. If the user clicks the not interested button, it is determined that the user is not interested.

[0241] S404: The interest value is set to 1 (the default value is 0).

[0242] When the user clicks an interest button corresponding to a certain extended information, and the interest value of the extended information in the database is 0 or 2, the online conference application sets the interest value of the extended information to 1.

[0243] For example, Fig.11 As shown, when the user clicks the interest button L3 in the smart prompt box M2, and the interest value of information S2 in the database is 0, the online conference application sets the interest value of information S2 in the smart prompt box M2 to 1, indicating that the user is interested in information S2.

[0244] S405: Prompt-5 is generated.

[0245] After the online conference application sets the interest value of the extended information that the user is interested in to 1, the information can be added to the template of prompt-5 to obtain prompt-5.

[0246] For example, the information that the user is interested in is information S2, and its specific information is "Open Principle (Shannon's motto - your enemies know you better)". The template of prompt-5 is "..., I want to know more about this information". After adding information S2 to the template of prompt-5, prompt-5 can be obtained: "Open Principle (Shannon's motto - your enemies know you better), I want to know more about this information".

[0247] S406: Input to the intelligent model.

[0248] After the online conference application generates prompt-5, prompt-5 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-5.

[0249] S407: Analyze the answer.

[0250] After prompt-5 is input into the intelligent model, the intelligent model can parse the question in prompt-5, and then generate and output the corresponding answer after the parsing is completed.

[0251] For example, the answer output by the intelligent model based on prompt-5 is "The open principle refers to adhering to the principles of transparency, verifiability and trustworthiness in information processing, data sharing, system design and technology implementation."

[0252] S408: Displaying extended information.

[0253] After the intelligent model outputs the answer to prompt-5, the online conference application can display the information in the answer as extended information. Fig.11 As shown, the extended information is information S6, and information S6 is displayed in the smart prompt box M6.

[0254] S409: The interest value is set to 2 (the default value is 0).

[0255] When the user clicks a "not interested" button corresponding to a certain extended information, and the interest value of the extended information in the database is 0 or 1, the online conference application sets the extended information to 2, indicating that the user is not interested in the extended information.

[0256] For example, Fig.11As shown, when the user clicks the not interested button L4 in the smart prompt box M1 and the interest value of information S1 in the database is 0, the online conference application sets the interest value of information S1 in the smart prompt box M1 to 2, indicating that the user is not interested in information S1.

[0257] S410: Prompt-6 is generated.

[0258] After the online conference application sets the interest value of the information that the user is not interested in to 2, the information that the user is not interested in can be added to the template of prompt-6 to obtain prompt-6.

[0259] For example, the information that the user is not interested in is information S1, and its specific information is "Three-plane isolation means that the management plane, control plane, and user are securely isolated from each other." The template of prompt-6 is "..., please remember not to introduce me to any more information related to this information." After adding information S1 to the template of prompt-6, prompt-6 can be obtained: "Three-plane isolation means that the management plane, control plane, and user are securely isolated from each other, please remember not to introduce me to any more information related to this information."

[0260] S411: Input to the intelligent model.

[0261] After the online conference application generates prompt-6, prompt-6 can be input into the intelligent model, and the intelligent model analyzes and processes prompt-6.

[0262] S412: Analyze the answer.

[0263] After prompt-6 is input into the intelligent model, the intelligent model can parse the question in prompt-6, and then generate and output the corresponding answer after the parsing is completed.

[0264] For example, the answer output by the intelligent model based on prompt-6 is "OK, I will try my best to avoid introducing this part of relevant information to you."

[0265] After the intelligent model outputs the answer, the online conference application obtains the answer and, in response to the answer, no longer pushes extended information that the user is not interested in to the user.

[0266] In this embodiment, during the meeting, the online conference application can determine the information that the user is interested in and the information that the user is not interested in based on the user's interactive behavior, and then adjust the information recommendation strategy based on the user's interests, such as increasing the recommendation of information that the user is interested in and reducing the recommendation of information that the user is not interested in, thereby improving the user experience.

[0267] After the meeting, the online conference application can generate an intelligent conference report for the user to provide value-added services to the user. The following is an introduction to function 4 of the online conference application.

[0268] Fig.12 This is a flowchart of the process of generating an intelligent analysis report provided in an embodiment of the present application. Fig.12 , the process includes the following steps:

[0269] S501: Intelligent analysis.

[0270] After the meeting, the online conference application starts the intelligent analysis function to analyze the user's meeting performance in the current meeting and generate an intelligent analysis report.

[0271] S502: Read the database.

[0272] The online conference application starts the intelligent analysis function and reads the database of the electronic device to obtain the information required to generate the intelligent analysis report.

[0273] S503: Obtain fields, keywords, and extended information of interest.

[0274] The online conference application reads the information recorded in the database of the electronic device to determine the field to which the current conference belongs, the keywords in the conference information, and the extended information of interest to the user.

[0275] S504: Obtain user questions and user answers.

[0276] The online conference application reads information stored in a database of the electronic device to obtain user questions and user answers during the conference.

[0277] S505: Generate a comprehensive diagnosis prompt.

[0278] After the online conference application obtains the domain, keyword, extended information of interest, user question and user answer, the domain, keyword, extended information of interest, user question and user answer can be added to the template of the comprehensive diagnosis prompt to obtain the comprehensive diagnosis prompt.

[0279] For example, a comprehensive diagnosis prompt may be “In the field of security and privacy, I have...questions, I am interested in..., I understand..., please evaluate my strengths and weaknesses based on the above information.”

[0280] S506: Input to the intelligent model.

[0281] After the online conference application generates a comprehensive diagnosis prompt, the comprehensive diagnosis prompt can be input into the intelligent model, and the intelligent model analyzes and processes the comprehensive diagnosis prompt.

[0282] S507: Analyze the answer.

[0283] After the comprehensive diagnosis prompt is input into the intelligent model, the intelligent model can parse the question in the prompt, and then generate and output the corresponding answer after the parsing is completed.

[0284] For example, the answer output by the intelligent model may be "In the field of security, you have a very good understanding of the concept and significance of security architecture design, and are interested in the design principles and cases of security architecture, but you lack professionalism in the specific implementation methods of security design."

[0285] S508: Obtain comprehensive diagnostic information.

[0286] After the intelligent model outputs the answer corresponding to the comprehensive diagnosis prompt, the online conference application obtains the answer as comprehensive diagnosis information.

[0287] S509: Generate an improvement suggestion prompt.

[0288] After the online conference application obtains the comprehensive diagnostic information, it generates an improvement suggestion prompt based on the comprehensive diagnostic information.

[0289] In some embodiments, the improvement suggestion prompt may be pre-set, for example, the improvement suggestion prompt may be "Are there any suggestions for improvement regarding my shortcomings?"

[0290] S510: Input to the intelligent model.

[0291] After the online conference application generates the improvement suggestion prompt, the improvement suggestion prompt may be input into the intelligent model, and the intelligent model analyzes and processes the improvement suggestion prompt.

[0292] S511: Analyze the answer.

[0293] After the improvement suggestion prompt is input into the intelligent model, the intelligent model can parse the question in the improvement suggestion prompt, and then generate and output the corresponding answer after the parsing is completed.

[0294] For example, the answer output by the intelligent model may be "You can combine security threat analysis with security architecture design, establish a complete security design pattern library and security design solution library, and continuously absorb excellent design cases in the industry."

[0295] S512: Obtain improvement suggestion information.

[0296] After the intelligent model outputs the answer corresponding to the improvement suggestion prompt, the online conference application obtains the answer as the "improvement suggestion" information.

[0297] S513: Generate a learning and expansion plan recommendation prompt.

[0298] The online conference application obtains improvement suggestion information and can generate a learning and expansion plan recommendation prompt based on the improvement suggestion information.

[0299] For example, the prompt for learning and expansion plan recommendations could be “How should I implement the improvement suggestions? Please make a learning plan for me, such as books, cases, papers, etc.”

[0300] S514: Input to the intelligent model.

[0301] After the online conference application generates the learning and expansion plan recommendation prompt, the learning and expansion plan recommendation prompt can be input into the intelligent model, and the intelligent model analyzes and processes the learning and expansion plan recommendation prompt.

[0302] S515: Analyze the answer.

[0303] After the learning and expansion plan recommendation prompt is input into the intelligent model, the intelligent model can parse the question in the learning and expansion plan recommendation prompt, and then generate and output the corresponding answer after the parsing is completed.

[0304] For example, the answer output by the intelligent model may be “Books: …; Cases: …; Papers: …”.

[0305] S516: Obtain learning and expansion plan recommendation information.

[0306] After the intelligent model outputs the answer corresponding to the learning and expansion plan recommendation prompt, the online conference application obtains the answer as the learning and expansion plan recommendation information.

[0307] S517: Display integration information.

[0308] After obtaining the comprehensive diagnosis information, improvement suggestion information, and learning and expansion plan recommendation information, the online conference application can integrate the information to obtain integrated information, that is, the intelligent analysis report. After the conference is over, when the user views the intelligent analysis report, the intelligent analysis report is displayed.

[0309] For example, Figure 7As shown, after the meeting, when the user clicks the button H5 to view the smart analysis report, the electronic device 100 opens the smart analysis report and displays the comprehensive diagnostic information S6, improvement suggestion information S7 and learning and expansion plan recommendation information S8 in the smart prompt box M6, smart prompt box M7 and smart prompt box M8 of window 12 respectively.

[0310] In this embodiment, after the meeting, the online conference application can provide users with an intelligent analysis report, that is, provide users with post-meeting value-added services. The intelligent analysis report includes comprehensive diagnostic information, improvement suggestion information, and learning and expansion plan recommendation information, which can help users understand their behavior in the meeting and help users improve their personal abilities in a targeted manner.

[0311] In some embodiments, the online conference application may include a content analysis module, a text synthesis module, an intelligent model analysis module, and a result processing module. Among them, the information analysis module includes a full-screen recognition module and an audio recognition module. The full-screen recognition module is used to perform text and / or image recognition on the conference shared file displayed on the screen of the electronic device to obtain the conference shared file information as the conference information. The audio recognition module is used to perform voice recognition on the audio signal collected by the electronic device to obtain the conference voice information as the conference information.

[0312] The text synthesis module is used to fill the text in the meeting information into the preset prompt template, obtain the corresponding prompt, and output the prompt.

[0313] The intelligent model analysis module is used to generate extended information related to the meeting information based on the prompt output by the text synthesis module.

[0314] The result processing module is used to organize and display the extended information of the conference information.

[0315] The following is an introduction to the first function of the online conference application in combination with the module: pushing extended information based on keywords and / or key images during the meeting.

[0316] Fig.13 Example of module interaction of online conference application provided in the embodiment of the present application Figure 1 .

[0317] like Fig.13 As shown, first, the content analysis module uses a screen recognition method (such as OCR technology, image recognition technology, etc.) to recognize the full screen of the electronic device to obtain the conference shared file information as the conference information. And the audio in the meeting is voice recognized to obtain the conference voice information as the conference information. After the information analysis module obtains the conference information, it outputs the conference information to the text synthesis module.

[0318] Secondly, the text synthesis module performs semantic analysis preprocessing on the meeting information output by the information parsing module to extract keywords and / or key images in the meeting information. For example, keywords can be events, places, nouns, etc. that are frequently discussed in the meeting. After the text synthesis module identifies the keywords and / or key images in the meeting information, it converts the keywords and / or key images into input information for the intelligent model (for example, prompts about keywords) and outputs them, for example, "In the field of security and privacy, regarding authority separation and object protection, please show me other relevant information in the form of entries in combination with important concepts, key technologies, hot events, etc."

[0319] Next, the intelligent model analysis module inputs the information output by the text synthesis module into the intelligent model, and the intelligent model outputs the extended information of the meeting information based on the information, for example, "1. Three-sided isolation refers to the security isolation of the management plane, control plane, and users. 2. The principle of openness (Shannon's motto - the enemy knows you better). 3. The principle of openness means that devices should not be secrets, and security should not rely on the confidentiality of mechanism design and implementation and the ignorance of attackers. The design of the product should be fully tested."

[0320] Finally, the result processing module organizes the information output by the intelligent model analysis module and outputs it for display. Figure 5A As shown, information S1, information S2 and information S3 output by the intelligent model analysis module are respectively displayed in the intelligent prompt box M1, the intelligent prompt box M2 and the intelligent prompt box M3.

[0321] Therefore, during the meeting, extended information of the meeting can be pushed to users in real time, and this information is not a summary or overview of the meeting information, nor is it information obtained by directly searching the meeting information or a literal interpretation of the meeting information. Instead, it is information that is enhanced, extended, and expanded by combining the meeting context, social cases, and other information.

[0322] Based on the above embodiments, in some embodiments, the online conference application further includes an effectiveness analysis module. The effectiveness analysis module is used to adjust the recommendation strategy of the extended information according to user interest feedback.

[0323] The following is an introduction to the second function of the online conference application in combination with the module: pushing extended information based on user interaction behavior during the meeting.

[0324] Fig.14 Example of module interaction of online conference application provided in the embodiment of the present application Figure 2 .

[0325] like Fig.14As shown, first, the effectiveness analysis module analyzes the user's satisfaction based on the user's interest feedback on the recommendation results, and then converts the user's satisfaction into guidance opinions for the intelligent model analysis module (for example, prompt-5 or prompt-6), so that the intelligent model can timely adjust and optimize the pushed extended information according to the guidance opinions. When the user clicks the "not interested button" or the "interested button", the effectiveness analysis module can timely adjust and optimize the push strategy of the extended information based on the user's click situation. For example, for information that the user is interested in, strengthen the push and provide more detailed and rich extended information. For information that the user is not interested in, reduce the push.

[0326] For example, Fig.11 As shown, when the user clicks the interest button L3 in the smart prompt box M2, it indicates that the user is interested in information S2. At this time, the validity analysis module generates and outputs prompt-5: "Open principle (Shannon's motto - the enemy knows you better), I want to know more about this information". Prompt-5 can guide the intelligent model to output more information related to information S2.

[0327] When the user clicks the "not interested" button L4 in the smart prompt box M1, it indicates that the user is not interested in the information S1. At this time, the validity analysis module generates and outputs prompt-6: "Three-plane isolation means that the management plane, control plane, and user are securely isolated from each other. Please remember not to introduce me to any more information related to this information." Prompt-6 can instruct the intelligent model to no longer generate information related to information S1.

[0328] Secondly, the intelligent model analysis module inputs the information output by the validity analysis module into the intelligent model for analysis and processing, and outputs the analysis and processing results. For example, when the intelligent model analysis module inputs prompt-5 into the intelligent model, the information output by the intelligent model is "The open principle refers to the principles of transparency, verifiability and trustworthiness in information processing, data sharing, system design and technical implementation", which is information S6. When the intelligent model analysis module inputs prompt-6 into the intelligent model, the information output by the intelligent model is "OK, I will try my best to avoid introducing this part of relevant information to you."

[0329] Finally, the result processing module organizes the information output by the intelligent model analysis module and displays the extended information of the information that the user is interested in, for example Fig.11 As shown, information S6 is displayed in the smart prompt box M6. After information S6 is displayed in the smart prompt box M6, the user can provide interest feedback to the effectiveness analysis module by clicking the interest button or the uninterest button of the smart prompt box M6. After receiving the user's interest feedback, the effectiveness analysis module can continue to adjust the information push strategy.

[0330] Therefore, during the meeting, the online conference application can timely adjust the push strategy of expanded information based on the user's interest feedback, for example, increase the push of information that the user is interested in and reduce the push of information that the user is not interested in, thereby providing users with more effective information and improving the user experience.

[0331] Based on the above embodiments, in some embodiments, the online conference application further includes an input processing module. The input processing module includes an interactive analysis module and a local recognition module. The interactive analysis module is used to analyze the user's interactive operation, for example, the user uses a finger or a stylus to circle the screen of the electronic device. The local recognition module is used to extract the information selected by the user using the local recognition method when the user performs a circle operation on the screen of the electronic device.

[0332] The following is an introduction to the third function of the online conference application in combination with the module: adjusting the push strategy of extended information based on interest feedback during the meeting.

[0333] Fig.15 Example of module interaction of online conference application provided in the embodiment of the present application Figure 3 .

[0334] like Fig.15 As shown, first, the input processing module identifies the information circled by the user on the screen of the electronic device. Figure 6 As shown, when the user circles the text "key", the input processing module can recognize that the user has selected the word "key".

[0335] Secondly, the text synthesis module performs semantic analysis preprocessing (such as filtering, classification, synthesis, etc.) on the recognition results of the input processing module to convert the recognition results into input information of the intelligent model, and then outputs the information. Figure 6 As shown, when the user circles the word "key" on the screen of the electronic device 100, the text synthesis module can recognize the word "key" and output prompt-4: "Please explain the key from the perspective of security and privacy."

[0336] Next, the intelligent model analysis module inputs the information output by the text synthesis module into the intelligent model for analysis and processing, and outputs the analysis and processing results. For example, when the intelligent model analysis module inputs prompt-4 into the intelligent model, the information output by the intelligent model may be information S5, and its specific information is "key, a key is a password or code used to encrypt and decrypt data, which can be a string, a number, or a complex combination of numbers and letters".

[0337] Finally, the result processing module organizes and displays the information output by the intelligent model analysis module. Figure 6 As shown, information S5 is displayed in the smart prompt box M5.

[0338] Therefore, during a meeting, when a user circles certain information on the screen of an electronic device, the online meeting application can directly push extended information of the user's circled information, which is convenient and fast, reduces the user's retrieval of meeting information, and thus saves the user's time and energy.

[0339] In some embodiments, the online conference application further includes an information synthesis module. The information synthesis module is used to record the extended information, user speeches, discussions, interest feedback, query information and other information pushed by the online conference application during the entire conference, and to comprehensively organize this information after the conference.

[0340] The following is an introduction to the fourth function of the online conference application in combination with the module: generating an intelligent analysis report after the meeting.

[0341] Fig.16 Example of module interaction of online conference application provided in the embodiment of the present application Figure 4 .

[0342] like Fig.16 As shown, first, the information integration module determines the field of the current meeting based on the meeting information, and then generates input information and outputs of the intelligent model based on the field of the current meeting, the information of interest to the local user, and the question and answer information, such as the comprehensive diagnosis prompt, the improvement suggestion prompt, and the learning and expansion plan recommendation prompt.

[0343] Secondly, the intelligent model analysis module generates comprehensive diagnosis information S6, improvement suggestion information S7 and learning and expansion plan information S8 according to the comprehensive diagnosis prompt, improvement suggestion prompt and learning and expansion plan recommendation prompt output by the information integration module.

[0344] Finally, the result processing module organizes the comprehensive diagnosis information S6, the improvement suggestion information S7 and the learning and expansion plan information S8 into an intelligent analysis report and displays it on the screen of the electronic device. Figure 7 As shown, after the meeting, when the user clicks the button H5 to view the intelligent analysis report, the comprehensive diagnosis information S6, the improvement suggestion information S7 and the learning and expansion plan information S8 are respectively displayed in the comprehensive diagnosis box M6, the improvement suggestion box M7 and the learning and expansion plan recommendation box M8.

[0345] Therefore, after the meeting, the online conference application can provide users with an intelligent analysis report of the current meeting to provide users with value-added services after the meeting.

[0346] Fig.17 A schematic diagram of the architecture of an online conference application provided in an embodiment of the present application.

[0347] like Fig.17 As shown, the online conference application includes an input unit, an analysis unit and an output unit.

[0348] The input unit includes an online conference unit, a conference voice unit, a conference shared material unit, a key information analysis and synthesis unit, a user interaction unit, a click feedback (point of interest selection) unit, a page circle picture text unit, and an information extraction unit. Among them, the online conference unit is used to collect conference audio and send it to the conference voice unit, and send the information displayed by the online conference application to the conference shared material (file) unit. The conference voice unit is used to perform voice recognition on the conference audio sent by the online conference unit to obtain conference voice information as conference information, and send the conference information to the key information analysis and synthesis unit. The conference shared material unit is used to extract conference shared file information from the information displayed by the online conference application as conference information, and send the conference information to the information analysis and synthesis unit. The key information analysis and synthesis unit is used to identify key information in the conference information, such as keywords, and generate corresponding prompts based on the key information and send them to the intelligent model unit. The user interaction unit is used to receive user interaction operations, such as click operations, circle operations, etc. The click feedback unit is used to feedback the buttons clicked by the user. The page circle picture text unit is used to identify the area circled by the user. The information extraction unit is used to extract the information corresponding to the button clicked by the user, or to extract the information of the area circled by the user, and then generate a corresponding prompt based on the extracted information and send it to the intelligent model unit.

[0349] The analysis unit includes an intelligent model unit, an intelligent analysis unit, and an intelligent evaluation unit. The intelligent model unit is used to analyze and process the prompt output by the key information analysis and synthesis unit and the information extraction unit, and then output the corresponding answer. The intelligent analysis unit is used to extract information from the answer output by the intelligent model unit as extended information, and output the extended information. The intelligent evaluation unit is used to analyze and process the information output by the intelligent model unit and then output an intelligent analysis report, which includes comprehensive diagnostic information, improvement suggestion information, and learning and expansion plan recommendation information.

[0350] Output units include online prompt extension units, comprehensive diagnosis units, improvement suggestion units and learning and extension plan recommendation units.

[0351] The online prompt extension unit is used to receive the extension information output by the intelligent analysis unit and display the extension information. The comprehensive diagnosis unit is used to display the comprehensive diagnosis information on the screen of the electronic device. The improvement suggestion unit is used to display the improvement suggestion information on the screen of the electronic device, and the learning and expansion plan recommendation unit is used to display the learning and expansion plan recommendation information on the screen of the electronic device.

[0352] The technical solution of this application is described below in conjunction with the architecture of an online conference application.

[0353] During the meeting, the online conference unit can collect conference audio and send it to the conference voice unit. Then the conference voice unit performs voice recognition on the conference audio to obtain conference voice information as conference information, and sends the conference information to the key information analysis and synthesis unit. The online conference unit can also extract conference shared file information from the information displayed by the online conference application as conference information, and send the conference information to the conference shared material unit. Then the conference shared material unit extracts conference shared file information from the information displayed by the online conference application as conference information, and sends it to the key information analysis and synthesis unit. After receiving the conference information, the key information analysis and synthesis unit can identify the key information of the conference information (such as keywords, etc.), generate a prompt corresponding to the key information, and then input the prompt into the intelligent model unit. The intelligent model outputs the corresponding answer according to the prompt, and sends the answer to the intelligent analysis unit. The intelligent analysis unit extracts the information in the answer as the extended information of the key information, sends it to the online prompt extension unit, and the online prompt extension unit displays the extended information on the screen of the electronic device.

[0354] In addition, during the meeting, when the user interaction unit receives a click operation from the user, the click feedback unit is notified to identify the button clicked by the user, and the information extraction unit extracts the extended information corresponding to the button clicked by the user, and then generates a corresponding prompt based on the user's interest and sends it to the intelligent model unit. The intelligent model outputs the corresponding answer based on the prompt, and sends the answer to the intelligent analysis unit and the intelligent evaluation unit. The intelligent analysis unit extracts the information in the answer as the extended information of the information of interest to the user, and sends it to the online prompt extension unit, which displays the extended information on the screen of the electronic device.

[0355] When the user interaction unit receives the user's circle operation, the notification click feedback unit identifies the area circled by the user, and the information extraction unit extracts the information in the area, such as pictures, text, etc., and then generates a corresponding prompt based on the information circled by the user and sends it to the intelligent model unit. The intelligent model outputs the corresponding answer according to the prompt and sends the answer to the intelligent analysis unit and the intelligent evaluation unit. The intelligent analysis unit extracts the information in the answer as the extended information of the information circled by the user, and sends it to the online prompt extension unit, which displays the extended information on the screen of the electronic device.

[0356] After the meeting, the intelligent evaluation unit generates comprehensive diagnosis information, improvement suggestion information and learning and expansion plan recommendation information, and sends them to the comprehensive diagnosis unit, improvement suggestion unit and learning and expansion plan recommendation unit respectively, which then display the information respectively.

[0357] Fig.18 1 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0358] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0359] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and a SIM card interface.

[0360] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0361] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0362] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0363] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0364] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0365] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0366] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0367] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0368] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0369] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0370] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0371] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other file information in the external memory card.

[0372] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0373] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0374] The various embodiments disclosed in the present application may be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0375] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application specific integrated circuit, or a microprocessor.

[0376] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0377] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or by other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, a floppy disk, an optical disk, an optical disk, a read-only memory, a magneto-optical disk, a read-only memory, a random access memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier wave, infrared signal digital signal, etc.) using the Internet in an electrical, optical, acoustic or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0378] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0379] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0380] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0381] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A method for generating an evaluation report, applied to an electronic device, characterized in that: The method comprises: Obtain interactive information during the operation of online interactive applications; Based on the interaction information, determining key information in the operation of the online interactive application; Based on the field to which the interactive information belongs and the key information, an evaluation report is obtained.

2. The method according to claim 1, characterized in that Determining key information in the operation of the online interactive application based on the interaction information includes: Extracting first key information from the interaction information; Based on the field to which the interactive information belongs, the first key information is expanded to obtain second key information.

3. The method according to claim 2, characterized in that The step of expanding the first key information based on the field to which the interactive information belongs to obtain the second key information includes: Based on the field to which the interactive information belongs, the first key information is expanded to obtain a plurality of key information to be selected; The plurality of key information to be selected are displayed to the user, and the key information selected by the user from the plurality of key information to be selected is used as the second key information.

4. The method according to claim 2, characterized in that: The extracting the first key information from the interaction information includes: Based on the evaluation model, the first key information is extracted from the interactive information, where the first key information includes text and pictures.

5. The method according to claim 2, characterized in that: The extracting the first key information from the interaction information includes: Acquiring display information selected by a user in an application interface of the online interactive application; The display information is used as the first key information.

6. The method according to claim 1, characterized in that Determining key information in the operation of the online interactive application based on the interaction information includes: Determine the user's question and answer information in the interactive information, and use the question and answer information as the key information, wherein the question and answer information includes the question and answer information in the collected user voice information and the question and answer information displayed on the interface of the online interactive application.

7. The method according to claim 1, characterized in that The obtaining of an evaluation report based on the field to which the interactive information belongs and the key information includes: Inputting the fields to which the key information and the interactive information belong into an evaluation model to obtain evaluation information output by the evaluation model; Based on the evaluation information, the evaluation report is obtained.

8. The method according to claim 4 or 7, characterized in that: The evaluation model includes at least one of the following: Generative pre-trained transformer GPT model, pre-trained bidirectional encoder BERT model, long short-term memory network LSTM model, transformer Transformer model.

9. The method according to claim 1, characterized in that: The online interactive application includes at least one of the following: an online conference application, an online teaching application, an online live broadcast application, and, The interactive information includes at least one of the following: information displayed by the online interactive application, and interactive voice information.

10. The method according to claim 9, characterized in that Corresponding to the online interactive application being the online conference application, the information displayed by the online interactive application includes file information and chat information displayed by the online conference application, and the interactive voice information includes audio data recorded during the operation of the online conference application; Corresponding to the online interactive application being the online teaching application, the information displayed by the online interactive application includes the courseware information and chat information displayed by the online teaching application, and the interactive voice information includes the audio data recorded during the operation of the online teaching application; Corresponding to the online interactive application being the online live broadcast application, the information displayed by the online interactive application includes product information and chat information displayed by the online live broadcast application, and the interactive voice information includes audio data recorded during the operation of the online live broadcast application.

11. The method according to claim 1, characterized in that: The evaluation report includes at least one of the following information: evaluation information, suggestion information, and study plan information.

12. An electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device; The processor, when executing the instructions in the memory, can enable the electronic device to execute the evaluation report generating method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the evaluation report generating method according to any one of claims 1 to 11.

Images