Conference summary generation method and electronic equipment

Abstract

The invention provides a conference summary generation method and electronic equipment, relates to the technical field of terminals, and aims to quickly generate high-quality conference summary and improve the conference summary generation efficiency and the subsequent backtracking efficiency. The method comprises the steps that the electronic equipment receives interaction operation of a user on a current conference display page, and interaction content is displayed; the electronic equipment collects voice data corresponding to the interaction content; the electronic equipment converts the voice data into text data; the electronic device generates a conference summary for the current conference, the conference summary including the interactive content and the textual data. The method and the device are applied to the process of generating the conference summary by the electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for generating meeting minutes and an electronic device. Background Art

[0002] With the development of Internet technology, scenarios of multi-person conversations such as online meetings and offline meetings (hereinafter referred to as meetings) through terminal products (which can also be described as electronic products) are becoming more and more common. Users often communicate in various aspects such as work and study through meetings.

[0003] At present, audio and video files are usually recorded during meetings so that meeting minutes can be generated based on them after the meeting. However, the information in audio and video files is complex and diverse, and includes a lot of invalid colloquial information, which greatly interferes with the extraction of meeting minutes and makes it impossible to accurately extract important information, resulting in low efficiency and accuracy in the generation of meeting minutes. Moreover, when the meeting minutes with low accuracy cannot help users quickly recall the key points of the meeting, the user experience is poor.

[0004] Therefore, how to efficiently and accurately generate meeting minutes and review meeting minutes has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a meeting minutes generation method and electronic device, which can optimize the meeting minutes generation strategy, efficiently and quickly generate high-quality meeting minutes, and improve the efficiency of meeting minutes generation and subsequent backtracking efficiency.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] In the first aspect, the present application provides a method for generating meeting minutes, which is applied to an electronic device. The method includes: the electronic device receives the user's interactive operation on the current meeting display page and displays the interactive content; the electronic device collects voice data corresponding to the interactive content; the electronic device converts the voice data into text data; the electronic device generates the meeting minutes of the current meeting, and the meeting minutes include the interactive content and text data.

[0008] In this way, the electronic device quickly links the interactive content and text data related to the interactive operation based on the user's interactive operation and voice semantic understanding, and then quickly and automatically generates high-quality meeting minutes, thereby improving the efficiency of meeting minutes generation and subsequent backtracking efficiency.

[0009] According to the first aspect, after the electronic device receives the user's interactive operation on the current conference display page and displays the interactive content, the method further includes: the electronic device generates an interactive event. In this way, the meeting minutes generated by the electronic device also include the interactive event.

[0010] In some examples, the electronic device receives a user's interactive operation on the current conference presentation page, displays the interactive content, and generates an interactive event.

[0011] In some examples, the electronic device collects voice data corresponding to the interactive content, including: the electronic device responds to the user's interactive operation on the current conference presentation page, starts recording from the start timestamp of the interactive operation, and ends the recording when it recognizes that the recorded audio does not contain relevant information of the interactive operation (such as interactive content) to obtain the voice data corresponding to the interactive content.

[0012] In this way, when the user triggers an interactive operation, the electronic device starts recording the current audio and recognizes the semantics in the audio. When it recognizes that the recorded audio does not contain relevant information of the interactive operation, it actively ends the recording, and intelligently and finely acquires voice data that is highly related to the interactive content, so as to improve the accuracy of the meeting minutes.

[0013] In some examples, the electronic device converts voice data into text through automatic speech recognition technology, and determines the converted text as text data.

[0014] In other examples, the electronic device converts voice data into text through automatic speech recognition technology, and performs semantic analysis and keyword capture on the converted text to obtain text data.

[0015] In this way, during the meeting, voice data is intelligently recognized and converted into text data. When the meeting minutes are generated, the text data can be directly added to the meeting minutes, which improves the efficiency of retrospection. Moreover, when retrospecting, the text helps users to review the key points of the meeting, without having to view the audio and video files, which can improve the efficiency of retrospection.

[0016] According to the first aspect, or any implementation of the first aspect above, the interactive operation includes a user's annotation operation on the first content, and the current conference display page includes the first content; the interactive content includes a first annotation; the first annotation is used to mark and annotate the first content; and the interactive event includes a first annotation event.

[0017] According to the first aspect, or any implementation method of the first aspect above, the electronic device receives the user's interactive operation on the current conference display page, displays the interactive content, and generates an interactive event, including: the electronic device receives the user's annotation operation on the first content in the current conference display page, displays the first annotation, and generates a first annotation event.

[0018] According to the first aspect, or any implementation of the first aspect above, the first annotation event includes one or more of the following: first content, first annotation, timestamp of the first annotation, location information of the first annotation, and user information triggering the first annotation.

[0019] In some examples, the annotation operation includes a preset operation for a user to annotate the currently displayed content in the current conference presentation page. The preset operation may be a quick operation for a user to annotate the currently displayed content of the electronic device. The preset operation may also be a voice command of the user.

[0020] In some examples, the first annotation includes one or more of a selective annotation, a directional annotation, and a supplementary annotation. A user can make multiple forms of annotations on the same content.

[0021] Specifically, the electronic device detects the user touching the content currently displayed by the electronic device through an input device or a finger. If it is detected that the time length of the contact position staying on the first content exceeds a time threshold, or it is detected that the contact position continuously moves around the first content to continuously form ink marks, the electronic device determines that the user has performed an annotation operation on the first content. Subsequently, the electronic device responds to the user's annotation operation on the first content, identifies the ink marks and the first annotation corresponding to the annotation operation, displays the first annotation, and generates a first annotation event according to the first annotation.

[0022] In some examples, the electronic device determines user information corresponding to each annotation operation through an input device, thereby generating a first annotation event.

[0023] In some examples, the electronic device may also collect user images to determine user information corresponding to each annotation, thereby generating a first annotation event.

[0024] In some examples, the electronic device may also collect user voice to determine user information corresponding to each annotation, thereby generating a first annotation event.

[0025] In some examples, the electronic device distinguishes and displays first annotations corresponding to different user information.

[0026] In this way, the electronic device can accurately identify the annotation information corresponding to each annotation operation, as well as the user information that triggers the annotation operation, and accurately determine the annotations and annotation events made by each user in the meeting, so as to subsequently generate accurate and rich meeting minutes.

[0027] According to the first aspect, or any implementation of the first aspect above, before the electronic device receives the user's annotation operation on the first content in the current conference display page, displays the first annotation, and generates a first annotation event, the method also includes: the electronic device obtains the conference file; the conference file is the file to be displayed for the current conference; the electronic device preprocesses the conference file and divides the conference file into different modules; the first annotation event also includes the module information where the first annotation is located.

[0028] In some examples, before the meeting starts, the electronic device receives a meeting file transmitted by the user through a wired connection or a wireless connection. The electronic device pre-processes the meeting file, divides the meeting file into different modules, and determines module information corresponding to each module.

[0029] In some examples, preprocessing includes layout recognition, semantic analysis, module segmentation and numbering, keyword capture, etc. Module information includes module number, module location information, and module content summary (such as keywords).

[0030] In this way, the electronic device can perform fine segmentation and semantic analysis on the user's meeting documents, which can reduce the granularity of automatically generated meeting minutes, so as to improve the accuracy and structure of the meeting minutes and improve the quality of the meeting minutes.

[0031] According to the first aspect, or any implementation method of the first aspect above, the minutes of the current meeting are generated, including: the electronic device establishes an association between the first content, the first annotation, the first annotation event and the text data corresponding to the first annotation, and generates the minutes content corresponding to the first annotation; the electronic device obtains the minutes content corresponding to each module based on the minutes content corresponding to the first annotation in each module; the electronic device obtains the minutes of the current meeting based on the minutes content corresponding to each module.

[0032] In some examples, the electronic device first generates minutes content corresponding to each module according to the minutes content corresponding to each first annotation in each module, and then generates minutes of the current meeting according to the minutes content corresponding to each module in the meeting document.

[0033] In this way, the minutes corresponding to each first mark in each module are integrated in chronological order to obtain the minutes corresponding to each module. Finally, the minutes corresponding to each module are integrated to obtain the minutes of the current meeting. This is equivalent to layering and structuring the meeting documents, and then obtaining structured meeting minutes so that users can better understand the key points of the meeting.

[0034] According to the first aspect, or any implementation method of the first aspect above, after generating the minutes of the current meeting, the method also includes: the electronic device responds to the user's touch operation on the module where the first content is located, and displays the minutes content corresponding to the module where the first content is located; the electronic device responds to the user's touch operation on the first annotation, and displays the minutes content corresponding to the first annotation.

[0035] In some examples, the electronic device adds meeting minutes to a meeting file; the electronic device receives a user's touch operation on a first module and displays the minutes content corresponding to the first module; the electronic device receives a user's touch operation on a first annotation in the first module and displays the minutes content corresponding to the first annotation.

[0036] Among them, the first module is the module that the user touches when reviewing the meeting.

[0037] In some examples, the electronic device adds the minutes content by overlaying a layer, but the overlay layer is displayed only when the user touches it. Alternatively, the minutes content is connected to the meeting file through a hyperlink.

[0038] In this way, each minute content in the meeting minutes is added to the meeting file in text form. Users can review the meeting minutes in an interactive way, and display the meeting files in layers from the whole to the details during the review process. This review method reduces the density of detailed information without losing key information, and supplements detailed content as needed, thereby improving the user's meeting review efficiency and reducing the user's mental workload.

[0039] According to the first aspect, or any implementation of the first aspect above, the interactive operation includes the user's first blackboard writing operation, the interactive content includes the first blackboard writing, the first blackboard writing is the ink content displayed in the current conference presentation page by the electronic device in response to the user's first blackboard writing operation; the interactive event includes the first blackboard writing event.

[0040] According to the first aspect, or any implementation method of the first aspect above, the electronic device receives the user's interactive operation on the current conference display page, displays the interactive content, and generates an interactive event, including: the electronic device receives the user's first blackboard operation on the current conference display page, displays the first blackboard, and generates a first blackboard event.

[0041] According to the first aspect, or any implementation method of the first aspect above, the first blackboard writing event includes one or more of the following: the first blackboard writing, the timestamp of the first blackboard writing, the location information of the first blackboard writing, the user information triggering the first blackboard writing, the input device type for inputting the first blackboard writing, and the contact type of the first blackboard writing.

[0042] In some examples, the electronic device responds to the user's first blackboard writing operation by displaying the first blackboard writing and generating a first blackboard writing event; the electronic device records the explanation audio within the time period according to the start and end timestamps of the first blackboard writing, and obtains the text data corresponding to the first blackboard writing based on the explanation audio.

[0043] According to the first aspect, or any implementation of the first aspect above, the method further includes: the electronic device receives a second blackboard operation of the user on the current conference presentation page, displays the second blackboard, and generates a second blackboard event.

[0044] According to the first aspect, or any implementation of the first aspect above, the second blackboard writing operation is the next blackboard writing operation after the user performs the first blackboard writing operation.

[0045] According to the first aspect, or any one of the implementation methods of the first aspect above, generating the minutes of the current meeting also includes: the electronic device establishes an association between the first blackboard writing, the first blackboard writing event and the text data corresponding to the first blackboard writing, and generates the meeting minutes corresponding to the first blackboard writing; the electronic device establishes an association between the second blackboard writing, the second blackboard writing event and the text data corresponding to the second blackboard writing, and generates the meeting minutes corresponding to the second blackboard writing; the electronic device obtains the minutes of the current meeting based on the meeting minutes corresponding to the first blackboard writing and the meeting minutes corresponding to the second blackboard writing.

[0046] According to the first aspect, or any implementation of the first aspect above, the method also includes: the electronic device determines the first module based on the first blackboard; the first module is the area in the current conference display page that contains the first blackboard; the electronic device determines whether the second blackboard matches the first module; if not, determining the second module based on the second blackboard; if matched, determining that the second blackboard is located in the first module.

[0047] In some examples, the electronic device determines whether the second blackboard book matches the first module based on the position of the second blackboard book. If the relative distance between the second blackboard book and the first module exceeds a preset distance, it is determined that the second blackboard book does not match the first module; otherwise, it is determined that the second blackboard book matches the first module.

[0048] In some examples, the electronic device determines whether the second blackboard writing matches the first module based on the time difference between the first blackboard writing operation and the second blackboard writing operation. If the time difference between the first blackboard writing operation and the second blackboard writing operation exceeds a preset time, it is determined that the second blackboard writing does not match the first module; otherwise, it is determined that the second blackboard writing matches the first module.

[0049] In some examples, the electronic device determines whether the second blackboard book matches the first module based on the keywords of the first blackboard book and the keywords of the second blackboard book. If the keywords of the first blackboard book and the keywords of the second blackboard book match, it is determined that the second blackboard book does not match the first module; otherwise, it is determined that the second blackboard book matches the first module.

[0050] According to the first aspect, or any implementation of the first aspect above, the method also includes: the electronic device obtains the minutes content corresponding to each module based on the minutes content corresponding to each blackboard in each module; the electronic device obtains the minutes of the current meeting based on the minutes content corresponding to each module.

[0051] In this way, during the blackboard writing process, the blackboard writing content is structured and divided and organized into different modules, so that structured meeting minutes can be obtained later, which is conducive to users' clear understanding of the meeting minutes and quick review of the key points of the meeting.

[0052] According to the first aspect, or any implementation of the first aspect above, after generating the minutes of the current meeting, the method further includes: the electronic device displays the minutes content corresponding to the first blackboard in response to the user's touch operation on the first blackboard.

[0053] According to the first aspect, or any implementation of the first aspect above, after generating the minutes of the current meeting, the method also includes: the electronic device responds to the user's touch operation on the module where the first blackboard is located, and displays the minutes content corresponding to the module where the first blackboard is located; the electronic device responds to the user's touch operation on the first blackboard, and displays the minutes content corresponding to the first blackboard.

[0054] In this way, the meeting minutes are generated in a structured manner according to the blackboard content, and the meeting minutes also include the blackboard content and the blackboard explanation content, which improves the accuracy and quality of the meeting minutes. Moreover, the real-time division of modules according to the blackboard content and the division of meeting minutes according to modules are conducive to the subsequent interactive review of meeting minutes. During the review process, the meeting documents are displayed in layers from the whole to the details, which can better assist users in understanding the blackboard content and improve the review efficiency.

[0055] In a second aspect, the present application provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0056] In a third aspect, the present application provides a chip system comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0058] In a fifth aspect, the present application provides a computer program product, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enable the computer to execute the method as described in the first aspect or any one of the embodiments of the first aspect.

[0059] It should be noted that the technical effects brought about by any design in the above-mentioned second to fifth aspects can refer to the technical effects brought about by the corresponding design in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of an existing process of generating meeting minutes provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of a participant making annotations during a meeting provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of a conference system provided in an embodiment of the present application Figure 1 ;

[0063] Figure 4 A schematic diagram of a conference system provided in an embodiment of the present application Figure 2 ;

[0064] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application Figure 1 ;

[0065] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application Figure 2 ;

[0066] Figure 7 A schematic diagram of a method for generating meeting minutes provided in an embodiment of the present application Figure 1 ;

[0067] Figure 8 A schematic diagram of an interface provided in an embodiment of the present application;

[0068] Fig. 9 A schematic diagram of a user annotation interface provided in an embodiment of the present application;

[0069] Fig.10 A schematic diagram of an electronic device for different application scenarios provided by an embodiment of the present application;

[0070] Fig.11 A schematic diagram of a method for electronic device to identify user information provided in an embodiment of the present application Figure 1 ;

[0071] Fig.12 A schematic diagram of a method for electronic device to identify user information provided in an embodiment of the present application Figure 2 ;

[0072] Fig.13A schematic diagram of a method for electronic device to identify user information provided in an embodiment of the present application Figure 3 ;

[0073] Fig.14 A schematic diagram of a method for electronic device to identify user information provided in an embodiment of the present application Figure 4 ;

[0074] Fig.15 A method flow diagram of a review meeting minutes provided in an embodiment of the present application Figure 1 ;

[0075] Fig.16 A schematic diagram of an interface for reviewing meeting minutes provided in an embodiment of the present application;

[0076] Fig.17 A schematic diagram of a user blackboard interface provided in an embodiment of the present application;

[0077] Fig.18 A schematic diagram of a method for generating meeting minutes provided in an embodiment of the present application Figure 2 ;

[0078] Fig.19 A method flow diagram of a review meeting minutes provided in an embodiment of the present application Figure 2 ;

[0079] Fig. 20 A method flow diagram of a review meeting minutes provided in an embodiment of the present application Figure 3 ;

[0080] Fig.21 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application Figure 3 ;

[0081] Fig. 22 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0083] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0084] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more. In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0085] In some examples, such as Figure 1 The figure shows a schematic diagram of the process of generating meeting minutes. The electronic device assigns an identity ID to each participant, records the screen during the meeting to generate a screen recording video, and generates corresponding annotation events based on the annotations of each participant. The annotation event includes the identity ID of the participant who made the annotation, the annotation time, the annotation content, etc. After the meeting, the electronic device generates meeting minutes based on the screen recording video and the annotation event. When looking back, the electronic device determines the corresponding screen recording segment from the screen recording video based on the annotation time in the annotation event selected by the user, so that the user can look back based on the screen recording segment. For example, as Figure 2 As shown, it is a schematic diagram of the participant making annotations during the meeting. The participant draws a single arrow with a stylus on the meeting document displayed on the electronic device, and the electronic device generates a corresponding annotation event based on the annotation, such as the time of the annotation, the participant's identity ID, and the information indicated by the annotation. However, this solution only generates meeting minutes through the participant's annotations and screen recording videos, and the accuracy and efficiency are low. Moreover, when backtracking according to the timeline, it is necessary to find the corresponding timestamp from the entire screen recording video according to the annotation time in the annotation event, and then backtrack based on the screen recording segment indicated by the timestamp, which is inefficient. Therefore, the accuracy and efficiency of the meeting minutes generated by this solution are low.

[0086] Since video files occupy a large amount of memory, in other examples, the electronic device generates corresponding annotation events based on the annotations of the participants, and records the discussion audio when the participants make annotations. If the participant stops the discussion for more than the preset time, the annotation event and the discussion audio are stored in association to form a session record. The electronic device generates meeting minutes based on the session record (annotated events and corresponding audio), and then replays the audio recorded in the session record to trace the meeting points indicated by the annotation. Although the memory usage of the audio file in this scheme is smaller than that of the video file, when the meeting minutes are generated based on the annotations and audio of the participants, when recording audio for a certain annotation, the recording is stopped only based on whether the discussion time exceeds the preset time, which easily leads to the problem of weak correlation between the annotation and the audio, and the problem of mis-segmentation. For example, after the user makes an annotation 1 on content 1, the electronic device records audio 1 in response to the annotation 1, but after the user discusses the content 1 indicated by the annotation 1, he continues to discuss the unannotated content 2. After discussing content 2, the user makes an annotation 2 on content 3, and the electronic device responds to the annotation 2 to end the recording of the audio 1 of the annotation 1 and start recording the audio 2 of the annotation 2. The audio 1 corresponding to the mark 1 in content 1 should only contain the discussion audio of content 1, but audio 1 also includes the discussion audio of content 2. The audio segmentation method based on the length of time when the speaker stops talking is relatively rough and prone to mis-segmentation. The accuracy of the meeting minutes generated based on the roughly segmented audio is also low.

[0087] In another example, when a participant makes an annotation, the electronic device identifies the keywords in the annotation and associates the keywords with the content in the meeting document and stores them; the electronic device generates meeting minutes based on the meeting document and the annotation. In this solution, the meeting minutes are generated based only on the generalized document information (meeting document and annotation content), and do not involve the discussion of the annotations by the participants, which makes it easy to miss details or important information, and the quality and accuracy of the meeting minutes are also poor.

[0088] In another example, an electronic device records audio and video files during a meeting, and generates an interaction event when a participant generates active user interaction (such as discussion, questioning and commenting, etc.), and the interaction event includes an interaction index number, interaction time, interaction content, interaction object, etc. The electronic device generates meeting minutes based on the audio and video files and interaction events. During subsequent retrospection, the electronic device locates relevant clips from the audio and video files based on the interaction events, so that the user can determine the meeting points of the interaction event based on the audio and video clips. This solution generates meeting minutes and retrospects meetings based on the complicated interaction events and audio and video files of the participants, and the readability and accuracy of the meeting minutes are poor. Moreover, when retrospecting, it is necessary to watch the audio and video clips before determining the main points of the meeting, which is inefficient.

[0089] In another example, an electronic device records an audio or video file of at least one participant, and extracts key information (which can be understood as an abstract or summary) from the audio or video file in chronological order. The key information includes keywords, key content, etc. The electronic device associates and stores the key information with the corresponding audio and video clips, and generates meeting minutes based on the audio and video files and meeting documents. In subsequent backtracking, the electronic device determines the key points of the meeting by retrieving the audio and video clips corresponding to the keywords. This solution generates meeting minutes based on audio and video files and meeting documents, and has low efficiency and accuracy.

[0090] In the above examples, most electronic devices generate meeting minutes based on the audio and video files of the meeting. When reviewing the meeting, it is necessary to search for relevant audio and video clips by keywords or determine the relevant audio and video clips by timestamps for review. However, audio and video files occupy a large amount of memory, and the audio and video files are long and contain a lot of colloquial invalid information, resulting in low efficiency in generating meeting minutes, and poor quality and accuracy of meeting minutes.

[0091] In order to solve the above technical problems, the embodiment of the present application provides a method for generating meeting minutes, which quickly links the meeting content indicated by the annotation, the meaning of the annotation, and the text data of the explanation annotation through pen-screen interaction and voice semantic understanding, thereby quickly and automatically generating high-quality meeting minutes. Moreover, when backtracking, the key points of the meeting can be backtracked in an interactive manner to improve the efficiency of backtracking.

[0092] The meeting minutes generation method provided in the embodiment of the present application can be applied to other business scenarios such as meeting scenarios and teaching scenarios. Optionally, the meeting scenario includes a meeting discussion scenario at work or in life. For example, a meeting in an enterprise using large-screen electronic devices to discuss products. Teaching scenarios include teaching blackboards, review and other scenarios. For example, a classroom in a school or educational institution where a teacher uses a smart screen to teach new course content. The embodiment of the present application does not impose any special restrictions on the specific form of the business scenario.

[0093] In the embodiment of the present application, each business scenario includes a meeting minutes generation scenario and a meeting minutes review scenario. For example, during a meeting, a meeting minutes is generated based on relevant information in the meeting. After the meeting, the main points of the meeting are reviewed based on the meeting minutes. Alternatively, during a teaching process, a class summary is generated based on teaching-related information in the classroom. After the class, the key points of the teaching are reviewed based on the class summary.

[0094] In the embodiment of the present application, the electronic device in the meeting minutes generation scenario may be an electronic device capable of displaying, for example, a computer, a mobile phone, a large-screen device, a smart screen, etc.

[0095] In some embodiments, the display screen of the electronic device is an electronic touch screen (such as a capacitive screen or an infrared screen with touch control), and the user can use a finger to slide on the display screen of the electronic device to input text or patterns, etc.

[0096] In other embodiments, the electronic device may also be equipped with an input device (such as a stylus). Exemplarily, the stylus can establish communication with the electronic device via wired or wireless (such as Bluetooth, Wi-Fi). The user can use the stylus to write information such as text or patterns on the display screen of the electronic device. The electronic device can display the motion track formed by the stylus on the display screen. In addition, the electronic device can record the motion track formed by the stylus on the display screen, and then recognize the corresponding text or pattern.

[0097] Optionally, one or more buttons may be set on the stylus, and the buttons may be physical buttons or virtual buttons. In some examples, each button corresponds to a key value, and when a user operates a button, the stylus may send the key value corresponding to the button to the electronic device via a wireless method such as Bluetooth. The electronic device performs a corresponding operation based on the received key value. In other examples, different key values ​​are set for different operations on each button. When the stylus detects different operations of the user on the button, it sends the key value corresponding to the operation to the electronic device, and the electronic device performs a corresponding operation based on the received key value.

[0098] For example, suppose that a key value 1 corresponding to a certain button 1 is pre-configured in the stylus, and the pre-configured key value 1 in the electronic device corresponds to starting the annotation mode. If the stylus detects that the user has performed a single-click operation on the button 1, the stylus can send the key value 1 corresponding to the single-click of the button 1 to the electronic device. After receiving the key value 1, the electronic device starts the annotation mode. Then, the application being displayed by the electronic device enters the handwriting mode, and the user can use the stylus to handwrite content in the display interface in the handwriting mode. For example, in the annotation mode, the electronic device can identify the annotation content handwritten by the user according to the movement trajectory of the stylus in the display interface.

[0099] The stylus pen may be a resistive stylus pen, an electromagnetic stylus pen or a capacitive stylus pen. The stylus pen may also include a passive stylus pen that does not send any signal and does not conduct electricity. The present application embodiment does not impose any limitation on this.

[0100] In other embodiments, the electronic device may also be an electronic device equipped with a microphone array, and the electronic device records the conference audio through the microphone array. In some examples, the voice data corresponds to (or is associated with) the conference members (or user identifications (identities, IDs) (or user accounts)) so that the electronic device can identify the participant information corresponding to the voice data based on the voice data in the audio.

[0101] In some other embodiments, the electronic device may also be an electronic device equipped with a camera, and the electronic device identifies conference members through the camera.

[0102] In an embodiment of the present application, the electronic device in the meeting minutes review scenario may be an electronic device capable of display, such as a computer, a mobile phone, etc.

[0103] The meeting minutes generation method provided in the embodiment of the present application can be applied to various conference systems, including but not limited to meetings conducted in various ways such as conference software and conference room systems.

[0104] Figure 3 FIG. 3 shows a conference system 300 provided in an embodiment of the present application. Figure 3 As shown, the conference system 300 includes multiple electronic devices 301, which can be electronic devices of the same type or different types. Exemplarily, these electronic devices 301 include but are not limited to electronic touch screens, smart screens, mobile phones, tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, netbooks, wearable electronic devices, and vehicle-mounted devices. Electronic whiteboards and other devices, this application does not impose special restrictions on the specific form of the electronic device 301. Each electronic device can communicate directly or indirectly, for example: forwarding communications through a server 302.

[0105] Among them, one or more conference applications (or conference software, conference plug-ins, etc.) are installed on the electronic device 301, and the conference application can realize the function of multi-person conversation. Users can hold a conference through the conference application on the electronic device. In the embodiment of the present application, the conference can be a telephone conference or a network conference, and the network conference includes but is not limited to voice conference, video conference, etc.

[0106] During the meeting, the electronic device 301 can obtain relevant information in the meeting (for example, audio information of meeting members, annotation information of meeting members, and meeting content, etc.), and generate the minutes of the meeting based on the relevant information. Optionally, the electronic device 301 can also send the obtained relevant information of the meeting to the server 302, and the server 302 generates the minutes. Optionally, the server 302 can also send the generated minutes to each electronic device 301.

[0107] After the meeting, the electronic device 301 reviews the main points of the meeting based on the meeting minutes to assist the user in understanding the content of the meeting.

[0108] Figure 4Another conference system 400 provided by an embodiment of the present application is shown. Figure 4 As shown, the conference system 400 includes one or more electronic devices 301 and one or more conference room systems 401. Alternatively, the conference system 400 includes one or more conference room systems 401. The introduction of the electronic device 301 in the conference system 400 can be referred to Figure 3 Related introduction in .

[0109] Among them, the conference room system 401 includes one or more electronic devices 402. Exemplarily, these electronic devices 402 include, but are not limited to, electronic touch screens, smart screens, tablet computers, personal computers (PCs), personal digital assistants (PDAs), netbooks, electronic whiteboards and other electronic devices. The electronic device 301 and the electronic device 402 can be electronic devices of the same type or different types, and the present application does not impose any special restrictions on the specific form of the electronic device 402. The conference room systems 401, the electronic devices 301, and the conference room systems 401 and the electronic devices 301 can communicate directly or indirectly, for example, by forwarding the communication through the server 302.

[0110] Among them, one or more conference applications are installed on both electronic device 301 and electronic device 402. The conference application can be used to realize the function of online conference. Members in conference room system 401 can hold meetings with users of other electronic devices 301 or members in other conference room systems 401 through the conference application installed on electronic device 402 in conference room system 401.

[0111] During the meeting, the electronic device 301 and the electronic device 402 can obtain relevant information in the meeting (for example, audio information of the meeting members, annotation information of the meeting members, and the meeting content, etc.), and generate the minutes of the meeting based on the relevant information. Optionally, the electronic device 301 and the electronic device 402 can also send the obtained relevant information of the meeting to the server 302, and the server 302 generates the minutes of the meeting. Optionally, the server 302 can also send the generated minutes of the meeting to the electronic device 301 and each electronic device 402.

[0112] Optionally, the conference system 400 may also include one or more display devices 403, such as but not limited to various devices with display functions such as projectors and electric screens, and the display device 403 may be used to display conference images on a large screen. In some embodiments of the present application, the display device 403 may also be used to display generated meeting minutes.

[0113] After the meeting, electronic device 301 and electronic device 402 review the main points of the meeting based on the meeting minutes to assist the user in understanding the content of the meeting.

[0114] It should be understood that Figure 3 , Figure 4 This is a simplified example diagram for ease of understanding only. In actual applications, the above conference system may also include other devices that are not shown in the diagram.

[0115] The solution provided in the embodiments of the present application can be applied to various electronic devices, such as Figure 3 The electronic device 301 shown, Figure 4 In the electronic device 402 shown in FIG. 4 , taking the electronic device as a mobile phone as an example, Figure 5 A schematic structural diagram of an electronic device is shown.

[0116] The electronic device may include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a sensor module 580, a button 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, etc.

[0117] The processor 510 may include one or more processing units, for example, the processor 510 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0118] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0119] The processor 510 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. The memory may store instructions or data that the processor 510 has just used or cyclically used. If the processor 510 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 510, and thus improves the efficiency of the system.

[0120] In some embodiments, processor 510 may include one or more interfaces.

[0121] The charging management module 540 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0122] The power management module 541 is used to connect the battery 542, the charging management module 540 and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540 to power the processor 510, the internal memory 521, the display screen 594, the camera 593, and the wireless communication module 560.

[0123] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 550, wireless communication module 560, modem processor and baseband processor.

[0124] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0125] The mobile communication module 550 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 550 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0126] The wireless communication module 560 can provide wireless communication solutions for application in electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0127] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 550, and antenna 2 is coupled to wireless communication module 560, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0128] In some embodiments of the present application, each electronic device in the conference system can establish a communication connection through the above-mentioned mobile communication module 550 and / or wireless communication module 560.

[0129] The electronic device implements the display function through a GPU, a display screen 594, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 may include one or more GPUs that execute program instructions to generate or change display information.

[0130] The display screen 594 is used to display images, videos, etc. The display screen 594 includes a display panel. In some embodiments, the electronic device may include 1 or N display screens 594, where N is a positive integer greater than 1. In some embodiments of the present application, the display screen 594 may be used to display the generated meeting minutes.

[0131] The electronic device can realize the shooting function through ISP, camera 593, video codec, GPU, display screen 594 and application processor.

[0132] The camera 593 is used to capture still images or videos. In some embodiments, the electronic device may include 1 or N cameras 593, where N is a positive integer greater than 1.

[0133] In some embodiments of the present application, the camera 593 can capture facial information to identify the identities of meeting members during a meeting, or the camera 593 can capture hand movements of users within the field of view of the electronic device during a meeting.

[0134] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0135] The external memory interface 520 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device.

[0136] The internal memory 521 can be used to store computer executable program codes, which include instructions. The internal memory 521 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. The processor 510 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 521 and / or instructions stored in a memory provided in the processor.

[0137] The electronic device can implement audio functions such as music playing and recording through the audio module 570 and the application processor.

[0138] The audio module 570 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 570 can also be used to encode and decode audio signals. In some embodiments, the audio module 570 can be arranged in the processor 510, or some functional modules of the audio module 570 can be arranged in the processor 510.

[0139] In some embodiments of the present application, the audio module 570 includes a microphone array, which is used to record conference audio.

[0140] Optionally, the sensor module 580 may include one or more of a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0141] In some embodiments of the present application, the electronic device may also be equipped with a stylus, and the user may use the stylus to write information such as text or patterns on the display screen of the electronic device. The electronic device may display the motion track formed by the stylus on the display screen.

[0142] The key 590 includes a power key, a volume key, etc. The key 590 can be a mechanical key or a touch key. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.

[0143] Motor 591 can generate vibration prompts.

[0144] Indicator 592 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0145] The SIM card interface 595 is used to connect a SIM card.

[0146] For example, the above only uses a mobile phone as an example to illustrate the structure of the electronic device in the embodiment of the present application, but does not constitute a limitation on the structure and form of the electronic device. The embodiment of the present application does not limit the structure and form of the electronic device. For example, Figure 6 Another exemplary structure of an electronic device is shown. Figure 6 As shown, the electronic device includes: a processor 601, a memory 602, a transceiver 603, and an input / output device 604. The implementation of the processor 601 and the memory 602 can refer to the implementation of the processor and memory of the mobile phone. The transceiver 603 is used for the electronic device to interact with other devices. The transceiver 603 can be a device based on Wi-Fi, Bluetooth or other communication protocols. The input / output device 604 can be used to input information to the electronic device or output information from the electronic device. For example: the input / output device 604 can be a display, an audio player, a speaker, etc., which can be used for the electronic device to output meeting minutes.

[0147] In some other embodiments of the present application, the electronic device may include Figure 5 , Figure 6 More or fewer components shown, or some components are combined, or some components are split, or some components are replaced, or different component arrangements are arranged. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0148] The technical solutions involved in the following embodiments can all be Figure 5 ,like Figure 6 The invention is implemented in the device of the structure shown.

[0149] Take the scenario where users use the large electronic touch screen in the corporate conference room to hold an offline meeting as an example for detailed introduction. Figure 7 , Figure 7 A flowchart of a method for generating meeting minutes provided in an embodiment of the present application, the method comprising the following steps:

[0150] S700: The electronic device pre-processes the conference files of the current conference and divides the conference files into different modules.

[0151] Among them, the conference file is a file to be discussed and displayed in the current meeting. Optionally, the conference file is a multimedia file formed by one or more information. The conference file includes one or more of the following files: a document file (which can also be described as a text file), a picture file, an audio file or a video file, etc. The embodiment of the present application does not impose any special restrictions on the specific format of the conference file.

[0152] Exemplarily, the conference file is an office document (such as word) containing text and pictures. Alternatively, the conference file is a presentation (such as ppt) containing text, pictures and videos.

[0153] In some examples, preprocessing includes operations such as layout recognition, semantic analysis, module segmentation and numbering, and keyword capture.

[0154] Among them, layout recognition is used to analyze and identify each layout in the conference document and extract useful information from each layout. For example, the electronic device divides the conference document into different layouts according to regions or paragraphs, analyzes and recognizes each layout, and extracts useful information from each layout.

[0155] Semantic analysis is used to deeply understand and interpret the text in conference documents. For example, the vocabulary, syntax and context in the text are analyzed to infer the meaning, sentiment, intention and other information of the text.

[0156] The module segmentation and numbering is used to divide and mark different parts or elements in the conference document according to the layout recognition results and semantic analysis results, divide them into multiple modules, and assign a unique number to each module. In the embodiment of the present application, the position information of each module after segmentation is determined.

[0157] Among them, keyword crawling is used to crawl the keywords, content summaries, etc. of each module after segmentation. For example, the most representative and important keywords or phrases in each module are crawled.

[0158] In the embodiment of the present application, before the meeting starts, the electronic device receives the meeting file transmitted by the user, pre-processes the meeting file, divides the meeting file into different modules, and determines the module information corresponding to each module. The module information includes information such as the module number, the location information of the module, and the module content summary (such as keywords).

[0159] In some examples, the electronic device pre-processes the entire conference file and divides it into different modules according to the content of the conference file. When displaying the conference file, a module matching the displayed content is determined according to the displayed content.

[0160] In other examples, the electronic device displays a conference document, preprocesses the current display interface in the conference document, and divides the current display interface into different modules. In this way, the modules are dynamically divided according to the changes in the display page. The embodiment of the present application does not limit the order of displaying the conference document and preprocessing the conference document.

[0161] In one possible implementation, an electronic device receives a conference file transmitted by a user through a wired connection or a wireless connection. For example, the electronic device is connected via a USB flash disk (also described as a USB flash drive) or near field communication (NFC) technology to transmit the conference file to the electronic device. Alternatively, the electronic device is wirelessly connected via a wireless local area network (WLAN) directly, a wireless fidelity (Wi-Fi) network or Bluetooth (BT) and other technologies to transmit the conference file to the electronic device.

[0162] For example, before the meeting starts, the user transfers the meeting file (PPT file) to the large electronic touch screen in the company's conference room via a USB flash drive. The large electronic touch screen performs pre-processing operations such as layout recognition, semantic analysis, module segmentation and numbering, and keyword capture on the presentation in the PPT file, divides the PPT file into different modules, and determines the module information such as the number, location information, and content summary of each module in the PPT file. When the electronic device displays the PPT file in response to the user's operation, the module information contained in the current page is determined based on the content displayed on the current page. Figure 8 As shown, the display interface of a certain page in the PPT file, the electronic device divides the interface into four modules after pre-processing, namely the first module 801, the second module 802, the third module 803 and the fourth module 804), and determines the module information of the four modules. For example, the module information of the first module 801 includes: number 1, position 1 (regional position information of the first module), chart 1, etc., the module information of the second module 802 includes: number 2, position 2 (regional position information of the second module), chart 2-quarter, etc., the module information of the third module 803 includes: number 3, position 3 (regional position information of the third module), text box one, etc., and the module information of the fourth module 804 includes number 4, position 4 (regional position information of the fourth module), text box two, etc.

[0163] S701: An electronic device receives a user's annotation operation on a first content in a conference file, displays a first annotation, and generates a first annotation event.

[0164] The first content is the content annotated by the user in the conference document. The first content may be text or a picture in the conference document.

[0165] The marking operation includes a preset operation for a user to mark the content currently displayed on the electronic device.

[0166] Optionally, the preset operation can be a quick operation for the user to mark the content currently displayed on the electronic device. The quick operation includes a quick operation of the user touching the electronic device through an input device or a finger, or a quick operation of the user through gestures, key combinations, etc. Among them, the input device includes a stylus, a mouse and other devices. The stylus can specifically be a resistive stylus, an electromagnetic stylus or a capacitive stylus. Capacitive styluses include active capacitive pens and passive capacitive pens. Active capacitive pens have built-in circuits and batteries, can actively emit electromagnetic signals and track through device sensors, have higher accuracy, low latency, and support more functions. Passive capacitive pens have no circuits and batteries, and are detected through electric field induction technology. They have low accuracy and high latency, and can only provide basic writing, drawing and marking functions. Exemplarily, a user uses a stylus or a finger to input annotations such as underlines on an electronic device.

[0167] Optionally, the preset operation may also be a user's voice command. For example, the user issues a marking command to the electronic device through a voice assistant. The embodiment of the present application does not limit the specific implementation form of the marking operation.

[0168] The first annotation is an annotation made by a user on the first content, and the first annotation is used to mark and annotate the first content.

[0169] In the embodiment of the present application, the first annotation includes one or more of a selective annotation, a directional annotation, and a supplementary annotation. The embodiment of the present application does not impose any special restrictions on the specific form and type of the first annotation. The user can make multiple forms of annotations on the same content.

[0170] Among them, the selection annotation is used to indicate that the user selects the visual element (which can also be described as content) indicated by the annotation, emphasizing or highlighting the visual element. The directional annotation is used to indicate the relationship between visual elements. The supplementary annotation is used to indicate the supplementary description of the visual element. The expression form and the information indicated by each type of annotation are different. The specific meanings of each type of annotation are introduced in detail below.

[0171] Among them, visual elements are the basic units that constitute visual objects. Visual elements can be understood as elements displayed by electronic devices such as text, graphics, images, and tables.

[0172] In the embodiment of the present application, the expression form of the selected neutral annotation includes all relatively closed graphics that frame the visual element (such as an ellipse or rectangle that encloses a word) and horizontal, non-closed graphics that are strongly associated with the position of the visual element (such as an underline or parentheses under the text). Among them, strong association can be understood as the distance between the position of the non-closed graphic and the position of the visual element meets the preset threshold, and the non-closed image is very close to the visual element.

[0173] It is understandable that when a user explains a conference document, in order to emphasize the relevance between the content being explained and the conference document, the visual elements in the conference document related to the explanation content will be selected to highlight and limit the visual elements. Therefore, the content indicated by the selected neutral mark is strongly related to the content of the current voice explanation. In other words, the content in the conference document indicated by the selected neutral mark is strongly related to the information after the current voice is converted into text.

[0174] For example, Fig. 9 As shown, when the user is explaining category 2 in chart 1, the information of category 2 is framed with a rectangular mark 901, indicating that the information of category 2 is emphasized. According to the mark 901, it can be determined that the important information in chart 1 is category 2.

[0175] In the embodiment of the present application, the expression form of the directional annotation includes annotation symbols with one or more obvious directional attributes (such as single arrows, double arrows, non-horizontal lines or curves, braces, etc.) The directional annotation is used to indicate the correlation between one or more visual elements.

[0176] For example, Fig. 9 As shown, after explaining Chart 1, the user draws a single arrow mark 902 at Chart 1 pointing to Chart 2. The single arrow mark 902 can indicate that Chart 2 is derived from the information in Chart 1, or that the explanation content is transferred from Chart 1 to Chart 2. Alternatively, when the user is explaining Text Box 1, the curly bracket mark 903 is used to frame all the contents in Text Box 1. The current explanation content can be determined based on the range and direction of the curly bracket mark 903. The current explanation content is all the contents of Text Box 1 within the range of the curly bracket mark 903.

[0177] It is understandable that the embodiments of the present application do not limit the directional meaning of the directional annotation, and the specific meaning of the directional annotation can be determined by performing semantic analysis, understanding and matching based on the directional annotation and the current explanation content.

[0178] In the embodiments of the present application, annotations that do not have a selection or pointing meaning are collectively referred to as supplementary annotations. Supplementary annotations are used to supplement the visual elements corresponding to the annotation; or, to supplement the module where the annotation is located. For example, if a user marks an asterisk on a module, it means that the module is highlighted and the content of the module is important information. Alternatively, if a user marks a question mark on a module, it means that there is a question about the content of the module. Alternatively, if a user marks text on a module, it means a supplementary explanation of the module or a content of the module.

[0179] For example, Fig. 9 As shown, after the user explains text box 1 and uses curly bracket annotation 903 to frame all the contents of text box 1, the user writes a text annotation 904 at the arrow of curly bracket annotation 903 - Quarter 2. According to the text annotation 904, it can be determined that the contents of text box 1 are all information of Quarter 2, and the information of Quarter 2 can be finely associated with all the contents contained in curly bracket annotation 903.

[0180] The first annotation event includes relevant information of the first annotation. Exemplarily, the first annotation event includes the first content, the first annotation, the timestamp of the first annotation, the location information of the first annotation, the user information triggering the first annotation, and the like.

[0181] In some embodiments of the present application, the electronic device receives a user's annotation operation on first content in content currently displayed on the electronic device, displays a first annotation, and generates a first annotation event.

[0182] In a possible implementation, the electronic device receives a user's annotation operation on first content in content currently displayed on the electronic device through an input device or a finger, displays a first annotation, and generates a first annotation event.

[0183] Specifically, the electronic device detects the user touching the content currently displayed by the electronic device through an input device or a finger. If it is detected that the time length of the contact position staying on the first content exceeds a time threshold, or it is detected that the contact position continuously moves around the first content to continuously form ink marks, the electronic device determines that the user has performed an annotation operation on the first content. Subsequently, the electronic device responds to the user's annotation operation on the first content, identifies the ink marks and the first annotation corresponding to the annotation operation, and displays the first annotation. A first annotation event is generated according to the first annotation.

[0184] In a possible implementation, the electronic device recognizes the ink corresponding to the annotation operation through optical character recognition (OCR) technology, determines the annotation type of the first annotation corresponding to the ink, and

[0185] In some examples, the electronic device detects a closed figure or a horizontal line segment or curve through OCR technology, and there is a visual element within a preset range (e.g., 20 pixels (px)) inside the closed figure or above the horizontal line segment or curve, then the closed figure or horizontal line segment or curve is determined to be a selected neutral annotation, and the visual element related to the selected neutral annotation is the first content.

[0186] For example, Fig. 9 The rectangular mark 901 in the figure is a selected mark, and the first content selected by the rectangular mark 901 is the chart information of category 2 in chart 1.

[0187] In other examples, when the electronic device detects a directional symbol or a non-horizontal line segment or curve through OCR technology, it determines that the directional symbol or the non-horizontal line segment or curve is a directional marking, and determines the first content related to the directional marking based on the visual elements indicated by the directional marking.

[0188] For example, Fig. 9 The single arrow mark 902 is a directional mark, and the first content related to the single arrow mark 902 is Chart 1 and Chart 2.

[0189] In other examples, the electronic device detects through OCR technology that the ink is not a directional annotation or a selective annotation, and the proportion of ink located in the same module in the overall ink exceeds a preset proportion, then the ink is determined to be a supplementary annotation for the content displayed in the module, and the first content related to the supplementary annotation is determined.

[0190] The preset ratio is a preset ratio, such as 80%. Fig. 9 The Chinese character annotation 904 is all located in the third module 803, and the ink of the text annotation 904 "quarter 2" located in the third module 803 accounts for 100% of the total ink, which exceeds the preset proportion of 80%, and the text annotation 904 is neither a directional annotation nor a selective annotation. It is determined that the text annotation 904 is a supplementary annotation, which is used to supplement and explain the content of the third module. The first content related to the text annotation 904 is the content in the third module.

[0191] It is understandable that if the supplementary annotation spans multiple modules, it is necessary to determine the proportion of the supplementary annotation ink in each module to the overall ink, and determine the module whose ink proportion exceeds the preset proportion as the module corresponding to the supplementary annotation. That is, the user of the supplementary annotation here supplements the module.

[0192] In some other embodiments of the present application, the electronic device receives a annotation command issued by a user through a voice command, displays a first annotation, and generates a first annotation event.

[0193] It is understandable that after the electronic device receives the user's annotation operation on the first content, it needs to determine the user information that triggered the annotation operation, accurately distinguish which users made which annotations, avoid confusion, and then accurately generate the first annotation event and meeting minutes.

[0194] pass Figure 10-Figure 14 Various application scenarios are introduced, and specific implementation methods for an electronic device to determine user information corresponding to each annotation operation and generate a first annotation event are introduced.

[0195] In some scenarios, multiple speakers may be required to participate in the conference presentation during the meeting. In this case, the electronic device needs to distinguish the user identities of these multiple people to determine the marked events corresponding to different users.

[0196] In some examples, the electronic device determines user information corresponding to each annotation operation through an input device, thereby generating a first annotation event.

[0197] In some application scenarios, such as Fig.10 As shown in (a), the electronic device 1000 is an electronic device including a touch screen or infrared screen 1001, and the electronic device is equipped with a passive capacitive pen 1002, and the user uses the passive capacitive pen or a finger to annotate, then at most two users are supported to annotate the conference document displayed by the electronic device at the same time. In this application scenario, the electronic device needs to distinguish the user information corresponding to each annotation to determine the annotation events corresponding to different users.

[0198] It is understandable that even if an electronic device is equipped with multiple passive capacitive pens, the contact information generated when different passive capacitive pens touch the electronic device is roughly the same, and it is difficult to distinguish multiple passive capacitive pens through the contact information. In other words, the passive capacitive pen is equivalent to a finger and is only used to assist the user in marking. The electronic device can only recognize that the user uses a passive capacitive pen to mark. Even if there are multiple passive capacitive pens marking on the electronic device, the electronic device will recognize the markings of multiple passive capacitive pens as the markings of the same passive capacitive pen, and it will not be able to recognize the user information of each passive capacitive pen.

[0199] Specifically, before the meeting starts, the electronic device determines the user information of the speaker among the users participating in the meeting, prompts the maximum number of speakers that the current electronic device can connect to, and prompts the speaker's input devices that can be bound. According to the speaker's use of input devices, the user information of the speaker who uses a passive capacitive stylus to mark and the user information of the speaker who uses a finger to mark are determined.

[0200] For example, if there is a presenter, the passive capacitive stylus is preferentially bound. Alternatively, the presenter can bind a passive capacitive stylus or a finger as desired.

[0201] For example, if there are two speakers, one speaker is bound to a passive capacitive stylus, and the other speaker is bound to a finger.

[0202] In a possible implementation, during a conference, the electronic device receives a marking operation input by a user, and if there are two speakers, the user information corresponding to the marking operation is determined based on the contact area and the major and minor axes of the contact.

[0203] The contact area is the size of the surface area occupied by the contact when the electronic device is touched. The long and short axes of the contact are the two axes of the contact area when the electronic device is touched. For example, an elliptical contact includes a longer axis and a shorter axis. The long and short axes of the contact are used to identify the position and direction of the touch.

[0204] Specifically, finger touch and pen tip touch are distinguished according to the touch area, the ratio of the contact length and the change of the touch area and the contact length within a preset time in the marking operation, and then the user information corresponding to the marking operation is determined.

[0205] Take the example of a meeting with two speakers and both speakers are marked at the same time. Fig.11 As shown, it is a flow chart of identifying user information based on a touch screen / infrared screen and a passive capacitive pen provided in an embodiment of the present application.

[0206] S1101. The electronic device determines user information corresponding to each input device.

[0207] In the embodiment of the present application, the input device includes a passive capacitive stylus and a finger. Before the meeting starts, the electronic device receives user information written by the user using the input device, and the electronic device determines the user information corresponding to each input device.

[0208] For example, when a user opens the conference software on an electronic device, the electronic device prompts the user to write the speaker's identity information. User 1 uses a passive capacitive pen to write his or her own user information (such as user 1's identity identifier U1) on the electronic device, and the electronic device determines that the user information corresponding to the passive capacitive pen is U1. User 2 uses his or her finger to write his or her own user information (such as user 2's identity identifier U2) on the electronic device, and the electronic device determines that the user information corresponding to the finger is U2. Subsequently, the electronic device opens the conference file (such as PPT) in response to the user's (such as the speaker's) operation.

[0209] S1102: The electronic device receives a user's annotation operation on the first content through an input device, and determines contact information corresponding to each annotation operation.

[0210] In an embodiment of the present application, the electronic device receives a user's annotation operation on the first content through an input device, and determines contact information corresponding to each annotation operation through optical character recognition (OCR) technology.

[0211] For example, during a meeting, user 1 and user 2 mark at the same time, and the electronic device receives two marking operations at the same time (e.g., user 1 marks first content 1 with a passive capacitive stylus, and user 2 marks first content 2 with a finger). The electronic device identifies the contact information corresponding to each marking operation through OCR technology, such as one marking operation corresponding to contact information P1, and another marking operation corresponding to contact information P2.

[0212] It is understandable that although the correspondence between input devices and user information is set before the meeting starts, the electronic device does not know which input device is marked. Therefore, the electronic device needs to determine which input device corresponds to each marking, and then the user information corresponding to each marking.

[0213] S1103: The electronic device determines the contact type according to the contact information.

[0214] The contact information includes the contact area, the contact major and minor axes, and the changes in the contact area and the contact major and minor axes within a preset time.

[0215] Among them, the contact types include pen tip contact and finger contact.

[0216] In an embodiment of the present application, the electronic device determines the contact type corresponding to each contact information based on the contact area, the contact major and minor axes, and the changes in the contact area and the contact major and minor axes within a preset time in each contact information.

[0217] For example, the electronic device determines that the contact type corresponding to the contact information P1 is a pen tip contact, and the type corresponding to the contact information P2 is a finger contact.

[0218] S1104: The electronic device determines the corresponding input device and user information according to the contact type.

[0219] Exemplarily, if the contact type corresponding to the contact information P1 is a pen tip contact, the corresponding input device is determined to be a passive capacitive pen, and according to the correspondence between the input device and the user information determined in step S1101, the user information is determined to be U1. If the contact type corresponding to the contact information P2 is a finger contact, the corresponding input device is determined to be a finger, and according to the correspondence between the input device and the user information determined in step S1101, the user information is determined to be U2.

[0220] S1105: The electronic device distinguishes and displays first annotations corresponding to different user information, and generates first annotation events corresponding to different first annotations.

[0221] In the embodiment of the present application, the electronic device displays labels corresponding to different user information by distinguishing colors.

[0222] In the embodiment of the present application, the electronic device integrates the first content, the first annotation, the timestamp of the first annotation, the location information of the first annotation, the user information triggering the first annotation, and other information into a first annotation event. The first annotation event may also include information such as the input device type and the contact type.

[0223] In another possible implementation, during a conference, the electronic device receives a marking operation input by a user through an input device, and if there are two speakers, the user information corresponding to the marking operation is determined based on the current or pressure.

[0224] Specifically, since the capacitance of the capacitive pen is higher, the current generated by the pen tip contact is higher than that of the finger contact. Therefore, by detecting the current of the contact, the finger contact and the pen tip contact are distinguished, and then the user information corresponding to the marking operation is determined.

[0225] Specifically, since the force a user needs to apply when touching an electronic device with a capacitive pen is greater than the force a user applies when touching an electronic device with a finger, the finger touch and the pen tip touch are distinguished by detecting the pressure of the touch point, and then the user information corresponding to the marking operation is determined.

[0226] In this way, during a meeting, if one of the two speakers uses a passive capacitive pen to input annotation operations on the electronic device, and the other speaker uses his finger to input annotation operations on the electronic device, the electronic device can accurately distinguish the user information of the input annotation operations in the above manner to avoid confusion and cause inaccurate subsequent meeting minutes.

[0227] In another possible implementation, during a conference, the electronic device receives a user's annotation operation. If there is only one speaker, the user information of the user who inputs the annotation operation can be directly determined.

[0228] In other application scenarios, such as Fig.10 As shown in (b), the electronic device 1003 is an electronic device including a touch screen 1004, and the electronic device is equipped with an active capacitive pen 1005, and the user uses the active capacitive pen 1005 to mark, so that multiple users can mark the meeting documents displayed by the electronic device at the same time. In this application scenario, the electronic device needs to distinguish the user information corresponding to each mark to determine the marking events corresponding to different users.

[0229] It is understandable that the active capacitive pen can actively emit electromagnetic signals (also described as writing signals), and each active capacitive pen emits different writing signals to the electronic device when marking, so that the electronic device can identify the user information corresponding to each active capacitive pen.

[0230] In an embodiment of the present application, the input device includes an active capacitive stylus. Before the meeting starts, the electronic device receives user information written by the user using the input device, and the electronic device determines the user information corresponding to each input device.

[0231] Take the example of an electronic device equipped with two active capacitive pens, a meeting with two speakers and both speakers marking at the same time. Fig.12 As shown, it is a flow chart of identifying user information based on a touch screen and an active capacitive pen provided in an embodiment of the present application.

[0232] S1201. The electronic device determines the user information and writing signal corresponding to each input device.

[0233] The writing signal is used to identify the input device (active capacitive pen). Each active capacitive pen has its corresponding writing signal.

[0234] It can be understood that when a user uses an active capacitive pen to touch the display screen of an electronic device to make annotations (which can be understood as the first annotation made by using the active capacitive pen to touch the display screen of an electronic device), the active capacitive pen sends a writing signal to the electronic device. The electronic device can receive different writing signals sent after multiple active capacitive pens touch the electronic device. The electronic device determines the active capacitive pen corresponding to the writing signal based on the writing signal.

[0235] In an embodiment of the present application, the input device includes an active capacitive stylus. Before the meeting starts, the electronic device receives user information written by the user using the input device (active capacitive stylus) and a writing signal sent by the input device, and the electronic device determines the user information and writing signal corresponding to each input device.

[0236] For example, the electronic device is equipped with two active capacitive pens, namely T1 and T2. Before the meeting starts, user 1 uses the active capacitive pen T1 to write his own user information (such as user 1's identity U1) on the electronic device, and at the same time, the active capacitive pen T1 sends a writing signal S1 to the electronic device. The electronic device determines that the user information corresponding to the active capacitive pen T1 is U1 based on the received information, and the writing signal corresponding to the active capacitive pen T1 is S1. User 2 uses the active capacitive pen T2 to write his own user information (such as user 2's identity U2) on the electronic device, and at the same time, the active capacitive pen T2 sends a writing signal S2 to the electronic device. The electronic device determines that the user information corresponding to the active capacitive pen T2 is U2 based on the received information, and the writing signal corresponding to the active capacitive pen T2 is S2.

[0237] S1202: The electronic device receives a user's annotation operation on the first content through an input device and a writing signal sent by the input device, and determines a contact point and an input device corresponding to each annotation operation.

[0238] In an embodiment of the present application, the electronic device receives a user's annotation operation on the first content through an input device and a writing signal sent by the input device, and determines a contact point and an input device corresponding to each annotation operation according to the writing signal.

[0239] For example, during a meeting, user 1 and user 2 perform annotations at the same time, and the electronic device simultaneously receives two annotation operations and the writing signal sent by each input device (e.g., user 1 uses an active capacitive pen T1 to perform annotation 1 on the first content 1, and the electronic device receives the writing signal S1 sent by the active capacitive pen T1. User 2 uses an active capacitive pen T1 to perform annotation 2 on the first content 2, and the electronic device receives the writing signal S1 sent by the active capacitive pen T1). The electronic device determines the contact point and input device corresponding to each annotation operation based on the correspondence between each input device and the writing signal in step S1201, such as the electronic device determines that the input device for inputting annotation 1 (i.e., annotating the first content 1) is the active capacitive pen T1 and the contact point for annotating the first content 1 is P1 based on the writing signal S1, and determines that the input device for inputting annotation 2 (i.e., annotating the first content 2) is the active capacitive pen T2 and the contact point for annotating the first content 2 is P2 based on the writing signal S2.

[0240] S1203: The electronic device determines user information corresponding to each annotation operation according to the input device corresponding to each annotation operation.

[0241] In an embodiment of the present application, the electronic device determines the user information corresponding to each annotation operation according to the correspondence between each input device and the user information and according to the input device corresponding to each annotation operation.

[0242] In an embodiment of the present application, after determining the user information corresponding to each annotation operation, the contact point, input device, writing signal and user information corresponding to each annotation operation are associated.

[0243] For example, based on the example in step S1202, the electronic device determines that the contact point corresponding to label 1 is P1, the input device corresponding to label 1 is active capacitive pen T1, the writing signal corresponding to label 1 is S1, and the user information corresponding to label 1 is U1. The contact point corresponding to label 2 is P2, the input device corresponding to label 2 is active capacitive pen T2, the writing signal corresponding to label 2 is S2, and the user information corresponding to label 2 is U2.

[0244] S1204: The electronic device distinguishes and displays first annotations corresponding to different user information, and generates first annotation events corresponding to different first annotations.

[0245] In the embodiment of the present application, the electronic device displays labels corresponding to different user information by distinguishing colors.

[0246] In the embodiment of the present application, the electronic device integrates the first content, the first annotation, the timestamp of the first annotation, the location information of the first annotation, the user information triggering the first annotation, and other information into a first annotation event. The first annotation event may also include information such as the input device type, the input signal, and the contact point.

[0247] In this way, the electronic device identifies the user information corresponding to each annotation through the input device, so as to accurately distinguish the annotation events corresponding to each user, which is conducive to the subsequent accurate generation of meeting minutes and ensures the accuracy of the meeting minutes.

[0248] In the above example, the electronic device only includes a touch screen and an input device. In actual applications, the electronic device may also include a camera. The electronic device may identify the user information corresponding to each annotation through the camera and the input device. The following describes in detail how to identify the user information corresponding to each annotation through the camera and the input device.

[0249] In other examples, the electronic device may also collect user images to determine user information corresponding to each annotation, thereby generating a first annotation event.

[0250] In some application scenarios, such as Fig.10 As shown in (c), the electronic device 1006 is an electronic device including a touch screen or an infrared screen 1007, and the electronic device also includes a camera 1008. The electronic device is equipped with a passive capacitive pen 1009, and the user uses the passive capacitive pen 1009 to annotate, so that multiple users can annotate the conference documents displayed by the electronic device at the same time. In this application scenario, the electronic device needs to distinguish the user information corresponding to each annotation to determine the annotation events corresponding to different users.

[0251] Take the example of a meeting with two speakers and two speakers marked. Fig.13 As shown, it is a flow chart of identifying user information based on a touch screen / infrared screen, a camera and a passive capacitive pen provided in an embodiment of the present application.

[0252] S1301. The electronic device determines the facial information corresponding to each user information.

[0253] In an embodiment of the present application, before the meeting starts, the electronic device identifies and enters the facial information and user information of each user participating in the meeting, and stores each user information and its corresponding facial information in an associated manner.

[0254] Exemplarily, before the meeting starts, when the electronic device receives the user information input by each user, it identifies the facial information of each user through the camera, and stores the user information and facial information of each user in association. For example, when the electronic device receives the identity identifier U1 input by user 1, the facial information F1 of user 1 is collected through the camera. The electronic device determines that the user information corresponding to user 1 is U1, and the corresponding facial information is F1. When the electronic device receives the identity identifier U2 input by user 2, the facial information F2 of user 2 is collected through the camera. The electronic device determines that the user information corresponding to user 2 is U2, and the corresponding facial information is F2.

[0255] S1302: The electronic device receives a user's annotation operation on the first content through an input device, and determines a touch point corresponding to each annotation operation.

[0256] In an embodiment of the present application, the electronic device receives a user's annotation operation on the first content through an input device, and determines a touch point corresponding to each annotation operation through OCR technology.

[0257] For example, during a meeting, user 1 and user 2 mark at the same time. User 1 marks 1 on first content 1 using a passive capacitive stylus T1, and the electronic device receives the mark 1 and determines the contact point P1 of the mark 1 in the electronic device. User 2 marks 2 on first content 2 using a passive capacitive stylus T2, and the electronic device receives the mark 2 and determines the contact point P2 of the mark 2 in the electronic device.

[0258] S1303. The electronic device determines the facial information corresponding to the touch point based on the touch point, and determines the user information based on the facial information.

[0259] In the embodiment of the present application, the electronic device captures the facial information of the person who generated the touch point through a camera according to the touch point, and determines the corresponding user information according to the facial information.

[0260] For example, during a meeting, the electronic device turns on the camera, and when the electronic device receives a touch operation, it captures an image in front of the electronic device. The electronic device determines the position information of the touch point P1 in the touch operation, determines the hand information H1 that generates the touch point P1 in the image based on the position information of the touch point P1, determines the face information F1 that triggers the touch point P1 in the image based on the hand information H1, and determines the user information corresponding to the face information F1 as U1 based on the correspondence between the face information and the user information. In this way, the face information corresponding to the touch point P1 is determined to be F1, and the user information corresponding to the touch point P1 is U1. The electronic device determines the position information of the touch point P2, determines the hand information H2 that generates the touch point P2 in the image based on the position information of the touch point P2, determines the face information F2 that triggers the touch point P2 in the image based on the hand information H2, and determines the user information corresponding to the face information F2 as U2 based on the correspondence between the face information and the user information. In this way, the face information corresponding to the touch point P2 is determined to be F2, and the user information corresponding to the touch point P2 is determined to be U2.

[0261] S1304: The electronic device distinguishes and displays first annotations corresponding to different user information, and generates first annotation events corresponding to different first annotations.

[0262] In the embodiment of the present application, the electronic device displays labels corresponding to different user information by distinguishing colors.

[0263] In the embodiment of the present application, the electronic device integrates the first content, the first annotation, the timestamp of the first annotation, the location information of the first annotation, the user information triggering the first annotation, and other information into a first annotation event. The first annotation event may also include information such as the input device type and the contact type.

[0264] Optional, such as Figure 7 As shown, after S1304, S1305 may also be included to quickly determine the user information corresponding to the marking operation.

[0265] S1305: The electronic device determines the annotated hot zone corresponding to each user information.

[0266] The annotation hotspot is an area predicted by the electronic device that the user may make annotations.

[0267] In an embodiment of the present application, in the process of determining user information according to the contact information in the above step S1303, the electronic device can also determine the marked hot zone of each user according to the hand information corresponding to the contact information.

[0268] Specifically, the electronic device identifies and locates the current position of each user's hand relative to the electronic device when performing a labeling operation, and determines that the current position of each user's hand relative to the electronic device is mapped to the labeling hot zone of the electronic device.

[0269] It is understandable that the annotation hot zone is an area where the user's hand is mapped to the electronic device. The annotation hot zone changes as the user's hand moves, and the user may perform annotation operations in the annotation hot zone.

[0270] S1306: The electronic device determines whether the new contact is located in the marked hot zone, and if so, executes S1307; otherwise, executes S1303.

[0271] The new touch point may be understood as a touch point generated when the electronic device receives again the user's annotation operation on the first content through the input device.

[0272] In an embodiment of the present application, the electronic device determines whether the new contact is located in the marked hot zone according to the position information of the new contact and the position information of the marked hot zone.

[0273] S1307: The electronic device determines that the user information corresponding to the new touch point is the user information corresponding to the annotated hot zone, and generates an annotation event corresponding to the new touch point.

[0274] It is understandable that the user performs the annotation operation with his hands, and the new contact points generated by the annotation operation are usually located near the user's hands, that is, within the annotation hot zone. If a new contact point appears that is not located within the annotation hot zone, it means that another user has performed an annotation, so the facial information needs to be recognized again. Moreover, if the user's hand is away from the electronic device, the electronic device cannot recognize the annotation hot zone corresponding to the user. If the user annotates again, the facial information needs to be recognized again. Optionally, the above example is that the electronic device determines the annotation hot zone after the user performs the annotation operation. In actual applications, the electronic device can also first determine the annotation hot zone corresponding to each user, and then receive the user's annotation operation to generate a annotation event.

[0275] Exemplarily, before the meeting starts, the electronic device identifies and enters the facial information and user information of each user participating in the meeting, and associates and stores each user information and its corresponding facial information. During the meeting, the electronic device turns on the camera, and when it detects that the distance of the user close to the electronic device is within a predetermined threshold, the camera of the electronic device identifies and locates the current position of each user's hand relative to the electronic device, and determines that the current position of each user's hand relative to the electronic device is mapped to the annotation hot zone of the electronic device. The electronic device receives annotation operations on the first content from different users through input devices, distinguishes and displays the first annotations of different users, and generates first annotation events corresponding to different first annotations.

[0276] It is understandable that when a user annotates information in an electronic device, he needs to first approach the electronic device and hold the input device to annotate on the display screen of the electronic device. Therefore, in the process of the user's hand approaching the electronic device and performing annotating operations on the electronic device, the camera of the electronic device recognizes the hand information of each user and determines the annotation hot zone corresponding to each user. Moreover, the contact position corresponding to each user's annotation operation is within the annotation hot zone corresponding to the user's hand information. In this way, the identity information corresponding to each annotation can be directly determined according to the annotation hot zone, and an annotation event can be quickly generated. In this way, the electronic device identifies the user information corresponding to each annotation based on the annotation operation contact information and face information, which can ensure the subsequent accurate generation of meeting minutes.

[0277] In other application scenarios, such as Fig.10 As shown in (d), the electronic device 1010 is an electronic device including a touch screen 1011, and the electronic device also includes a camera 1012. The electronic device is equipped with an active capacitive pen 1013, and the user uses the active capacitive pen 1013 to annotate, then multiple users are supported to annotate the conference documents displayed on the electronic device at the same time.

[0278] In an embodiment of the present application, the input device includes an active capacitive stylus. Before the meeting starts, the electronic device receives user information written by the user using the input device, and the electronic device determines the user information corresponding to each input device.

[0279] Take the example of an electronic device equipped with two active capacitive pens and cameras, a meeting with two speakers and both speakers annotating at the same time. Fig.14 As shown, it is a flow chart of identifying user information based on a touch screen, a camera and an active capacitive pen provided in an embodiment of the present application.

[0280] S1401. The electronic device determines the facial information corresponding to each user information.

[0281] In the embodiment of the present application, the specific implementation method of step S1401 refers to S1301 above.

[0282] S1402: The electronic device determines the user information and writing signal corresponding to each input device.

[0283] In the embodiment of the present application, the specific implementation method of step S1402 refers to S1201 above.

[0284] S1403: The electronic device receives a user's annotation operation on the first content through an input device and a writing signal sent by the input device, and determines a contact point and an input device corresponding to each annotation operation.

[0285] In the embodiment of the present application, the specific implementation method of step S1403 refers to S1202 above.

[0286] S1404. The electronic device determines the user information corresponding to each annotation operation based on the input device corresponding to each annotation operation and the facial information corresponding to each touch point.

[0287] In the embodiment of the present application, the specific implementation method of step S1404 refers to S1203 and S1303 above.

[0288] S1405: The electronic device distinguishes and displays first annotations corresponding to different user information, and generates first annotation events corresponding to different first annotations.

[0289] In the embodiment of the present application, the specific implementation method of step S1405 refers to S1404 above.

[0290] S1406: The electronic device determines the annotated hot zone corresponding to each user information.

[0291] In the embodiment of the present application, the specific implementation method of step S1406 refers to S1305 above.

[0292] S1407, the electronic device determines whether the new contact is located in the marked hot zone, if so, executes S1408; otherwise, executes step S1404.

[0293] In the embodiment of the present application, the specific implementation method of step S1407 refers to S1306 above.

[0294] S1408: The electronic device determines that the user information corresponding to the new touch point is the user information corresponding to the annotated hot zone, and generates an annotation event corresponding to the new touch point.

[0295] In the embodiment of the present application, the specific implementation method of step S1408 refers to S1307 above.

[0296] In this way, the user information corresponding to each annotation is dually identified based on the writing signal and the facial information, which can ensure the subsequent accurate generation of meeting minutes.

[0297] In this way, the electronic device identifies the user information corresponding to each annotation through the input device and the camera, so as to accurately distinguish the annotation events corresponding to each user, which is conducive to the subsequent accurate generation of meeting minutes and ensures the accuracy of the meeting minutes.

[0298] The electronic device in the above example includes a touch screen, an input device or a camera. In practical applications, the electronic device may also include a microphone array. The electronic device may collect the voice data of each annotation input through the microphone array and determine the user information corresponding to the voice data. The following is a detailed introduction on how to identify the user information corresponding to each annotation through the microphone array.

[0299] In other examples, the electronic device may also collect user voice to determine user information corresponding to each annotation, thereby generating a first annotation event.

[0300] In some other application scenarios, the electronic device is an electronic device including a microphone array. Before the meeting starts, the electronic device determines the user information and voice data of the speaker among the users participating in the meeting. For example, when the user opens the conference software on the electronic device, the electronic device prompts the speaker to voice-enter his or her own user information. In this way, the electronic device determines the user information and corresponding voice data of the speaker. During the meeting, when it is detected that the user inputs a marking operation, the electronic device records the current voice data and identifies and determines the user information corresponding to the voice data. Alternatively, the electronic device responds to the marking command issued by voice, displays the marking and determines the user information corresponding to the marking command. In this way, the user information corresponding to each marking can be clearly determined by voice recognition, so as to accurately distinguish the marking events corresponding to each user.

[0301] In this way, the user information corresponding to each annotation operation is identified in the above manner, and the annotations and annotation events made by each user in the meeting are accurately determined, so that accurate and rich meeting minutes can be generated subsequently.

[0302] Through the various implementations described in S701 above, the electronic device displays the first annotation of the user on the first content and generates a first annotation event. Afterwards, the electronic device can continue to generate the meeting minutes through the following steps.

[0303] S702: The electronic device determines first voice information corresponding to the first annotation.

[0304] The first voice information is the voice information (which can also be described as text data) obtained by the electronic device recording the explanation audio in real time and converting the explanation audio into text when the user makes the first annotation on the first content. For example, the content summary or keywords of the voice-to-text conversion. Optionally, the first voice information may also include the start timestamp and end timestamp of recording the voice corresponding to the first annotation.

[0305] In some embodiments of the present application, after the meeting starts, the voice of the current meeting is recorded and converted into text (which can also be described as text data). In response to the first annotation, the electronic device obtains the first voice information after the current voice is converted into text.

[0306] In a possible implementation, the electronic device automatically converts the speech of the current meeting into text through automatic speech recognition (ASR) technology.

[0307] Specifically, the electronic device receives a user's annotation operation on a first content in the currently displayed content, obtains text converted from the current voice, performs semantic analysis and keyword capture on the converted text, and determines the first voice information corresponding to the first annotation.

[0308] It is understandable that during a meeting, when the speaker annotates a certain content, he usually explains the annotation. Therefore, when the electronic device displays the first annotation, it can also obtain the text converted from the user's current explanation, extract and summarize important information, and assist the user in understanding the key points or meaning of the first annotation. At the same time, it can also be judged based on the text converted from the voice whether the user has finished explaining the annotated content. If the user has finished explaining, the voice information corresponding to the annotation will no longer be obtained. Wait for the next annotation operation to obtain the voice information corresponding to the annotation operation.

[0309] It is understandable that during the meeting, the speech-to-text recorded throughout the meeting is segmented according to the annotations, and each segmented text corresponds to one or more annotations.

[0310] In other embodiments of the present application, the electronic device records the current voice in response to the first annotation, converts the current voice into text, and determines the first voice information corresponding to the first annotation based on the converted text.

[0311] Specifically, the electronic device performs current voice recording and conversion only after receiving the first annotation, performs semantic analysis and keyword capture on the converted text, and determines the first voice information corresponding to the first annotation.

[0312] In other embodiments of the present application, the electronic device receives a user's annotation operation on a first content in the currently displayed content, determines the module where the first annotation is located according to the position information, module information and first voice information of the first annotation, and highlights the module.

[0313] Optionally, specific implementation methods of highlighting the module include: highlighting and / or enlarging display, etc.

[0314] For example, Fig. 9 As shown, when the user explains the chart 1, the information of the column category 2 is marked with a rectangle 901, and the electronic device receives the information according to the position information of the rectangle mark 901 input by the user and the above Figure 8 The information of each module is divided, and it is determined that the user inputs a selection mark in the form of a rectangle around category 2 in chart 1, so the electronic device displays a rectangular mark 901 around category 2 and highlights ( Fig. 9 (not shown in the figure) The module of Chart 1 indicates that the content of the module of Chart 1 is currently being explained, and Category 2 in Chart 1 is important information.

[0315] S703: The electronic device associates the first content, the first annotation, the first annotation event, and the first voice information, and generates minutes content corresponding to the first annotation.

[0316] In the embodiment of the present application, the keyword or symbol information of the first annotation is determined by using OCR technology and the first voice information.

[0317] In the embodiment of the present application, minutes content corresponding to the first annotation is generated based on the first content, the first annotation, the first annotation event and the first voice information.

[0318] Specifically, the information related to the first annotation is integrated to generate the minutes content corresponding to the first annotation. The minutes content may include the first content, the keyword or symbol information of the first annotation, the timestamp of the first annotation, the location information of the first annotation, the user information that triggers the first annotation, the contact type of the first annotation, the first voice information, and the module information where the first annotation is located.

[0319] Exemplarily, the first voice information, the first annotation event and the first annotation are stored as a JSON file, the first content is stored as a PPTX file, and the minutes content in the format of a txt document is generated according to the above two types of files.

[0320] In the embodiment of the present application, the database may store associated information of the first content, the first annotation, the first annotation event, and the first voice information.

[0321] It can be understood that the minutes content of the embodiment of the present application includes the content information of the annotations indicated by the user, the symbol information of the annotations made by the user, and the text information of the user's explanation of the annotations, etc., and the minutes content is generated by combining three different types of information.

[0322] S704: The electronic device obtains the meeting minutes corresponding to the current meeting according to the minutes content corresponding to the first annotations in different modules.

[0323] In some embodiments of the present application, the electronic device first generates minutes content corresponding to each module according to the minutes content corresponding to each first annotation in each module, and then generates meeting minutes corresponding to the current meeting according to the minutes content corresponding to each module in the meeting document.

[0324] In a possible implementation, the minutes content corresponding to each module may be the minutes content summarized from the minutes content corresponding to the first annotation contained in the module and the content information contained in the module. That is, the minutes content corresponding to each module may be concise and summarized minutes content.

[0325] In another possible implementation, the minutes content corresponding to each module may be the minutes content obtained by integrating the minutes content corresponding to all the first annotations contained in the module. That is, the minutes content corresponding to each module may include the minutes content of each detail.

[0326] It is understandable that in the embodiment of the present application, the meeting documents are first divided into different modules. During the meeting, for each first annotation in each module by the user, the minutes content corresponding to the first annotation is generated. The minutes content corresponding to each first annotation in each module is integrated in chronological order to obtain the minutes content corresponding to each module. Finally, the minutes content corresponding to each module is integrated to obtain the minutes corresponding to the current meeting. This is equivalent to layering and structuring the meeting documents, and then obtaining a structured meeting minutes so that users can better understand the key points of the meeting.

[0327] It is understandable that the above steps S701-S704 generate corresponding minutes content for each annotation. In other embodiments, it is possible to determine whether the annotation operation is located in the same module based on the contact position of the annotation operation, and generate the minutes content corresponding to the module in chronological order for the annotation operations whose contact positions are located in the same module. Then, the minutes content of each module is integrated to obtain the meeting minutes corresponding to the current meeting.

[0328] In some examples, after the electronic device generates the meeting minutes, the meeting minutes can be stored separately for users to view.

[0329] In other examples, after generating the meeting minutes, the electronic device may also integrate the meeting minutes with the meeting documents, so that the user can determine the corresponding meeting minutes at any time while viewing the meeting documents.

[0330] Optionally, the electronic device sends the minutes content corresponding to each annotation of each module to the server, and the server generates the meeting minutes. The server can also send the generated meeting minutes to the electronic device.

[0331] In some examples, after a meeting, the user may need to view the meeting minutes generated by the electronic device. Figure 7 As shown, after S704, S705 may be further included to trigger the electronic device to display the meeting minutes to assist the user in reviewing the main points of the meeting.

[0332] S705: In response to the user's touch operation on the first annotation, the electronic device displays the minutes content corresponding to the first annotation.

[0333] It is understandable that the electronic device may be the electronic device that generates the meeting minutes in S701-S704 above, and after generating the meeting minutes, the electronic device traces back the meeting points according to the meeting minutes. The electronic device may also be other electronic devices that obtain the meeting minutes and meeting documents, and trace back the meeting points according to the meeting minutes and meeting documents.

[0334] like Fig.15 As shown, the specific implementation steps of step S705 are shown.

[0335] S1500. The electronic device adds the meeting minutes to the meeting files.

[0336] In the embodiment of the present application, the electronic device adds the minutes content corresponding to each module in the meeting minutes to each module, and adds the minutes content corresponding to each first annotation to each first annotation in parallel.

[0337] In some implementations, the minutes content is added by overlaying a layer, but the overlay layer is displayed only when the user touches it. Alternatively, the minutes content is connected to the meeting file through a hyperlink.

[0338] For example, a new layer is superimposed on the first annotation, and the new layer contains the minutes content, but the layer is only displayed when the user touches the first annotation. Fig. 9 As shown, the conference file displays the conference file information and annotation information, but does not display the minutes content. Alternatively, a hyperlink is added near the first annotation, and the hyperlink points to the minutes content txt document corresponding to the first annotation.

[0339] S1501: The electronic device receives a user's touch operation on a first module and displays minutes content corresponding to the first module.

[0340] Among them, the first module is the module that the user touches when reviewing the meeting.

[0341] In the embodiment of the present application, the first module is determined according to the touch point position of the user's touch operation.

[0342] In some embodiments of the present application, the electronic device pops up a hyperlink pointing to the minutes content corresponding to the first module according to the user's touch operation on the first module, and the user clicks the hyperlink to display the minutes content corresponding to the first module.

[0343] In other embodiments of the present application, the electronic device covers and displays a layer of minutes content corresponding to the first module near the first module according to the user's touch operation on the first module.

[0344] S1502: The electronic device receives a user's touch operation on a first annotation in a first module, and displays minutes content corresponding to the first annotation.

[0345] Among them, the first annotation is the annotation touched by the user when reviewing the meeting.

[0346] In the embodiment of the present application, the first annotation is determined according to the touch point position of the user's touch operation.

[0347] In some embodiments of the present application, the electronic device pops up a hyperlink pointing to the minutes content corresponding to the first annotation according to the user's touch operation on the first annotation in the first module, and the user clicks the hyperlink to display the minutes content corresponding to the first annotation.

[0348] In other embodiments of the present application, the electronic device overlays and displays a layer of minutes content corresponding to the first module near the first annotation according to a user's touch operation on the first annotation in the first module.

[0349] For example, when the user is backtracking, he touches the rectangular mark 901 in the first module, such as Fig.16 As shown, in response to the user's touch operation, the electronic device pops up a text box 1601 on the first module, and the text box includes the minutes content 1602 corresponding to the rectangular mark 901.

[0350] It can be understood that the above steps S1500-S1502 describe how to review the meeting from the perspective of module to annotation (from the whole to the details), but in actual use, the user can also directly touch the first annotation, and the electronic device responds to the user's touch operation and directly displays the minutes content corresponding to the first annotation.

[0351] Through the above process, the electronic device in the embodiment of the present application pre-processes the meeting files before the meeting starts, divides the meeting files into different modules, and during the meeting, the electronic device displays the first annotation according to the user's annotation operation on the first content, generates a first annotation event, and obtains the first voice information corresponding to the first annotation. Then, the electronic device generates the minutes content corresponding to the first annotation according to the first content, the first annotation, the first annotation event and the first voice information. Finally, the electronic device integrates the minutes content corresponding to the first annotation contained in each module in the meeting file to obtain the minutes of the current meeting. The electronic device in the embodiment of the present application performs refined segmentation and semantic analysis on the user's meeting files, which can reduce the granularity of the automatically generated meeting minutes and improve the accuracy and structure of the meeting minutes. Moreover, the meeting minutes are generated according to the three types of information, namely, meeting files, annotation information and explanation information, which can improve the accuracy of the meeting minutes. Furthermore, each minute in the meeting minutes is added to the meeting file in text form, and users can interactively trace back the meeting minutes, and display the meeting files in layers from the whole to the details during the tracing process. This tracing method reduces the density of detailed information without losing key information, and supplements the detailed content as needed, improving the efficiency of user meeting tracing and reducing the user's mental load. In addition, this application can also intelligently identify user information, distinguish and display the annotations of different users, and is more applicable.

[0352] The above examples are explained using the conference scenario as an example. The following is a detailed description using the teaching scenario as an example. Fig.17 As shown, it is a schematic diagram of the teaching scenario of this application. The teacher stands in front of the electronic device, holding the input device and leaving blackboard notes on the electronic device while explaining or discussing.

[0353] It is understandable that in the conference scenario, the conference documents are available before the meeting begins. However, in the teaching scenario, the electronic device does not display any content at the beginning, and the conference documents are gradually formed according to the content written by the user during the teaching process.

[0354] In the embodiment of the present application, during the writing process, the electronic device performs real-time regional segmentation on the written content. Specifically, the written content can be segmented into different modules according to the distance between different written contents, the writing time difference, or the semantics of different written contents recognized by OCR technology.

[0355] For example, take the example that a user first writes content 1 on the blackboard of an electronic device, and then writes content 2 on the blackboard of an electronic device. If the distance between written content 1 and written content 2 exceeds the preset distance, it is determined that written content 2 is not located in the module area where written content 1 is located, and real-time segmentation is performed based on the positions of written content 1 and written content 2 (such as segmentation based on the midpoint of the distance between the two). Alternatively, if the writing time difference between written content 1 and written content 2 exceeds the preset time, real-time segmentation is performed based on the positions of written content 1 and written content 2. Alternatively, OCR technology is used to identify written content 1 as relevant information of content 1, and written content 2 as relevant information of content 2, and there is no correlation between content 1 and content 2, then real-time segmentation is performed based on the positions of written content 1 and written content 2. The embodiments of the present application do not limit the way in which the electronic device performs regional segmentation, the size of the segmented area, etc.

[0356] The preset distance may be a distance determined based on big data or experience. The preset distance varies according to the size of the display screen of the electronic device. For example, the preset distance of an electronic device with a larger display screen is larger, and the preset distance of an electronic device with a smaller display screen is smaller.

[0357] In some embodiments of the present application, the electronic device records the audio in the entire teaching scenario and converts the audio into text.

[0358] In other embodiments of the present application, the electronic device records the current audio and converts the current audio into text in response to the user's blackboard writing operation.

[0359] It is understandable that the user usually writes a certain content on the blackboard while explaining the current content of the writing on the blackboard. Therefore, the content explained by the user when writing on the blackboard is strongly related to the writing content. The electronic device records the audio in the same time period according to the start and end timestamps of the blackboard writing operation, converts the audio into text, and determines the voice information corresponding to the writing content according to the converted text.

[0360] The voice information can be understood as the electronic device responding to the user's blackboard operation, recording the explanation audio in real time, and converting the explanation audio into text to obtain the voice information. Optionally, the voice information can be text converted from the explanation audio, or the voice information can also be concise and general voice information obtained by semantic analysis, keyword capture, etc. based on the converted text.

[0361] refer to Fig.18 , Fig.18 A flowchart of another method for generating meeting minutes provided in an embodiment of the present application, the method comprising the following steps:

[0362] S1801. In response to a first blackboard writing operation by a user, the electronic device displays a first blackboard writing, generates a first blackboard writing event, and obtains first voice information corresponding to the first blackboard writing.

[0363] The electronic device may be an electronic whiteboard or an electronic device that is installed with whiteboard software and is capable of displaying.

[0364] The first blackboard writing operation is a preset operation of the user to write on the electronic device.

[0365] Optionally, the preset operation can be a quick operation for the user to write on the blackboard of the current display content of the electronic device through an input device. The quick operation includes a quick operation of the user touching the electronic device through a capacitive pen or a finger, or a quick operation of the user through gestures, key combinations, etc. Among them, the input device includes a capacitive pen, a mouse and other devices, and the capacitive pen includes an active capacitive pen and a passive capacitive pen. The input device can also be a finger. Exemplarily, the user uses a capacitive pen or a finger to input text and other blackboard writing on the electronic device.

[0366] Optionally, the preset operation may also be a user's voice command. For example, the user issues a blackboard writing command to the electronic device through a voice assistant. The embodiment of the present application does not limit the specific implementation form of the blackboard writing operation.

[0367] The first blackboard writing is the ink content displayed by the electronic device in response to the user's first blackboard writing operation. For example, if the user uses a capacitive pen to input "content 1" on the electronic device (here, inputting "content 1" is the first blackboard writing operation), the electronic device recognizes the user's input and displays "content 1" (here, "content 1" is the first blackboard writing).

[0368] The first blackboard writing event includes relevant information of the first blackboard writing. Exemplarily, the first blackboard writing event includes the first blackboard writing, the start and end timestamps of the first blackboard writing, the location information of the first blackboard writing, the user information that triggers the first blackboard writing, the input device type for inputting the first blackboard writing, the contact type of the first blackboard writing, and other information.

[0369] The first voice information is the electronic device's response to the user's first blackboard operation, which records the current explanation audio in real time and converts the explanation audio into text to obtain the voice information.

[0370] In some embodiments of the present application, the electronic device responds to the user's first blackboard writing operation, displays the first blackboard writing, and generates a first blackboard writing event; the electronic device records the explanation audio within the time period according to the start and end timestamps of the first blackboard writing, and obtains the first voice information corresponding to the first blackboard writing based on the explanation audio.

[0371] In other embodiments of the present application, the electronic device responds to the user's first blackboard writing operation, displays the first blackboard writing, generates a first blackboard writing event, and identifies keywords of the first blackboard writing through OCR technology; the electronic device starts recording from the starting timestamp of the first blackboard writing, ends recording when the recorded explanation audio does not contain the keywords of the first blackboard writing, and obtains the first voice information corresponding to the first blackboard writing according to the recorded explanation audio.

[0372] S1802. The electronic device determines a first module according to the first blackboard writing.

[0373] Among them, the first module is the area containing the first blackboard writing.

[0374] In the embodiment of the present application, the first module is determined according to the position information of the first blackboard writing. The embodiment of the present application does not limit the specific implementation method of the electronic device determining the first module.

[0375] For example, an area with a preset length as the radius and the first blackboard as the center is determined as the first module. The preset length may be data determined by experiments or big data. For example, 20 centimeters. (It is understandable that after writing the first blackboard, the user will usually write the relevant content of the first blackboard near the first blackboard. If the user writes other content, he or she will usually write other content in an area farther away from the first blackboard. Moreover, the normal palm length is approximately 20 centimeters, and the user is more likely to write relevant content within the range of palm movement).

[0376] It is understandable that after determining the first module, the area is highlighted so that the user can determine where to write the content to be explained according to the content to be explained. For example, after highlighting the first module, if the content to be explained later is related to the first blackboard writing in the first module, the content to be explained is written in the first module. Otherwise, the content to be explained is written outside the first module to divide a new module.

[0377] S1803: The electronic device generates minutes content corresponding to the first blackboard writing according to the first blackboard writing, the first blackboard writing event and the first voice information.

[0378] In the embodiment of the present application, the electronic device establishes a connection between the first blackboard writing, the first blackboard writing event and the first voice information, and generates the minutes content corresponding to the first blackboard writing.

[0379] In the embodiment of the present application, the electronic device determines the keywords of the first blackboard writing through OCR technology and the first voice information, and integrates the first blackboard writing, the keywords of the first blackboard writing, the first blackboard writing event and the first voice information to obtain the minutes content corresponding to the first blackboard writing.

[0380] S1804. The electronic device responds to the second blackboard writing operation of the user, displays the second blackboard writing, generates a second blackboard writing event, and obtains second voice information corresponding to the second blackboard writing.

[0381] The second blackboard writing operation can be understood as a preset blackboard writing operation performed by the user on the electronic device after the first blackboard writing operation.

[0382] The second blackboard writing is the ink content displayed by the electronic device in response to the user's second blackboard writing operation.

[0383] The second voice information is the electronic device's response to the user's second blackboard operation, real-time recording of the current explanation audio, and voice-to-text conversion of the explanation audio to obtain voice information.

[0384] It is understandable that if the user information triggering the second blackboard writing operation is different from the user information triggering the first blackboard writing operation, the electronic device will distinguish and display the blackboard writings of different users.

[0385] In the embodiment of the present application, the specific implementation method of step S1804 refers to the above step S1801 and will not be repeated here.

[0386] S1805. The electronic device determines whether the second blackboard book matches the first module. If so, execute step S1807; if not, execute S1806.

[0387] In some embodiments of the present application, the electronic device determines whether the second blackboard book matches the first module according to the position of the second blackboard book. If the relative distance between the second blackboard book and the first module exceeds the preset distance, it is determined that the second blackboard book does not match the first module, and S1806 is executed; if the relative distance between the second blackboard book and the first module does not exceed the preset distance, it is determined that the second blackboard book matches the first module, and S1807 is executed.

[0388] For example, a user first writes content 1 on the electronic device and determines that the first module corresponding to the written content 1 is module 1, and then writes content 2 on the electronic device. If the distance between the written content 1 and the first module exceeds the preset distance, it is determined that the written content 2 is not located in module 1, and real-time segmentation is performed based on the position of the written content 2 and the position of module 1.

[0389] In some other embodiments of the present application, the electronic device determines whether the second blackboard writing matches the first module according to the time difference between the first blackboard writing operation and the second blackboard writing operation. If the time difference between the first blackboard writing operation and the second blackboard writing operation exceeds the preset time, it is determined that the second blackboard writing does not match the first module, and S1806 is executed; if the time difference between the first blackboard writing operation and the second blackboard writing operation does not exceed the preset time, it is determined that the second blackboard writing matches the first module, and S1807 is executed.

[0390] For example, based on the above example, if the writing time difference between written content 1 and written content 2 exceeds the preset time, it is determined that written content 2 is not located in module 1, and real-time segmentation is performed based on the location of written content 2 and the location of module 1.

[0391] In some other embodiments of the present application, the electronic device determines whether the second blackboard writing matches the first module based on the keywords of the first blackboard writing and the keywords of the second blackboard writing. If the keywords of the first blackboard writing and the keywords of the second blackboard writing match, it is determined that the second blackboard writing does not match the first module, and S1806 is executed; if the keywords of the first blackboard writing and the keywords of the second blackboard writing do not match, it is determined that the second blackboard writing matches the first module, and S1807 is executed.

[0392] In one possible implementation, the keywords of the first blackboard book and the keywords of the second blackboard book are identified through OCR technology. If there are identical keywords in the keywords of the first blackboard book and the keywords of the second blackboard book, and the proportion of the identical keywords exceeds a preset ratio, it is determined that the keywords of the first blackboard book and the keywords of the second blackboard book match.

[0393] For example, based on the above example, the OCR technology is used to identify written content 1 as relevant information of content 1, and written content 2 as relevant information of content 2. There is no association between content 1 and content 2, so it is determined that written content 2 is not located in module 1, and real-time segmentation is performed based on the location of written content 2 and the location of module 1. The embodiments of the present application do not limit the way in which the electronic device performs region segmentation, the size of the region after segmentation, etc.

[0394] It is understandable that in an educational scenario, users need to write different content and explain different content, so users will input multiple blackboard operations on the electronic device, and the location and content of each blackboard are different. Therefore, for each blackboard writing, it is necessary to determine the relevance of the current blackboard writing with the previous blackboard writing (which can also be described as matching). If it matches, it is determined that the current blackboard writing is associated with the previous blackboard writing, and the module where the current blackboard writing is located is merged into the module of the previous blackboard writing. In this way, by detecting the relevance of the blackboard writing, the blackboard writing is divided into different modules. The same module contains related blackboard writings, and the blackboard writing contents of different modules are different.

[0395] S1806. The electronic device determines the second module according to the second blackboard writing.

[0396] Among them, the second module is the area containing the second blackboard writing.

[0397] In the embodiment of the present application, the specific implementation method of step S1806 refers to the above step S1802 and will not be repeated here.

[0398] S1807. The electronic device generates minutes content corresponding to the second blackboard writing according to the second blackboard writing, the second blackboard writing event and the second voice information.

[0399] In the embodiment of the present application, the specific implementation method of step S1806 refers to the above step S1803 and will not be repeated here.

[0400] S1808. The electronic device receives a closing operation from the user, obtains a meeting file according to the first blackboard writing and the second blackboard writing, and generates meeting minutes according to the minutes content corresponding to the first blackboard writing and the minutes content corresponding to the second blackboard writing.

[0401] Among them, the close operation is a preset operation for the user to exit the blackboard writing.

[0402] Among them, the meeting file is the document obtained based on the content of the blackboard after the user completes the blackboard writing.

[0403] In an embodiment of the present application, when the user has finished explaining or discussing and needs to exit the blackboard writing program, the user inputs a close operation to the electronic device to exit the blackboard writing. In response to the close operation, the electronic device saves the first blackboard writing and the second blackboard writing generated during the blackboard writing process as a conference file. For example, the first blackboard writing and the second blackboard writing are saved as files in CWB format.

[0404] In an embodiment of the present application, if the first blackboard writing and the second blackboard writing are located in the first module, the electronic device integrates the minutes content of the first module according to the minutes content corresponding to the first blackboard writing and the minutes content corresponding to the second blackboard writing in the first module, and determines the minutes content of the first module as the meeting minutes.

[0405] In other embodiments of the present application, if the first blackboard writing is located in the first module and the second blackboard writing is located in the second module, the electronic device determines the minutes content corresponding to the first blackboard writing in the first module as the minutes content corresponding to the first module; the electronic device determines the minutes content corresponding to the second blackboard writing in the second module as the minutes content corresponding to the second module; the electronic device integrates the minutes content corresponding to the first module and the minutes content corresponding to the second module to obtain the meeting minutes.

[0406] For example, save the meeting minutes as a file in JSON format.

[0407] In some other embodiments of the present application, the electronic device stores the meeting files and the meeting minutes in association. For example, a minutes table is established in the database, and the table stores the address of the meeting file 1 and the address of the meeting minutes 1 corresponding to the meeting file 1. Alternatively, the table stores the information of the meeting file 1 and the information of the meeting minutes 1 corresponding to the meeting file 1.

[0408] It is understandable that the above example is based on two blackboard writings in an educational scenario. In actual applications, users can write on the blackboard multiple times. The electronic device determines the correlation between the current blackboard writing and the previous blackboard writing, and then divides different blackboard writings into different modules. The electronic device integrates the minutes content corresponding to each module to obtain a complete meeting minutes.

[0409] Optionally, the electronic device sends each blackboard writing and the minutes content corresponding to each blackboard writing to the server, and the server generates a meeting file and a meeting minutes. The server can also send the generated meeting file and meeting minutes to the electronic device.

[0410] In some examples, after the user indicates to exit and close the blackboard, he may need to view the meeting minutes generated by the electronic device. Fig.18 As shown, after S1808, S1809 may also be included to trigger the electronic device to display the meeting minutes.

[0411] S1809. The electronic device displays the meeting minutes in response to the user's touch operation.

[0412] It is understandable that the electronic device may be the electronic device that generates the meeting minutes in S1801-S1808 above, and after generating the meeting minutes, the electronic device traces back the key points according to the meeting minutes. The electronic device may also be another electronic device that obtains the meeting minutes and meeting documents and traces back the key points according to the meeting minutes and meeting documents.

[0413] In the embodiment of the present application, in the process of reviewing the blackboard writing, the user opens the stored meeting file on the electronic device and first freely reads and reviews the meeting file. After the user has mastered the overall content of the meeting file, if he wants to review a certain content in depth, the user can input a touch operation on the content. In response to the touch operation, the electronic device displays the minutes content related to the content in the meeting minutes.

[0414] In one possible implementation, Fig.19 As shown, the specific implementation steps of step S1809.

[0415] S1900. The electronic device adds the meeting minutes to the meeting files.

[0416] In an embodiment of the present application, the electronic device adds the minutes content corresponding to each module in the meeting minutes to each module, and adds the minutes content corresponding to each blackboard book to each blackboard book in parallel.

[0417] In some implementations, the minutes content is added by overlaying a layer, but the overlay layer is displayed only when the user touches it. Alternatively, the minutes content is connected to the meeting file through a hyperlink.

[0418] For example, a new layer is superimposed on the first blackboard, and the new layer contains the minutes content corresponding to the first blackboard, but the layer only displays the minutes content corresponding to the first blackboard when the user touches near the first blackboard. Alternatively, a hyperlink is added near the first blackboard, and the hyperlink points to the minutes content corresponding to the first blackboard.

[0419] S1901: The electronic device receives a user's touch operation on a first module and displays minutes content corresponding to the first module.

[0420] Among them, the first module is the module that the user touches when reviewing the meeting.

[0421] In the embodiment of the present application, the first module is determined according to the touch point position of the user's touch operation.

[0422] In some embodiments of the present application, the electronic device pops up a hyperlink pointing to the minutes content corresponding to the first module according to the user's touch operation on the first module, and the user clicks the hyperlink to display the minutes content corresponding to the first module.

[0423] In other embodiments of the present application, the electronic device covers and displays a layer of minutes content corresponding to the first module near the first module according to the user's touch operation on the first module.

[0424] S1902: The electronic device receives a user's touch operation on a first blackboard in a first module, and displays the minutes content corresponding to the first blackboard.

[0425] Among them, the first blackboard writing is the blackboard writing content touched by the user when reviewing the meeting.

[0426] In an embodiment of the present application, the first blackboard writing is determined according to the contact point position of the user's touch operation.

[0427] In some embodiments of the present application, the electronic device pops up a hyperlink pointing to the minutes content corresponding to the first blackboard book in the first module according to the user's touch operation on the first blackboard book, and the user clicks the hyperlink to display the minutes content corresponding to the first blackboard book.

[0428] In other embodiments of the present application, the electronic device covers and displays a layer of minutes content corresponding to the first module near the first blackboard according to the user's touch operation on the first blackboard in the first module.

[0429] It is understandable that the above steps S1900-S1902 describe how to review the meeting from the perspective of module to blackboard (from the whole to the details), but in actual use, the user can also directly touch the first blackboard.

[0430] In another possible implementation, the electronic device directly displays the minutes content corresponding to the first blackboard writing in response to the user's touch operation on the first blackboard writing.

[0431] Through the above process, the embodiment of the present application can generate meeting minutes in a structured manner according to the blackboard content in the teaching scenario, and the meeting minutes also include the blackboard content and the blackboard explanation content, which improves the accuracy and quality of the meeting minutes. Moreover, dividing the modules in real time according to the blackboard content and dividing the meeting minutes according to the modules is conducive to the subsequent interactive review of the meeting minutes. In the review process, the meeting documents are displayed in layers from the whole to the details, which can better assist users in understanding the blackboard content and improve the review efficiency.

[0432] It can be understood that the above embodiments describe the method of deploying electronic devices in various business scenarios and in a manner of interaction between multiple electronic devices, such as Fig. 20 As shown, it is a flow chart of another electronic device deployment method provided in an embodiment of the present application, and the method includes the following steps S2001-S2004.

[0433] S2001. The electronic device receives a user's interactive operation on a current conference presentation page and displays the interactive content.

[0434] In an embodiment of the present application, after the electronic device receives the user's interactive operation on the current conference presentation page and displays the interactive content, it also includes: generating an interactive event.

[0435] In some examples, the electronic device receives a user's interactive operation on the current conference presentation page, displays the interactive content, and generates an interactive event.

[0436] In some examples of the present application, the interactive operation includes a user's annotation operation on the first content, and the current conference presentation page includes the first content; the interactive content includes a first annotation; the first annotation is used to mark and annotate the first content; and the interactive event includes a first annotation event.

[0437] In a possible implementation, the electronic device receives a user's interactive operation on a current conference presentation page, displays interactive content, and generates an interactive event, including: the electronic device receives a user's annotation operation on a first content in the current conference presentation page, displays a first annotation, and generates a first annotation event.

[0438] The first annotation event includes one or more of the following: the first content, the first annotation, the timestamp of the first annotation, the location information of the first annotation, and the user information triggering the first annotation.

[0439] Optionally, before the electronic device receives the user's annotation operation on the first content in the current conference display page, displays the first annotation, and generates a first annotation event, the electronic device obtains a conference file, pre-processes the conference file, and divides the conference file into different modules.

[0440] The conference file is a file to be displayed in the current conference; the first annotation event also includes module information where the first annotation is located.

[0441] In other examples of the present application, the interactive operation includes the user's first blackboard writing operation; the interactive content includes the first blackboard writing, which is the ink content displayed in the current conference presentation page by the electronic device in response to the user's first blackboard writing operation; and the interactive event includes the first blackboard writing event.

[0442] In a possible implementation, the electronic device receives a user's interactive operation on a current conference presentation page, displays interactive content, and generates an interactive event, including: the electronic device receives a user's first blackboard operation on the current conference presentation page, displays the first blackboard, and generates a first blackboard event.

[0443] The first blackboard writing event includes one or more of the following: the first blackboard writing, the timestamp of the first blackboard writing, the location information of the first blackboard writing, the user information triggering the first blackboard writing, the input device type for inputting the first blackboard writing or the contact type of the first blackboard writing.

[0444] Optionally, the electronic device may also receive a second blackboard writing operation from the user, display a second blackboard writing, and generate a second blackboard writing event.

[0445] The second blackboard writing operation is the next blackboard writing operation after the user performs the first blackboard writing operation.

[0446] Optionally, the electronic device may also determine the first module based on the first blackboard writing; the electronic device determines whether the second blackboard writing matches the first module; if not, determines the second module based on the second blackboard writing; if matching, determines that the second blackboard writing is located in the first module.

[0447] Among them, the first module is the area containing the first blackboard in the current conference display page.

[0448] S2002. The electronic device collects voice data corresponding to the interactive content.

[0449] In some examples, the electronic device responds to the user's interactive operation on the current conference presentation page by starting recording from the start timestamp of the interactive operation, and ends recording when it recognizes that the recorded audio does not contain relevant information of the interactive operation (such as interactive content) to obtain voice data corresponding to the interactive content.

[0450] S2003. The electronic device converts the voice data into text data.

[0451] In some examples, the electronic device converts voice data into text through automatic speech recognition technology, and determines the converted text as text data.

[0452] In other examples, the electronic device converts voice data into text through automatic speech recognition technology, and performs semantic analysis and keyword capture on the converted text to obtain text data.

[0453] S2004. The electronic device generates minutes of the current meeting, where the minutes include interactive content and text data.

[0454] In the embodiment of the present application, the meeting minutes also include interaction events.

[0455] In some examples of the present application, the electronic device generates minutes of the current meeting including: the electronic device associates the first content, the first annotation, the first annotation event, and the text data corresponding to the first annotation to generate minutes content corresponding to the first annotation; the electronic device obtains minutes content corresponding to each module based on the minutes content corresponding to the first annotation in each module; the electronic device obtains the minutes of the current meeting based on the minutes content corresponding to each module.

[0456] Optionally, after the electronic device generates the minutes of the current meeting, it also includes: the electronic device responds to the user's touch operation on the module where the first content is located, and displays the minutes content corresponding to the module where the first content is located; the electronic device responds to the user's touch operation on the first annotation, and displays the minutes content corresponding to the first annotation.

[0457] In other examples of the present application, an electronic device generates minutes of a current meeting, including: the electronic device associates a first blackboard writing, a first blackboard writing event, and text data corresponding to the first blackboard writing, and generates meeting minutes corresponding to the first blackboard writing; the electronic device associates a second blackboard writing, a second blackboard writing event, and text data corresponding to the second blackboard writing, and generates meeting minutes corresponding to the second blackboard writing; the electronic device obtains meeting minutes corresponding to the current meeting based on the meeting minutes corresponding to the first blackboard writing and the meeting minutes corresponding to the second blackboard writing.

[0458] Optionally, after the electronic device generates the minutes of the current meeting, the method further includes: the electronic device displays the minutes content corresponding to the first blackboard writing in response to the user's touch operation on the first blackboard writing.

[0459] In another possible implementation, the electronic device obtains the minutes content corresponding to each module according to the minutes content corresponding to each blackboard in each module; the electronic device obtains the minutes of the current meeting according to the minutes content corresponding to each module.

[0460] In another possible implementation, the electronic device displays the minutes content corresponding to the module where the first blackboard is located in response to the user's touch operation on the module where the first blackboard is located; the electronic device displays the minutes content corresponding to the first blackboard in response to the user's touch operation on the first blackboard.

[0461] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

[0462] The present application is an embodiment that can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0463] like Fig.21 2100 is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. The electronic device 2100 can be used to implement the methods described in the above method embodiments. Exemplarily, the electronic device 2100 may include: a processing module 2101, an acquisition module 2102, and a display module 2103.

[0464] The processing module 2101 is used to support the electronic device 2100 to execute Figures 7 to 20 The processing function of any one of the items.

[0465] The acquisition module 2102 is used to execute the support electronic device 2100 to execute Figures 7 to 20 The acquisition function of any item in. For example, obtaining the currently displayed image, etc.

[0466] The display module 2103 may be used to display a screen according to the display driving data, etc. And / or, the display module 2103 may also be used to support the electronic device 2100 to perform other display operations performed by the electronic device in the embodiment of the present application.

[0467] Optional, Fig.21 The electronic device 2100 shown may also include a communication module ( Fig.21 (not shown), the communication module is used to support the electronic device 2100 to execute the steps of communication between the electronic device and other devices in the embodiment of the present application.

[0468] Optional, Fig.21 The electronic device 2100 shown may also include a storage module ( Fig.21 (not shown in the figure), the storage module stores a program or instruction. When the processing module 2101 executes the program or instruction, Fig.21 The electronic device 2100 shown can execute the method shown in the above method embodiment.

[0469] Fig.21 The technical effects of the electronic device 2100 shown can refer to the technical effects of the method described in the above method embodiment, and will not be repeated here. Fig.21 The processing module 2101 involved in the electronic device 2100 shown can be implemented by a processor or a processor-related circuit component, which can be a processor or a processing module. The communication module can be implemented by a transceiver or a transceiver-related circuit component, which can be a transceiver or a transceiver module. The display module 2103 can be implemented by display screen-related components.

[0470] The present application also provides a chip system, such as Fig. 22 As shown, the chip system 2200 includes at least one processor 2201 and at least one interface circuit 2202. As an example, when the chip system 2200 includes one processor and one interface circuit, the one processor may be Fig. 22 The processor 2201 shown in the solid line frame (or the processor 2201 shown in the dotted line frame) may be Fig. 22 The interface circuit 2202 shown in the solid line frame (or the interface circuit 2202 shown in the dotted line frame). When the chip system 2200 includes two processors and two interface circuits, the two processors include Fig. 22 The processor 2201 shown in the solid line frame and the processor 2201 shown in the dotted line frame, the two interface circuits include Fig. 22 The interface circuit 2202 is shown in the solid line box and the interface circuit 2202 is shown in the dotted line box.

[0471] There is no limitation on this.

[0472] The processor 2201 and the interface circuit 2202 can be interconnected via a line. For example, the interface circuit 2202 can be used to receive a signal. For another example, the interface circuit 2202 can be used to send a signal to another device (such as the processor 2201). Exemplarily, the interface circuit 2202 can read an instruction stored in a memory and send the instruction to the processor 2201. When the instruction is executed by the processor 2201, the various steps in the above embodiment can be executed. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0473] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0474] Optionally, the chip system may also include a memory ( Fig. 22 As shown in the figure, the memory may be one or more, and the memory may be integrated with the processor or may be separately arranged from the processor, which is not limited in the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or may be arranged on different chips, and the present application does not specifically limit the type of memory and the arrangement of the memory and the processor.

[0475] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0476] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0477] An embodiment of the present application further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the above method embodiment.

[0478] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0479] An embodiment of the present application provides a computer program product, which includes: a computer program or instructions, when the computer program or instructions are executed on a computer, the computer executes the method described in the above method embodiment.

[0480] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the device executes the methods in the above-mentioned method embodiments.

[0481] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0482] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a compact disc read-only memory (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0483] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0484] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The various embodiments can be combined with each other or referenced to each other without conflict. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0485] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0486] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0487] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0488] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating meeting minutes, characterized in that: Applied to electronic equipment, the method comprises: Receive user interaction on the current conference display page and display the interactive content; Collecting voice data corresponding to the interactive content; Converting the voice data into text data; A meeting minutes of the current meeting is generated, wherein the meeting minutes includes the interaction content and the text data.

2. The method for generating meeting minutes according to claim 1, characterized in that: After receiving the user's interactive operation on the current conference display page and displaying the interactive content, the method further includes: Generate an interaction event; the meeting minutes also include the interaction event.

3. The method for generating meeting minutes according to claim 2, characterized in that: The interactive operation includes a user's annotation operation on the first content, and the current conference display page includes the first content; the interactive content includes a first annotation; and the first annotation is used to mark and annotate the first content; The interaction event includes a first marking event; The receiving of the user's interactive operation on the current conference display page, displaying the interactive content, and generating the interactive event includes: receiving a user's annotation operation on the first content in the current conference display page, displaying the first annotation, and generating the first annotation event; The first annotation event includes one or more of the following: the first content, the first annotation, a timestamp of the first annotation, location information of the first annotation, and user information that triggers the first annotation.

4. The method for generating meeting minutes according to claim 3, characterized in that: Before receiving a user's annotation operation on the first content in the current conference display page, displaying a first annotation, and generating a first annotation event, the method further includes: Obtaining a conference file; the conference file is a file to be displayed in the current conference; The conference file is pre-processed to divide the conference file into different modules; the first annotation event also includes module information where the first annotation is located.

5. The method for generating meeting minutes according to claim 3 or 4, characterized in that: The generating of the minutes of the current meeting includes: Associating the first content, the first annotation, the first annotation event, and text data corresponding to the first annotation to generate minutes content corresponding to the first annotation; According to the minutes content corresponding to the first annotation in each module, obtain the minutes content corresponding to each module; According to the minutes content corresponding to each module, the minutes of the current meeting are obtained.

6. The method for generating meeting minutes according to any one of claims 3 to 5, characterized in that: After generating the minutes of the current meeting, the method further includes: In response to a user's touch operation on the module where the first content is located, displaying the minutes content corresponding to the module where the first content is located; In response to a user's touch operation on the first annotation, the minutes content corresponding to the first annotation is displayed.

7. The method for generating meeting minutes according to claim 2, characterized in that: The interactive operation includes a first blackboard writing operation of a user, and the interactive content includes a first blackboard writing, where the first blackboard writing is ink content displayed by the electronic device in the current conference display page in response to the first blackboard writing operation of the user; The interactive event includes a first blackboard writing event; The receiving of the user's interactive operation on the current conference display page, displaying the interactive content, and generating the interactive event includes: receiving a first blackboard writing operation of a user on the current conference display page, displaying the first blackboard writing, and generating the first blackboard writing event; The first blackboard writing event includes one or more of the following: the first blackboard writing, the timestamp of the first blackboard writing, the location information of the first blackboard writing, the user information triggering the first blackboard writing, the input device type for inputting the first blackboard writing, and the contact type of the first blackboard writing.

8. The method for generating meeting minutes according to claim 7, characterized in that: The method further comprises: A second blackboard writing operation of the user on the current conference display page is received, a second blackboard writing is displayed, and a second blackboard writing event is generated; the second blackboard writing operation is the next blackboard writing operation after the user performs the first blackboard writing operation.

9. The method for generating meeting minutes according to claim 8, characterized in that: The generating of the minutes of the current meeting includes: Associating the first blackboard writing, the first blackboard writing event and the text data corresponding to the first blackboard writing, and generating the meeting minutes corresponding to the first blackboard writing; Associating the second blackboard writing, the second blackboard writing event and the text data corresponding to the second blackboard writing, and generating the meeting minutes corresponding to the second blackboard writing; The minutes of the current meeting are obtained according to the minutes of the meeting corresponding to the first blackboard writing and the minutes of the meeting corresponding to the second blackboard writing.

10. The method for generating meeting minutes according to claim 8, characterized in that: The method further comprises: Determine a first module according to the first blackboard writing; the first module is an area of ​​the current conference display page containing the first blackboard writing; Determine whether the second blackboard writing matches the first module; if not, determine the second module based on the second blackboard writing; if matching, determine that the second blackboard writing is located in the first module.

11. The method for generating meeting minutes according to claim 9, characterized in that: After generating the minutes of the current meeting, the method further includes: In response to the user's touch operation on the first blackboard writing, the minutes content corresponding to the first blackboard writing is displayed.

12. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the method as claimed in any one of claims 1 to 11.

13. A chip system, characterized in that: It includes at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and the at least one processor executes the instructions, and the at least one processor executes the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images