Information presentation method, device, storage medium, and program product

Abstract

The present disclosure discloses an information presentation method, device, storage medium and program product, and belongs to the technical field of computers. The method comprises: displaying a first interface, the first interface comprising first audio information, a first identification element of the first audio information, and at least one matter element associated with the first audio information. In response to an operation acting on the first identification element, text information corresponding to the first identification element is displayed, the text information being a text segment or extended information obtained by recognizing the first audio information. In response to an operation acting on a target matter element, a matter corresponding to the target matter element is displayed, the matter being an event that needs to be paid attention to by a user and obtained by summarizing the first audio information. By integrating audio information, identification elements and pre-summarized matter elements, and cooperating with a responsive operation mechanism, the interactivity, efficiency and user experience of information presentation can be improved, the burden of information acquisition can be reduced, the prominent presentation of key information can be enhanced, and the information acquisition efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an information presentation method, device, storage medium, and program product. Background Technology

[0002] The automatic generation of information minutes is a typical high-value downstream application scenario of audio transcription (ASR) technology. With the popularization of remote work, online meetings, and all-weather audio acquisition devices, how to efficiently present the information extracted from audio information has gradually become a key requirement for improving collaboration efficiency and information traceability. Summary of the Invention

[0003] This disclosure provides an information presentation method, apparatus, storage medium, and program product, which can improve the efficiency of information acquisition. The technical solution is as follows: Firstly, an information presentation method is provided, the method comprising: The first interface is displayed, which includes first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information. In response to an operation performed on the first identifier element, text information corresponding to the first identifier element is displayed, wherein the text information is a text fragment or extended information obtained by recognizing the first audio information; In response to an operation performed on a target item element, the item corresponding to the target item element is displayed, the item being an event that the user needs to pay attention to, summarized based on the first audio information.

[0004] Secondly, an information presentation device is provided, the device comprising: A first display module is used to display a first interface, the first interface including first audio information and a first identifier element of the first audio information, and at least one item element associated with the first audio information; The second display module is used to display text information corresponding to the first identifier element in response to an operation performed on the first identifier element. The text information is a text fragment or extended information obtained by recognizing the first audio information. The third display module is used to respond to the operation applied to the target item element and display the item corresponding to the target item element, wherein the item is an event that the user needs to pay attention to, which is summarized based on the first audio information.

[0005] Thirdly, an electronic device is provided, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the information presentation method described in the first aspect above.

[0006] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the information presentation method described in the first aspect.

[0007] Fifthly, a computer program product is provided, which, when run on a computer device, causes the computer device to execute the information presentation method described in the first aspect.

[0008] In this disclosure, a first interface is displayed, comprising first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information. In response to an operation applied to the first identifier element, text information corresponding to the first identifier element is displayed; the text information is a text fragment or extended information obtained by recognizing the first audio information. In response to an operation applied to a target item element, the item corresponding to the target item element is displayed; the item is an event that the user needs to focus on, summarized based on the first audio information. By integrating audio information, identifier elements, and pre-summarized item elements in the first interface, and in conjunction with a responsive operation mechanism, rapid location of audio content, seamless switching of text fragments or extended information, and precise guidance on key items can be achieved. This effectively improves the interactivity, efficiency, and user experience of information presentation, reduces the burden of information acquisition, and enhances the prominent presentation of key information, thereby improving the efficiency of information acquisition for users. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the structure of an information generation system provided in an embodiment of this disclosure; Figure 2 This is a structural diagram of a text fragment provided in an embodiment of this disclosure; Figure 3 This is an architecture diagram of an information generation system provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of a first interface provided in an embodiment of this disclosure; Figure 5 This is a flowchart of an information presentation method provided in an embodiment of this disclosure; Figure 6 This is a timing diagram of a multi-dimensional retrieval method for an audio assistant provided in an embodiment of this disclosure; Figure 7 This is a flowchart illustrating the triggering of differentiated extended information provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram of a structured extended information data model provided in an embodiment of this disclosure; Figure 9 This is a diagram illustrating a many-to-many relationship between extended information, audio information, and text fragments, provided in this embodiment. Figure 10 This is a schematic diagram of the structure of an information presentation device provided in an embodiment of this disclosure; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0010] It should be understood that, when used in this disclosure and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0011] The term "multiple" as used in this disclosure refers to two or more. In the description of this disclosure, unless otherwise stated, " / " signifies "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0012] To facilitate a clear description of the technical solutions disclosed herein, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0013] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0014] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0015] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0016] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0017] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0018] This disclosure provides an information presentation method. In this method, a first interface is displayed, including first audio information, a first identifier element for the first audio information, and at least one item element associated with the first audio information. In response to an operation applied to the first identifier element, text information corresponding to the first identifier element is displayed. The text information is a text fragment or extended information obtained by recognizing the first audio information. In response to an operation applied to a target item element, the item corresponding to the target item element is displayed. The item is an event that the user needs to focus on, summarized based on the first audio information. By integrating audio information, identifier elements, and pre-summarized item elements in the first interface, and using a responsive operation mechanism, rapid location of audio content, seamless switching of text fragments or extended information, and precise guidance on key items can be achieved. This effectively improves the interactivity, efficiency, and user experience of information presentation, reduces the burden of information acquisition, and enhances the prominent presentation of key information, thereby improving the efficiency of information acquisition for users.

[0019] Figure 1 This is a schematic diagram of the structure of an information generation system provided in an embodiment of the present disclosure. The information generation system 10 may include a first device 101, a second device 102 and a third device 103.

[0020] The first device 101 can communicate with the second device 102 and the third device 103 via wired or wireless connection.

[0021] The second device 102 is used to collect audio information; transcribe the audio information to obtain at least one text fragment; and send the at least one text fragment to the first device 101.

[0022] In one scenario, if the second device 102 fails to detect an audio signal for a first preset duration, it can send at least one text segment corresponding to the acquired audio information to the first device 101 to instruct the first device 101 to generate extended information and / or event information. The first preset duration can be pre-set. For example, the first preset duration can be set to 4 minutes, 5 minutes, or 6 minutes, etc.

[0023] Optionally, before the second device 102 sends at least one text segment to the first device 101, it can determine the time interval between the current time and the time of the last text segment transmission; if the time interval is greater than or equal to a second preset time interval, the second device 102 sends the text segment corresponding to the audio information to the first device 101; if the time interval is less than the second preset time interval, the second device 102 accumulates the text segments so that at least one accumulated text segment is sent when the time interval is greater than or equal to the second preset time interval.

[0024] Optionally, the text fragments have a fragment structure. For example, such as... Figure 2 As shown, the fragment structure of a text segment may include one or more of the following: transcribed text (txt) (i.e., text segment), time information (start_time / end_time), speaker identifier (speaker), text segment identifier (transcript_id), whether the wearer spoke (is_current_user), and end flag (is_final). Here, the text segment is the text content transcribed from the collected audio information; the time information is the acquisition time of the audio information corresponding to the text segment, which may include a start time (e.g., milliseconds) and an end time; the speaker identifier is used to identify the user identity corresponding to the text segment; for example, the speaker identifier indicates which user's audio information was transcribed from the text segment; the text segment identifier can be an encoding or a number used to mark the text segment; whether the wearer spoke indicates whether the text segment is spoken by the user wearing the second device 102; and the end flag indicates whether the text segment is the last text segment; for example, if the text segment is the last text segment, is_final=true, otherwise is_final=false.

[0025] Optionally, at least one text segment may also include an identifier for corresponding audio information to indicate audio information associated with at least one text segment.

[0026] In one scenario, the second device 102 can identify whether the collected audio information belongs to the user wearing the second device 102. If it does, it can be marked using the aforementioned field indicating whether the wearer is speaking. To distinguish audio information from that of the wearer, audio information collected by the second device 102 belonging to a non-wearer can also be marked using the aforementioned speaker identifier. In this way, a basis can be provided for determining the speaker's identity during subsequent evidence re-identification, thereby improving the credibility of the evidence re-identification.

[0027] In one scenario, the second device 102 may collect audio information in multiple modes, such as conference mode and recording mode. In conference mode, the audio information collection is triggered and started by the user in a conference scenario. In the conference scenario, the second device 102 activates the microphone to collect the audio information of the user wearing the second device 102, and simultaneously collects the audio information of non-wearers. After the conference ends, the microphone is turned off and audio information collection ceases. In recording mode, the audio information collection is triggered and started by the user in a scenario where the user is recording information. In the recording scenario, the second device 102 can keep the microphone on for an extended period of time to collect only the audio information of the user wearing the second device 102.

[0028] In one scenario, the second device 102 may include an ASR module (which could be an offline ASR service module). Thus, after acquiring audio information, the second device 102 can use the ASR module to transcribe the audio information to obtain the corresponding text fragments.

[0029] In another scenario, the second device 102 may not include an ASR module. In this case, after obtaining the audio information, the second device 102 can directly send the audio information to the first device 101, which will then transcribe the audio information to obtain the corresponding text segment.

[0030] Optionally, the audio information sent by the second device 102 to the first device 101 may include an audio information identifier.

[0031] The first device 101 generates extended information and / or event information based on at least one text fragment. For example, the first device 101 can process the at least one text fragment using a large language model to generate extended information and / or event information.

[0032] In one scenario, extended information is a set of information obtained by identifying and organizing the at least one text fragment according to a predefined standardized text structure. This includes one or more of the following: extended information identifier, acquisition mode, summary, time, and keywords. The extended information identifier is used to identify and distinguish different extended information; it can generally be a title representing the main content of the multiple text fragments. The summary is a concise text summarizing the core content, theme, and characteristics of the discussion in at least one text fragment, allowing users to quickly understand what the at least one text fragment is about. The time is the acquisition time (including start and end times) of the audio information corresponding to the text fragment. Keywords are words or phrases characterizing the core content of the multiple text fragments, enabling users to match, retrieve, and identify information.

[0033] In one scenario, event information may include one or more of the following: decision items, to-do items, and notice items. Event information can be obtained by performing information identification, extraction, and separation based on the at least one text fragment to obtain decision items, to-do items, and notice items. Decision items are information such as confirmed viewpoints, resolutions, and judgment results extracted and separated from the at least one text fragment; to-do items are tasks that the user needs to perform subsequently, extracted and separated from the at least one text fragment; and notice items are matters that the user needs to pay attention to or be mindful of, extracted and separated from the at least one text fragment.

[0034] Optionally, the first device 101 may include a recording module (i.e., a recording service), a structured information extraction module (i.e., a structured minutes extraction service), a storage module (such as a relational database), and a retrieval module. For example, such as... Figure 3 As shown, the recording module can push one or more text fragments to the structured information extraction module, and send one or more text fragments to the storage module for storage. The structured information extraction module can generate extended information and / or events based on these text fragments and send them back to the recording module. The recording module associates and stores the text fragments with audio information and extended information, and builds a full-text index based on the text fragments and extended information to support user queries. The retrieval module can perform retrieval based on the received query request.

[0035] In some cases, after generating extended information, the first device 101 can actively send the extended information, the audio information associated with the extended information, at least one text fragment, and event information to the third device 103. Alternatively, the first device 101 can receive an information acquisition request sent by the third device 103 and send the extended information, the audio information associated with the extended information, at least one text fragment, and event information to the third device 103.

[0036] The third device 103 is used to receive extended information, audio information associated with the extended information, at least one text fragment, and event information sent by the first device 101, and to display a first interface, such as... Figure 4 As shown, the first interface includes first audio information and a first identifier element for the first audio information, and at least one item element associated with the first audio information. In response to an operation performed on the first identifier element or the item element, information corresponding to that element is displayed. In response to a user's query operation, a query request is sent to the first device 101; the target query result sent by the first device 101 is received, and the target query result is played and / or displayed. For example, the query operation can be triggered manually by the user or through an audio assistant.

[0037] It should be noted that in the embodiments of this disclosure, the first device 101, the second device 102, and the third device 103 may be the same device or different devices.

[0038] For example, when the first device 101, the second device 102, and the third device 103 are different devices, the first device 101 may be a computer device (such as a cloud server), the second device 102 may be a smart wearable device (such as smart glasses), and the third device 103 may be a mobile device (such as a smartphone). When all three are the same device, the device may be a smartphone or smart glasses with a display function. When the first device 101 and the second device 102 are the same device, the device may be smart glasses without a display function, and the third device 103 may be a smartphone. When the first device 101 and the third device 103 are the same device, the device may be a smartphone, and the second device 102 may be smart glasses. When the second device 102 and the third device 103 are the same device, the device may be smart glasses with a display function or a smartphone, and the first device 101 may be a cloud server.

[0039] Figure 5 This is a flowchart of an information presentation method provided in an embodiment of this disclosure. See also... Figure 5 The method may include the following steps: Step 501: The electronic device displays a first interface, which includes first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information.

[0040] For example, the electronic device could be smart glasses with a display or a smartphone.

[0041] like Figure 4 As shown, the first interface presents the first audio information, event elements, and extended information.

[0042] The first audio information consists of audio data collected by users in different application scenarios. It can be presented as multiple audio segments, categorized and displayed under different application scenarios. A categorization identifier indicates the category of the application scenario to which the first audio information belongs. For example, audio information collected in a meeting scenario will be categorized and displayed under the meeting scenario category, allowing users to quickly identify the audio information through scenario classification, thus improving information browsing and retrieval efficiency.

[0043] The first identifier element of the first audio information marks the audio segment of the first audio information, so as to mark that an audio segment was recorded in a certain time period, and the first audio information or the corresponding text segment can be triggered to open through the first identifier element.

[0044] The event element is used to identify events that require user attention, such as decision items, to-do items, and precautions, extracted from the audio information. For a detailed explanation, please refer to the relevant instructions above.

[0045] By displaying the first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information, users can intuitively view audio information in various scenarios, as well as at least one item associated with the first audio information. They can also view the detailed content of the audio information by triggering the first identifier element. This not only improves the efficiency of users obtaining information but also provides an entry point for information tracing and subsequent interaction.

[0046] Step 502: In response to the operation applied to the first identifier element, the electronic device displays the text information corresponding to the first identifier element. The text information is a text fragment or extended information obtained by recognizing the first audio information.

[0047] For example, the operation performed on the first identifier element can be a click, a long press, or a swipe. For example, the text information corresponding to the first identifier element can be displayed in the second interface. For example, the second interface can be a pop-up window, a secondary interface, or a replacement display interface for the first interface.

[0048] The text information corresponding to the first identifier element is a text fragment obtained by processing the first audio information.

[0049] By manipulating the first identifier element, users can locate a specific audio segment corresponding to the first audio information and view the corresponding text segment or extended information, making it convenient for users to view relevant information.

[0050] In some cases, electronic devices can also display one or more audio elements, event elements, etc., associated with text information. Audio elements are used to trigger the playback of audio information associated with the text information, while event elements are used to trigger the display of events associated with the text information.

[0051] In this way, users can trace back to audio information and events associated with the text information, making the text information viewable and verifiable, thus improving the credibility of the information.

[0052] Step 503: The electronic device responds to the operation applied to the target item element by displaying the item corresponding to the target item element, which is an event that the user needs to pay attention to, summarized based on the first audio information.

[0053] For example, the items corresponding to the target item element can be displayed in a third interface, which can be a pop-up window, a secondary interface, etc.

[0054] The target item element is the item element that the user wants to view.

[0055] By displaying the target information, users can quickly learn about the relevant events without having to read detailed text or audio clips, which can greatly save information acquisition time and improve information acquisition efficiency.

[0056] In some cases, electronic devices respond to user query requests; identify keywords in the query request, and search for information in an extended information set based on the keywords; when a first information set matching the keywords is found, the target query result is generated based on the information set; when no first information set matching the keywords is found, the second information set is obtained by searching for text fragments based on the keywords; the target query result is generated based on the second information set; and the target query result is displayed.

[0057] Keywords are obtained by performing keyword extraction on the query request. For example, if the query request is "What were the key points of yesterday's meeting?", the keywords could be "yesterday", "meeting", and "key points".

[0058] For example, electronic devices can perform full-text searches in natural language or Boolean mode. For instance, full-text search can use MySQL's MATCH-AGAINST search.

[0059] The first information set is the result obtained from querying the extended information set.

[0060] The second set of information consists of the results obtained from querying the set of text fragments.

[0061] By prioritizing searches within extended information and only performing a full-text search if no results are found, traversing large amounts of full text can be avoided, thus improving retrieval efficiency. Furthermore, full-text searches reduce the risk of missed results, ensuring accuracy. This hierarchical retrieval approach balances performance and completeness.

[0062] In other cases, the electronic device responds to the user's query request; determines filtering parameters and retrieval parameters based on the query request, the filtering parameters including one or more of time range and keywords, and the retrieval parameters including one or more of title, description, summary, decision items, and to-do items; determines a third information set from the extended information set based on the filtering parameters; determines a fourth information set from the third information set based on the retrieval parameters; and generates the target query results based on the fourth information set.

[0063] The fourth information set consists of information with the same dimensions as the search parameters. For example, if the query request is "What were the tasks for yesterday's meeting?", the second information set obtained based on the query request could include "Perform regression testing", "Go live tomorrow night at 11:30", "Test team on night shift", etc. Optionally, the fourth information set can be sorted and quotated based on conditions such as time, information length, and the number of search results (e.g., 5, 10, etc.).

[0064] Targeted retrieval allows for the precise and efficient extraction of specific information needed by users from a vast amount of expanded information, avoiding information redundancy, reading burden, and computational overhead caused by returning entire summaries at once. This improves retrieval efficiency.

[0065] Optionally, after the electronic device generates the target query results, an audio assistant can also read the target query results aloud.

[0066] For example, Figure 6 This is a timing diagram of the multidimensional retrieval of the audio assistant provided in the embodiments of this disclosure.

[0067] like Figure 6As shown, the user inputs the search audio "What were the tasks for yesterday's meeting?" into the audio assistant. After receiving the search audio, the audio assistant identifies and parses the semantics of "What were the tasks for yesterday's meeting?" to obtain key parameters. The audio assistant sends a query request to the retrieval service (i.e., the retrieval module mentioned above), which includes the key parameters. The retrieval service determines the filtering parameters and retrieval parameters based on the query request. Based on the filtering parameters, the retrieval service performs a search in the database using a MATCH-AGAINST + time window + keyword search method to obtain a third information set. Then, based on the retrieval parameters, it retrieves a fourth information set from the third information set. The retrieval service sends the fourth information set to the audio assistant. The audio assistant receives the fourth information set and replies with the target query results in a spoken format, as well as displaying text fragments associated with the fourth information set.

[0068] In one scenario, before the electronic device displays the first interface, it can identify the first audio information to obtain at least one text fragment corresponding to the first audio information, and extended information corresponding to the first audio information. The extended information includes at least one or more of the title, summary, time, and keywords corresponding to the first audio information; and associate and store the at least one text fragment with the extended information and the first audio information.

[0069] By identifying the first audio information to obtain at least one text fragment and extended information, rich structured data can be generated, providing a data foundation for user interaction. Titles, summaries, keywords, and other data can be used independently for quick previewing, reducing the user's reading burden. Furthermore, by associating and storing at least one text fragment with extended information and the first audio information, users can access other devices from any end, enabling flexible navigation and knowledge linking across audio information and fragments, laying the data foundation for cross-conference topic aggregation and complete evidence chain tracing.

[0070] By associating and storing at least one text fragment with extended information and first audio information, a connection is formed between the data, which provides underlying support for subsequent bidirectional traceability and can eliminate information silos.

[0071] Optionally, the operation of the electronic device to identify the first audio information and obtain at least one text segment corresponding to the first audio information and extended information corresponding to the first audio information can be: identifying the first audio information based on the target voiceprint to obtain at least one text segment corresponding to the first audio information; generating extended information corresponding to the first audio information based on at least one text segment.

[0072] The target voiceprint can be pre-recorded. For example, the target voiceprint can be the voiceprint of a user using a second device. There is a correspondence between the target voiceprint and the user identifier, and this correspondence can be pre-set.

[0073] In this way, the wearer's and others' statements can be accurately distinguished based on text fragments, providing fine-grained evidence for the attribution of subsequent tasks (such as which were promised by the wearer) and the tracing of responsibility, thus improving the accuracy of evidence tracing. In addition, since one or more text fragments contain complete dialogue flow information and semantic units carrying rich context and metadata, relatively accurate first extended information can be generated based on these text fragments.

[0074] In some implementations, the operation of an electronic device generating extended information corresponding to first audio information based on at least one text fragment can be as follows: obtaining first extended information from a historical extended information set based on at least one text fragment; generating extended information corresponding to first audio information based on at least one text fragment and the first extended information.

[0075] If the historical extended information set contains first extended information, it indicates that at least one text fragment corresponds to the same topic as the meeting corresponding to the first extended information, meaning that they are different meetings with the same topic. Second extended information can be generated based on the audio transcription sequence and the first extended information, thus aggregating multiple meetings into extended information for a single topic, thereby improving the efficiency of extended information generation and the efficiency of subsequent information access for users.

[0076] In this way, extended information on the same topic can be intelligently merged into a single extended information item, allowing users to view all the content of multiple related meetings through a single extended information item. This not only eliminates information silos and enables dynamic updates of extended information, but also improves the intelligence of extended information generation and the user experience.

[0077] Optionally, the first device can obtain the topic of at least one text segment and the topic of each piece of extended information in the historical extended information set through a large language model; determine the similarity between the topic of at least one text segment and the topic of each piece of extended information; and determine the extended information with the highest similarity and greater than the similarity threshold as the first extended information.

[0078] In this way, the extended information (i.e., the first extended information) related to at least one text fragment can be determined more accurately.

[0079] In this embodiment of the disclosure, the operation of the electronic device to generate extended information based on at least one text fragment is related to the application scenario to which the first audio information corresponding to at least one text fragment belongs.

[0080] Optionally, the operation of the electronic device to generate extended information corresponding to the first audio information based on at least one text fragment can include the following two methods.

[0081] The first method: When the application scenario of the first audio information is a conference scenario, the electronic device, upon acquiring a text segment sequence, generates second extended information based on the text segment sequence if the text segment sequence is the first text segment sequence acquired in the conference scenario; if the text segment sequence is not the first text segment sequence, updates the second extended information based on the text segment sequence; and if the text segment sequence is the last text segment sequence of the current conference, determines the second extended information as the extended information corresponding to the first audio information; wherein, the text segment sequence includes one or more text segments.

[0082] Since meetings are typically lengthy, generating extended information after capturing the complete audio data could require processing extremely long texts at once, placing a huge strain on computing power and memory. Furthermore, users would have to wait an exceptionally long time, failing to meet the real-time preview requirements of extended information and negatively impacting user experience. Therefore, a segmented upload and breakpoint correction approach can be used to generate extended information corresponding to the first audio data. This allows users to view text snippets in real time, reducing the initial viewing time; and segmented upload only requires calculating the text snippets with anomalies, eliminating the need to recalculate the entire document, thus improving computational efficiency.

[0083] In some implementations, the operation of an electronic device generating second extended information based on an audio transcription sequence can be as follows: obtaining first extended information from a set of historical extended information based on the audio transcription sequence; and generating second extended information based on the audio transcription sequence and the first extended information.

[0084] For example, semantic information of the title and / or summary can be extracted from the audio transcription sequence, and then the first extended information can be determined from the historical extended information.

[0085] In this way, extended information on the same topic can be intelligently merged into a single extended information, thereby achieving extended information integration. This eliminates information silos, enables dynamic updates of extended information, and improves the intelligence of extended information generation.

[0086] In some implementations, the electronic device determines that the audio transcription sequence is the first audio transcription sequence of the meeting if the audio transcription sequence includes a start audio information marker; and / or, the first device determines that the audio transcription sequence is the last audio transcription sequence of the meeting if the last text segment in the audio transcription sequence has a preset marker.

[0087] Preset flags can be set in advance. For example, a preset flag can be an audio message end flag (i.e., is_final=true).

[0088] By introducing a preset marker as an explicit signal for the end of a meeting, the system can clearly determine the start and end times of the meeting, solving the problem of not being able to determine the end in streaming scenarios. This ensures that the triggering time for generating extended information is accurate and reliable, avoiding logical errors caused by premature or delayed fixing, and guaranteeing the accuracy and stability of extended information generation.

[0089] The second approach: When the application scenario of the first audio information is an all-weather scenario (recording scenario), the electronic device generates extended information corresponding to the first audio information based on the text segment sequence for each text segment sequence acquired; wherein, the text segment sequence includes one or more text segments.

[0090] In this way, event-level instant recording can be achieved without waiting for subsequent audio transcription sequences.

[0091] By designing differentiated extended information triggering and generation strategies for different application scenarios, the meeting scenario can support real-time preview of text fragments and incremental generation of extended information, while the all-weather scenario can support event-level instant output. This not only meets the time-bound scenarios, such as the integrity requirements of meetings, but also adapts to the real-time requirements of time-unbound scenarios, such as long-term recording scenarios, achieving unified and flexible processing of application scenarios.

[0092] For example, Figure 7 This is a flowchart illustrating an application scenario-driven triggering process for differentiated extended information, as provided in this embodiment. The process for triggering differentiated extended information driven by the acquisition mode may include the following steps 701 to 705.

[0093] Step 701: The electronic device receives at least one text fragment.

[0094] Step 702: The electronic device determines the application scenario corresponding to at least one text fragment.

[0095] If at least one text fragment corresponds to a meeting scenario, then proceed to step 703; if at least one text fragment corresponds to an all-weather scenario, then proceed to step 705.

[0096] Step 703: The electronic device determines whether the preset marker of the last text segment in at least one text segment is true.

[0097] If the preset flag is true, proceed to step 704; if the preset flag is not true, proceed to step 705.

[0098] Step 704: The electronic device generates / updates the second extended information.

[0099] Step 705: The electronic device determines the second extended information as extended information corresponding to at least one text fragment.

[0100] In some implementations, the electronic device can identify first audio information to obtain at least one text fragment corresponding to the first audio information; perform keyword analysis based on at least one text fragment to obtain event information that the user needs to pay attention to; classify the event information into corresponding event elements according to event type; and display the event elements on the first interface.

[0101] Keyword analysis refers to the process of performing semantic recognition and event discrimination on at least one text fragment to extract event information. For example, keyword analysis may include steps such as text preprocessing, key information identification, event type discrimination, and attention assessment. Text preprocessing involves cleaning, segmenting, and word-segmenting the transcribed text fragment to provide structured text units for subsequent analysis; key information identification involves identifying content with specific semantic weights from the text fragment; event type discrimination involves classifying the identified key information into different event types; and attention assessment involves determining whether the information "requires user attention," filtering out repetitive and useless information, and retaining content that users need to know or act upon.

[0102] Since at least one text fragment contains relatively complete semantic information, keyword analysis based on at least one text fragment can yield relatively accurate event information. Furthermore, categorizing event information into corresponding event elements according to event type provides users with convenient access to targeted information on the first screen.

[0103] For example, Figure 8 This is a structured extended information data model for traceable evidence provided in this embodiment of the disclosure. Figure 8 In the extended information data model, the extended information notes (including extended information identifiers, collection modes, and descriptions) include one or more to-do items, one or more decision items, and a set of text fragment identifiers and a set of audio information identifiers associated with the extended information notes. The evidence for each to-do item in the extended information notes refers to the text fragment associated with the to-do item; the text fragment includes a text fragment identifier, a speaker identifier, and time information. The evidence for each decision item in the extended information notes refers to the set of text fragment identifiers associated with the decision item. The audio information identified by the audio information identifier can be viewed through the audio information identifier in the set of audio information identifiers associated with the extended information.

[0104] For example, Figure 9 This is an entity relationship diagram of extended information, audio information, and text fragments with many-to-many association provided in this embodiment. Figure 9In this context, N text fragments can serve as evidence for M pieces of extended information; one piece of extended information can be associated with N pieces of audio information; N pieces of audio information can be associated with one piece of extended information; one piece of audio information can generate a decision; and N decision-making matters can serve as evidence for M pieces of extended information.

[0105] In this embodiment of the disclosure, the electronic device integrates audio information, identification elements, and pre-summary items into the first interface. With the help of a responsive operation mechanism, it can achieve rapid location of audio content, seamless switching of text fragments or extended information, and precise guidance of key items. This effectively improves the interactivity, efficiency, and user experience of information presentation, reduces the burden of information acquisition, and enhances the prominent presentation of key information, thereby improving the efficiency of users in obtaining information.

[0106] Figure 10 This is a schematic diagram of the structure of an information presentation device provided in an embodiment of this disclosure. The device can be implemented as part or all of an electronic device by software, hardware, or a combination of both. The electronic device can be as described below. Figure 11 The electronic device shown. See also Figure 10 The device includes a first display module 1001, a second display module 1002, and a third display module 1003.

[0107] The first display module 1001 is used to display a first interface, which includes first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information. The second display module 1002 is used to respond to the operation applied to the first identifier element and display the text information corresponding to the first identifier element. The text information is a text fragment or extended information obtained by recognizing the first audio information. The third display module 1003 is used to respond to the operation applied to the target item element and display the item corresponding to the target item element. The item is an event that the user needs to pay attention to, which is summarized based on the first audio information.

[0108] Optionally, the device further includes: The determination module is used to determine the application scenario to which the first audio information belongs; The fourth display module is used to display the classification identifier of the application scenario on the first interface and to classify and display the first audio information under the classification identifier.

[0109] Optionally, the device further includes: The first recognition module is used to recognize the first audio information to obtain at least one text segment corresponding to the first audio information, and extended information corresponding to the first audio information. The extended information includes at least one or more of the title, summary, time and keywords corresponding to the first audio information. An associated storage module is used to associate and store at least one text fragment with extended information and first audio information.

[0110] Optionally, the device further includes: The second recognition module is used to recognize the first audio information to obtain at least one text segment corresponding to the first audio information; The analysis module is used to perform keyword analysis based on at least one text fragment to obtain event information that users need to pay attention to; The categorization module is used to categorize event information into the corresponding event elements according to event type; The fifth display module is used to display item elements on the first interface.

[0111] Optionally, the event type shall include at least one of the following: events for which the user has made a decision, events for which the user needs to handle, and events for which the user needs to pay attention; The item element is used to identify the item type.

[0112] Optionally, the device further includes: The corresponding module is used to respond to user query requests; The third identification module is used to identify keywords in the query request and search the extended information set based on the keywords. The first generation module is used to generate target query results based on the first information set that matches the keyword when the query obtains the first information set. The second generation module is used to query the text fragment set based on the keyword to obtain the second information set when no first information set matching the keyword is found; and to generate the target query result based on the second information set. The sixth display module is used to display the target query results.

[0113] Optionally, the first identification module is used for: Based on the target voiceprint recognition of the first audio information, at least one text segment corresponding to the first audio information is obtained, and the target voiceprint is the user's voiceprint. Generate extended information corresponding to the first audio information based on at least one text fragment.

[0114] In this embodiment, a first interface is displayed, including first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information. In response to an operation applied to the first identifier element, text information corresponding to the first identifier element is displayed. The text information is a text fragment or extended information obtained by recognizing the first audio information. In response to an operation applied to a target item element, the item corresponding to the target item element is displayed. The item is an event that the user needs to focus on, summarized based on the first audio information. By integrating audio information, identifier elements, and pre-summarized item elements in the first interface, and in conjunction with a responsive operation mechanism, rapid location of audio content, seamless switching of text fragments or extended information, and precise guidance on key items can be achieved. This effectively improves the interactivity, efficiency, and user experience of information presentation, reduces the burden of information acquisition, and enhances the prominent presentation of key information, thereby improving the efficiency of information acquisition for users.

[0115] It should be noted that the information presentation device provided in the above embodiments is only illustrated by the division of the above functional modules when generating and displaying information. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0116] The functional modules in the above embodiments can be integrated into one processing unit, or each functional module can exist as a separate physical processing unit, or two or more functional modules can be integrated into one processing unit. The processing unit can be implemented in hardware or software. Furthermore, the specific names of the functional modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this disclosure.

[0117] The information presentation device and information presentation method embodiments provided in the above embodiments belong to the same concept. The specific working process and technical effects of the functional modules in the above embodiments can be found in the method embodiment section, and will not be repeated here.

[0118] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 11 As shown, the electronic device 110 includes: a processor 1100, a memory 1101, and a computer program 1102 stored in the memory 1101 and executable on the processor 1100. When the processor 1100 executes the computer program 1102, it implements the steps in the information presentation method in the above embodiments.

[0119] Electronic device 110 can be a general-purpose computer device or a special-purpose computer device. In specific implementations, electronic device 110 can be smart glasses or a smartwatch. Those skilled in the art will understand that... Figure 11 This is merely an example of electronic device 110 and does not constitute a limitation on electronic device 110. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0120] Processor 1100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0121] In some embodiments, memory 1101 may be an internal storage unit of electronic device 110, such as a hard disk or memory of electronic device 110. In other embodiments, memory 1101 may be an external storage device of electronic device 110, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 1100. Furthermore, memory 1101 may include both internal and external storage units of electronic device 110. Memory 1101 is used to store operating system, applications, boot loader, data, and other programs. Memory 1101 may also be used to temporarily store data that has been output or will be output.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0123] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0124] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0125] This disclosure provides a computer program product that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above method embodiments of this disclosure can be implemented by a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a computer device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices. The computer-readable storage medium mentioned in this disclosure can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0127] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented in whole or in part as a computer program product. A computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0129] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this disclosure according to actual needs.

[0130] The embodiments described above are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. An information presentation method, characterized in that, The method includes: The first interface is displayed, which includes first audio information, a first identifier element of the first audio information, and at least one item element associated with the first audio information. In response to an operation performed on the first identifier element, text information corresponding to the first identifier element is displayed, wherein the text information is a text fragment or extended information obtained by recognizing the first audio information; In response to an operation performed on a target item element, the item corresponding to the target item element is displayed, the item being an event that the user needs to pay attention to, summarized based on the first audio information.

2. The method according to claim 1, characterized in that, The method further includes: Determine the application scenario to which the first audio information belongs; The first interface displays the classification identifier of the application scenario and displays the first audio information under the classification identifier.

3. The method according to claim 1, characterized in that, The method further includes: Identify the first audio information to obtain at least one text segment corresponding to the first audio information, and extended information corresponding to the first audio information. The extended information includes at least one or more of the title, summary, time and keywords corresponding to the first audio information. The at least one text fragment is associated with and stored in conjunction with the extended information and the first audio information.

4. The method according to claim 1, characterized in that, The method further includes: Identify the first audio information to obtain at least one text segment corresponding to the first audio information; Based on the at least one text fragment, keyword analysis is performed to obtain event information that users need to pay attention to; The event information is categorized into the corresponding event elements according to the event type; The item element is displayed on the first interface.

5. The method according to claim 4, characterized in that, The event type includes at least one of the following: events for which the user has made a decision, events for which the user needs to handle, and events for which the user needs to pay attention; The item element is used to identify the item type.

6. The method according to claim 1, characterized in that, The method further includes: Responding to the user's query request; Identify keywords in the query request, and query the extended information set based on the keywords; When a first set of information matching the keyword is obtained, the target query result is generated based on the first set of information. If no first set of information matching the keyword is found, a second set of information is obtained by querying the text fragment set based on the keyword; and a target query result is generated based on the second set of information. Display the results of the target query.

7. The method according to claim 3, characterized in that, The step of identifying the first audio information to obtain at least one text segment corresponding to the first audio information, and extended information corresponding to the first audio information, includes: Based on the target voiceprint recognition of the first audio information, at least one text segment corresponding to the first audio information is obtained, wherein the target voiceprint is the user's voiceprint; Based on the at least one text fragment, generate extended information corresponding to the first audio information.

8. An electronic device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device performs the method as described in any one of claims 1 to 7.

Images